Method for setting illumination in a fluorescence microscope and corresponding fluorescence microscope

Abstract

The invention relates to a method for automatically determining a to-be-set illumination brightness (P k ) of at least two light sources (120 k ) for exciting at least one fluorophore (130 j ) in a sample (110) to be imaged in a fluorescence microscope (100), wherein each of the at least two light sources (120 k ) is individually controllable with respect to its illumination brightness (P k ) and at least two detectors (140 i ) detect an image intensity (I i ) of the microscopically imaged sample (110) each, wherein the to-be-set illumination brightness (P k ) of the at least two light sources (120 k ) is automatically determined such that a predetermined nominal value of a signal-to-noise ratio per fluorophore (130 j ) is achieved, wherein, for determining the illumination brightness (P k ) of the at least two light sources (120 k ), a crosstalk of the detectors for different emission spectra of the fluorophore (130 j ) and / or a cross-excitation of the fluorophore (130 k ) for different illumination spectra of the light sources (120 j ) is taken into account. The invention also relates to a corresponding fluorescence microscope (100).

Description

TECHNICAL FIELD

[0001] The present application relates to a method for automatically determining a setting of an illumination intensity of at least two light sources for exciting at least one fluorophore in a sample to be imaged in a fluorescence microscope. BACKGROUND

[0002] In a fluorescence microscope, a sample dyed with a fluorophore is microscopically imaged. A fluorophore is a pigment that is excited by the frequency of an excitation light, which in turn emits radiation. Usually, for each fluorophore, a light source of a suitable wavelength or a light source from which a suitable wavelength can be filtered out by a filter is required. The fluorescence emitted by the sample is received by a suitable detector. Here, usually, the fluorescence radiation of each fluorophore is detected separately. This can be achieved either by a single detector with a correspondingly narrow band of sensitivity or by a broadband detector with a pre-filter that only allows the wavelength of the relevant fluorescence radiation to pass. Further details on the structure and mode of operation of a fluorescence microscope can be found in the relevant literature. Fluorescence microscopes are used, inter alia, for examining, in particular, living cells.

[0003] US 10,200,625 B2 discloses a system and a method for imaging a biological sample in a culture medium. First, an image of the sample is taken at a preset value. Then the system determines the saturated pixels and the signal-to-noise ratio pixel by pixel. If the number of saturated pixels is above a certain threshold or the signal-to-noise ratio of the relevant pixels is below a predetermined threshold, a new image is taken, for which a new value of the photon flow and / or the exposure time is set. This method is repeated until either the unsaturated pixels reach a predefined signal-to-noise ratio threshold or until a predefined maximum duration of image taking has elapsed. In the document, the photon flow refers to the light intensity (“Light Intensity”). It indicates the number of photons per unit area and time that reach the sensor of the camera. The exposure time (“Exposure Time”) is the time during which the sensor of the camera accumulates the signal.

[0004] With the aid of this method or other known methods, the user can now set the illumination intensity of the light sources present in the fluorescence microscope for exciting the relevant fluorophores. It has been shown that this method, in particular for unskilled users, is complex and time-consuming to design and usually does not result in the desired high image quality.

[0005] There is therefore a need to set microscope parameters, for example the illumination intensity of the excitation light sources in a fluorescence microscope, in a user-friendly manner, in particular in an automated manner. SUMMARY

[0006] The inventors have found that one of the reasons for the difficulties in optimally setting the microscope parameters in a fluorescence microscope is the cross-talk in the different detection channels when probing different dyes / fluorophores and the cross-excitation of different dyes / fluorophores by different light sources. In the following, the cross-talk in the different detection channels will also be referred to as "cross-emission" (which can also be referred to as "Cross-Talk" of the detectors) and the cross-excitation of fluorophores by different light sources will be referred to as "cross-excitation".

[0007] Embodiments according to the concept of the present application comprise a method for automatically determining the illumination intensity to be set for at least two light sources for exciting at least one fluorophore in a sample to be imaged in a fluorescence microscope, wherein each of the at least two light sources is individually controllable with respect to its illumination intensity and wherein at least two detectors detect the image intensity of the microscopically imaged sample, respectively, wherein the illumination intensity to be set for the at least two light sources is automatically determined such that a predetermined nominal value of the signal-to-noise ratio per fluorophore is achieved, wherein for determining the illumination intensity of the at least two light sources the cross-talk of the detectors for different emission spectra of the fluorophores and / or the cross-excitation of the fluorophores for different illumination spectra of the light sources is taken into account.

[0008] In the concept according to the present application, the term "light source" comprises any light-emitting device suitable for exciting a fluorophore (also referred to as dye). For exciting a fluorophore, a predetermined excitation wavelength has to be present in the spectrum of the light source. Accordingly, a "light source" according to the present application can comprise a broadband light source comprising the excitation wavelength, a narrowband light source comprising the excitation wavelength, or a light source having a filter connected downstream which filters from the spectrum of the light source a spectrum comprising the excitation wavelength. Correspondingly, such a "detector" according to the present application. The detector has to be able to detect the wavelength of the emitted fluorescent radiation of the fluorophore in question. For this purpose, a correspondingly broadband detector or a correspondingly narrowband detector can be used which has sufficient sensitivity for the respective wavelength of the fluorescent radiation. Furthermore, a broadband detector with a pre-positioned filter can be used, wherein the filter filters the relevant wavelength of the fluorescent radiation and makes the sensor of the detector accessible to this wavelength. The above-mentioned filter downstream of the light source or pre-positioned to the detector can be configured as a filter sheet or filter plate, designed as a spectral separation layer, a beam splitting device or a monochromator device, but also, for example, in the form of an acousto-optic or liquid-crystal-based system.

[0009] Each light source according to the application is set for directly exciting fluorophores, here the user assumes that fluorophores are present in the sample to be imaged. Each of these light sources can be set individually in terms of its illumination intensity. Thus, the flow of photons present on the directly assigned fluorophores can be set and thus also the flow of photons resulting from the fluorescent radiation ultimately emitted onto the detector. According to the idea of the application, it is taken into account in this setting that a particular light source, due to its emitted spectrum, not only excites the fluorophores directly assigned to it, but also (to some extent) other fluorophores present in the sample. As is explained further below, this cross-excitation can be taken into account by determining this cross-excitation during the process of setting the illumination intensity.

[0010] Alternatively or additionally, for determining the illumination intensity of at least two light sources according to the application, it is taken into account that a detector according to the application assigned to a particular fluorophore also (to some extent) detects fluorescent radiation originating from other fluorophores based on its sensitive spectrum. As explained below, by determining the degree of this crosstalk during the setting of the illumination intensity of the light sources, this "cross-emission" can be taken into account.

[0011] In summary, therefore, the illumination intensity of the light sources according to the idea of the application is set such that a predetermined nominal value of the signal-to-noise ratio per fluorophore is achieved taking into account the "cross-emission" and / or "cross-excitation". Thus, according to the idea of the application, the time-consuming and complex setting of the microscope parameters in a fluorescence microscope is reduced as far as possible for the user. The implementation of the idea by means of a fast converging algorithm also minimizes the number of images to be acquired and, therefore, the light load of the sample sensitive to bleaching and the phototoxicity in live cell microscopy.

[0012] It should be noted that the term "determination" is intended to include both "calculation" and "experimental determination" as well as hybrids thereof. If a mathematical model is too complex and thus estimation leads to faster achievement of the goal, this can also include "estimation". It can also refer to "estimation" in the sense of a statistical estimate.

[0013] In a fluorescence microscope, for example, work can be carried out with different parameters, the target values of which can be predetermined by the user and / or by the system and finally can be set by the system itself. Possible parameters are the "speed", i.e. the speed of imaging, which is influenced, inter alia, by the setting of the exposure time. Furthermore, a possible parameter is the "image quality", which is significantly influenced by the signal-to-noise ratio of the image. Here, the number of detected photons or generated photoelectrons plays a major role, wherein the photons or photoelectrons can be assigned to a fluorophore. Since the photons follow Poisson statistics, the signal-to-noise ratio is essentially proportional to the square root of the detected photoelectrons assignable to a fluorophore and thus to the square root of the detected image intensity per fluorophore. Another possible parameter is the "sample load". A symbol of the sample load imposed by the irradiation of the sample to be examined is the bleaching.

[0014] A particularly advantageous design according to the concept of the application is to optimize the image quality at a constant imaging speed and to use the sample load as an interruption criterion. Further details on this are given in the following description. The optimization of the image quality should be discussed in detail first.

[0015] It is advantageous to determine the signal-to-noise ratio per fluorophore from at least one image intensity per fluorophore detected by at least one detector, wherein it is more advantageous to introduce as many, in particular all, image intensities per fluorophore detected by the detectors due to the crosstalk of the detectors. In other words, the detected image intensities per fluorophore are thus accumulated, in particular on the existing detectors.

[0016] It is furthermore particularly advantageous to take into account the cross-excitation of the fluorophores when determining the signal-to-noise ratio per fluorophore by also basing the image intensity per fluorophore detected by one of the detectors on the excitation of the respective fluorophore by the existing different light sources instead of or in addition to the crosstalk of the detectors. Thus, not only the excitation light source directly assigned to the fluorophore in question is taken into account, but also the spectrum of the remaining light sources. In other words, the detected image intensities per fluorophore are accumulated on the existing light sources.

[0017] The advantageous design options according to the concept of the application will be described mathematically below.

[0018] The system comprises K light sources k, wherein k = 0,..., K-1, I detectors i, wherein i = 0,..., I-1, and J fluorophores j, wherein j = 0,..., J-1. The number of channels l is then the number of all possible connections of light sources and detectors L = K*I, wherein l = 0,..., L-1. The intensity I of the pixel of channel l is then l For

[0019]

[0020] where the exposure time is τ, the mixing matrix is M lj and the fluorophore concentration of the J fluorophores is c j The size of the mixing matrix is determined by the cross-excitation and cross-emission as

[0021] M lj =∫Em j (λ)·Sens i(l) (λ)dλ·∫Exc j (λ')·Ill l (λ')dλ' (2)

[0022] where the emission spectrum of the fluorophore j is Em j (λ), and the excitation spectrum of the fluorophore j is Exc j (λ'), and the spectral sensitivity of the detector i assigned to the channel l is Sens i(l) (λ), and the illumination spectrum of the channel l is Ill l (λ'). Now, the illumination spectrum can be composed by superimposing K light sources with a single luminance value, which have all the individual spectra IllLED k (λ') and are controlled in the channel l with the value P kl Then, the accumulated illumination spectrum is

[0023]

[0024] Furthermore, the continuous mixing matrix

[0025]

[0026] is calculated such that the intensity of the pixel is

[0027]

[0028] The simultaneous detection of the illumination should be realized, that is, only in channels that can be read simultaneously. The number of channels is given by the number of detectors, since a detector can only read all channels that arrive at the detector simultaneously, one can identify the channel l with the detector i, and the symbol can be abbreviated as l ≡I i and Since here only the simultaneous illumination should be concerned (sequential illumination is always possible and included, so of course combinations of channels l that cannot be read simultaneously can also be concerned - the matching of the model is simple, but the understandability of the symbol is complicated here), the index l can be omitted at P kl and P k ≡Pkl It can be calculated. Therefore, equation (5) becomes

[0029]

[0030] Therefore, the cumulative detected intensity per fluorophore (on all detectors) is

[0031]

[0032] Therefore, according to equation (7), the illumination luminance P k It should be selected such that the image intensity I measured per fluorophore j Compared with the preset value I j 额定 Correspondingly, the preset value I j 额定 Corresponding to the desired photoelectrons per pixel, the desired photoelectrons per pixel are predetermined based on the desired signal-to-noise ratio. As can be seen, accumulation is made in Equation (7) on the number of k-sources and also on the number of i-detectors, such that in this example, not only detector crosstalk but also fluorophore cross-excitation is considered. In principle, only one of these two effects can also be considered; in this case, accumulation is made only on the number of k-sources or only on the number of i-detectors.

[0033] According to the concept of the invention, the optimal selection of illumination brightness for all light sources can be performed jointly, and especially when simultaneously reading existing detectors. Furthermore, the concept of the invention can also be implemented with different light sources set sequentially and / or with different detectors read sequentially, wherein the determined values ​​of illumination brightness are simply applied sequentially to the light sources. It can also be a hybrid form of sequential illumination with different illumination brightness levels superimposed.

[0034] In an advantageous optimization method, to determine the illumination brightness of at least two light sources, an initial value for the illumination brightness is first predetermined for each light source, and the associated image intensity of each fluorophore is measured, and the corresponding associated signal-to-noise ratio is calculated. Then, in an iterative method, the value of the illumination brightness is progressively changed until a corresponding predetermined nominal value for the signal-to-noise ratio of each fluorophore is reached. Changing the illumination brightness in this iterative method includes increasing or decreasing the illumination brightness. It should be noted that in this optimization method, the initial value of the brightness can also be 0 or can be changed to be set to 0.

[0035] In an advantageous design, the image enhancement of the at least two detectors is taken into account in determining the illumination intensity of the at least two light sources. To this end, the predetermined image enhancement of the detectors is taken into account in determining the illumination intensity, or conversely, a suitable image enhancement of the detectors is output or set in determining the illumination intensity. The same applies to the exposure times of the individual detectors. These exposure times can in particular be determined and kept constant. The above-mentioned imaging parameter "speed" thus corresponds to a determined, predetermined value. The value of the exposure time can in turn be determined depending on the image enhancement of the detectors. Conversely, the image enhancement of the detectors can also be determined depending on the predetermined exposure time. It generally applies that the image enhancement can be selected lower when the exposure time is higher, and conversely, the exposure time can be selected lower when the image enhancement is higher. In general, these two settings determine the imaging parameter "speed". In particular, the selection of the image enhancement can be made dependent on the dynamic range of the analog-digital conversion of the detectors and the expected photoelectron based on a predetermined signal-to-noise ratio value.

[0036] As mentioned above, a third possible parameter of the imaging according to the concept of the application is the "sample load". Since the imaging is usually carried out on a living sample, care is taken that no thermal or photochemical damage occurs due to too high radiation intensity. A measure of the sample load of a sample dyed with a fluorophore is the bleaching of the dye.

[0037] According to a further aspect, in particular as the above-mentioned design according to the concept of the application, but also independently thereof, it is provided that a bleaching coefficient per fluorophore is determined during the determination of the illumination intensity of the at least two light sources. Note that this aspect can be independent of the first-mentioned aspect of taking into account "cross-excitation" and "cross-emission", and thus it relates to an aspect to be protected on its own. Thus, within the scope of the present disclosure, this second aspect relates to a method for automatically determining a set value of an illumination intensity of at least two light sources for exciting at least one fluorophore in a sample to be imaged in a fluorescence microscope, wherein each of the at least two light sources is individually controllable with respect to its illumination intensity, and wherein at least two detectors detect an image intensity of the microscopically imaged sample, respectively, wherein the set value of the illumination intensity of the at least two light sources is automatically determined such that a predetermined nominal value of a signal-to-noise ratio per fluorophore is achieved, wherein a bleaching coefficient per fluorophore is determined during the determination of the illumination intensity of the at least two light sources. In an iterative method for optimizing the set value of the illumination intensity (with or without taking into account cross-excitation and cross-emission), the bleaching coefficient per fluorophore can in particular be determined (at least approximately) mathematically. From the thus determined bleaching coefficient, the concentration c jThe bleaching kinetics are known. When the concentration falls to a predetermined minimum value, the "maximum bleaching value" is reached. In the known bleaching kinetics, the minimum value is assigned a certain duration of time for reaching the maximum bleaching value. A possible bleaching kinetics is based, for example, on an exponential decrease in the concentration of the fluorophore over time. The maximum bleaching value can then be calculated from the bleaching coefficient, the determined illumination intensity and the detected intensity per fluorophore.

[0038] On the basis of the relationships described, an interruption criterion can be established for the imaging of the sample with respect to the respective fluorophore on the basis of the determined maximum bleaching value or the duration of time corresponding thereto. Such an interruption criterion can provide, for example, that the illumination intensity of the respective light source is reduced at the time of reaching the maximum bleaching value or the assigned duration of time or correspondingly before or earlier, wherein the image enhancement of the respective detector can optionally be increased. For the fluorophore of interest, it is often sufficient to reduce the directly assigned excitation light source, wherein the image enhancement and / or the exposure time of the respective directly assigned detector can be increased in order to compensate for the reduction in image brightness. In practice, however, excitation light sources of very short wavelength, in particular, also bleach other fluorophores, the excitation of which also lies in the red spectral range ("Stokes shift"). In this case, it can also be advantageous to reduce the illumination intensity of these light sources. When cross-excitation is taken into account, there is a matrix of the intensities of the individual light sources on the excitation of the individual fluorophores, and the matrix of the bleaching coefficients of the individual fluorophores can be determined experimentally individually or in the course of the signal-to-noise ratio setting, so that one can here very specifically intervene.

[0039] With the illumination intensities thus reset, the signal-to-noise ratio per fluorophore and the new maximum bleaching value can be calculated again. In addition, the interruption criterion can provide that, upon reaching another maximum bleaching value, the illumination intensity of the respective excitation light source or of the respective associated light source is reduced to 0 in order to prevent damage to the sample.

[0040] In the concept according to the application, it is advantageous to determine fluorophores and / or light sources which do not affect the detected image intensity during the determination of the illumination intensity. This can be the case, for example, when a certain fluorophore is not present in the sample or in the selected field of view despite a contrary indication by the user and / or the system, or when the light source set for this reason or for other reasons has no effect on the detected image intensity.

[0041] Furthermore, it is particularly advantageous that, in order to determine the illumination brightness of at least two light sources as a result of objective lens replacement in fluorescence microscopy, the change in the detected image intensity due to the objective lens replacement is considered by taking into account the imaging relationship in the sample, which involves the geometric flow of the illumination light and the geometric flow of the fluorescence light imaged from the system onto the detector element. This specifically involves the illumination pupil, which depends on the system illumination, the magnification of the emission field aperture, the detector cone of the objective lens, and the size of the detector element imaged onto the sample.

[0042] The following is a brief mathematical description of possible optimization methods for optimally determining the illumination brightness of at least two light sources.

[0043] For example, optimization can be achieved using an iterative method based on the Newton-Raphson-Methode approach:

[0044] 1. Measure the initial lighting settings. strength This image was taken with (good) starting values ​​for illumination. Calculated from instrument spectral and fluorophore data. In the simplest case, the instrument spectrum is known from the design process or calibration measurements, just as fluorophore data are known through calibration data and are available, for example, in a database. However, matrix terms can also be determined or improved through iterative methods (“on the fly”), as is common in quasi-Newton methods such as BFSS. Target values ​​are generated, for example, by Reuleaux-Control as explained below. And if possible, generate data related to pigments.

[0045] 2. Estimate using appropriate methods This could be linear unmixing, which is known in principle from professional literature, or phasor unmixing.

[0046] 3. Calculate the cumulative intensity of each fluorophore.

[0047] 4. Estimate the bleaching factor for each pigment.

[0048] 5. By differentiating equation (7) as follows, calculate the Jacobian matrix of the cumulative intensity of each fluorophore in relation to the illumination setting.

[0049]

[0050] 6. Check the Jacobian matrix in order to determine invalid light sources and missing fluorophores. If a row is empty, this means that the corresponding fluorophore is not in the image. If a column is empty, the corresponding light source has no influence on the number of detected photoelectrons.

[0051] 7. Compute new illumination settings as Newton step

[0052]

[0053] Here, the matrix inversion can be the Moore-Penrose-Pseudoinverse.

[0054] 8. Set to positive values restricted to the control region. If remove light source k from the solution and set

[0055] 9. Check interruption criteria

[0056] (a) is smaller than an admissible value. Set success.

[0057] (b) is smaller than an admissible value. Set success.

[0058] (c) The Jacobian matrix is not invertible. Report error, explain reason. The reason for the non-invertibility has already been found by removing empty rows and columns in step 6. In order to explain the reason, an error report can be shown to the user.

[0059] (d) Bleaching is too strong.

[0060] (e) Maximum number of iterations reached.

[0061] 10. Perform new image acquisition and return to step 2.

[0062] The idea according to the invention also relates to a computing unit, which is set up to perform the method according to the above-described embodiments of the idea according to the invention. In this embodiment, the method can be performed completely automatically based on the computing unit. Thus, the corresponding microscope parameters can be determined in the highest degree of user-friendliness. The determined values can be displayed and / or set immediately on the corresponding elements of the fluorescence microscope.

[0063] In another embodiment, the idea according to the invention relates to a computer program with program code, which, when executed on a processor, in particular on the mentioned computing unit, serves to perform the method according to the above-described idea according to the invention.

[0064] Finally, the idea according to the invention relates to a fluorescence microscope having at least two light sources for exciting at least one fluorophore in a sample imaged by means of the fluorescence microscope, wherein each of the at least two light sources is individually controllable with respect to its illumination intensity, the fluorescence microscope further having at least two detectors for detecting the image intensity of the microscopically imaged sample, respectively, and having a computing unit which is set up to carry out the above-mentioned method according to the idea of the invention in order to automatically determine the illumination intensity to be set for the at least two light sources.

[0065] When determining the signal-to-noise ratio per fluorophore from the detected image intensity per fluorophore, it is advantageous for the computing unit to be in communication with the detectors of the fluorescence microscope.

[0066] Furthermore, it is advantageous for the computing unit to be in communicative connection with means for automatically setting the illumination intensity of the at least two light sources in order to set the illumination intensity to a predetermined nominal value of the signal-to-noise ratio per fluorophore, wherein, for determining the illumination intensity of the at least two light sources, the cross-talk of the detectors for the different emission spectra of the fluorophores and / or the cross-excitation of the fluorophores for the different illumination spectra of the light sources is taken into account.

[0067] Furthermore, the embodiments of the method according to the invention are similarly applicable to the fluorescence microscope according to the invention and its design variants and the advantages resulting therefrom.

[0068] The above-mentioned, so-called Luer element provides the possibility of inputting the three mentioned parameters "speed", "image quality" and "sample protection". Here, generally a polygonal form of the input face is involved, wherein the coordinate origin can be an angular point or an edge midpoint between each two adjacent angular points. Preferred is a circular-arc polygon, i.e. a polygon in which the edges are not straight lines, but circular arcs around opposite angular points. Of the three parameters considered here - without limiting generality - this element is designed, in particular, as a circular-arc triangle or Luer triangle. In the Luer triangle, the distance of each point of the edge from the opposite vertex is constant.

[0069] The selection of qualitative parameters, such as "speed", "image quality" and "sample protection" allows an intuitive setting without the user having to know or understand the technical background. The Lecher triangle presents the three desires in two dimensions and thereby indicates that these desires cannot be fulfilled at the same time. According to the idea of the present application, "image quality" is now quantified in the sense of the signal-to-noise ratio and "speed" is given by the exposure time (and / or gain) used. The number of photons detected strongly depends on the sample and the fluorophore used as well as the labeling density, so that this observability is optimized to reach the desired value. The exposure time is here preferably kept fixed and the illumination is adapted. The size of "sample protection" or "sample-protected imaging" is considered as an interruption criterion in order not to exceed a certain sample load.

[0070] The predetermination of the signal-to-noise ratio can be directly made by the user. But it is also possible that the user limits the signal-to-noise ratio to a predetermined range. The predetermined range can for example be generated by priority information or parameterized, for example considering an otherwise known bleaching sensitivity of the fluorophore, a predetermined exposure time or other criteria. Generally, the predetermination of the signal-to-noise ratio always contains a compromise with the exposure time and the bleaching performance, in the scope of which the first should be maximized.

[0071] Thus, at a predetermined "speed" (i.e. here for example the exposure time), the user has a selection of "image quality", wherein the sample load, which is dynamically changed during the illumination of the sample, must be kept below a limit value at the same time. Furthermore, when there are fluorophores that are particularly sensitive to bleaching, the system itself can reduce the target-SNR-value and / or make other adjustments. Further embodiments with respect to the Lecher triangle can be derived from the respective examples, which is advantageously implemented as a GUI ("graphical user interface") for operating a fluorescence microscope.

[0072] Further advantages and design solutions of the present application result from the description and the drawings.

[0073] It is to be understood that the features mentioned above and those to be explained below can be used not only in the respective combinations indicated, but also in other combinations or in isolation without departing from the scope of the present application.

[0074] The present application is schematically shown in the drawings by means of examples and described below with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 A fluorescence microscope according to an embodiment according to the present application is schematically shown,

[0076] Figure 2This demonstrates the possibility of parameter setting using Rellow control, and

[0077] Figure 3 An embodiment of the method flow according to the present invention is shown. Detailed Implementation

[0078] Figure 1 The diagram schematically illustrates K light sources 120 for emitting K different excitation wavelengths. k The fluorescence microscope 100, k = 0, ..., K⁻¹. Advantageously, these light sources are LEDs or lasers with corresponding spectra, wherein filters can be connected downstream if needed. In principle, broadband light sources can also be used, and the desired excitation wavelengths can be sequentially filtered out from the spectrum of the broadband light source by means of filters or filter plates. However, one of the main advantages of the concept according to the invention is that, in the presence of multiple light sources, the illumination brightness of these light sources can be set simultaneously, so that sequential operation or sequential setting is not required.

[0079] like Figure 1 As shown, each light source is 120. k An illumination beam 164 is emitted and guided to the objective lens 160 of the microscope 100 via a spectral separation element 166. The spectral separation element 166 is a dichroic element that deflects the relevant excitation wavelength and is transparent to the corresponding fluorescence radiation. Details of the optics of the fluorescence microscope are described in... Figure 1 The image is shown only schematically, as its details are well known from the prior art. Illumination path 164 is guided onto sample 110 via objective lens 160 and excites fluorophores 130j, j = 0, ..., J⁻¹ located therein for emitting fluorescent radiation. The emitted fluorescent radiation is imaged onto a corresponding detector 140 via objective lens 160 and possibly other optical imaging elements. i Above, i = 0, ..., I-1. For each light source emitting an excitation wavelength of 120 nm in the sense of this application. k Distribute the corresponding fluorophores 130 j fluorophore 130 j It emits fluorescent radiation from that direction, and the fluorescent radiation is then detected by a corresponding detector 140. i Detection. Therefore, I detectors are assigned accordingly to J distinguishable fluorophores. The detectors can be—similar to light sources—multiple individual detectors, with filters connected in front of these individual detectors, especially since the inventive concept allows for simultaneous reading of I detectors. However, in principle, one or more broadband detectors can also be operated using appropriate filters or plates, and thus the detectors can be operated entirely or partially sequentially. The detection light path falling on the detector is marked with 162.

[0080] existFigure 1 The invention exemplarily illustrates a fluorescence microscope; however, it should be understood that the concept of the invention can also be implemented in other fluorescence microscope systems, such as sheet microscopes, confocal microscopes, multiphoton microscopes, etc.

[0081] For each light source 120 k For example, each light source is 120. k Lighting brightness P k ("Illumination power") and therefore the emitted power can be set individually. In particular, the concept according to the invention enables the simultaneous setting of the illumination brightness P of all K light sources. k Therefore, in the embodiment observed here, a calculation unit 150 is provided, which is used to set the illumination brightness P of the light source. k The means are in a communication connection or action connection. Detector 140 i The distribution of corresponding fluorophores in the images captured of sample 110 is detected, and thus the image intensity I is detected respectively. i Computing unit 150 and detector 140 i It is in a communication or operational connection so as to be able to receive signals corresponding to the intensity of each image. According to the concept of the present invention, I detectors can be read simultaneously.

[0082] Now, the lighting brightness P to be set k The calculation unit 150 automatically determines a predetermined signal-to-noise ratio (SNR) value for each distinguishable fluorophore, taking into account crosstalk to the detector caused by other emission spectra of the fluorophore not directly assigned to that detector, and cross-excitation of the fluorophore by other illumination spectra of the light source not directly assigned to that fluorophore. Here, the SNR value for each fluorophore is the square root of the cumulative intensity detected by each fluorophore on all detectors according to equation (7).

[0083] In this embodiment, the predetermined signal-to-noise ratio (SNR) per fluorophore is achieved using the Reuleaux-Dreieck triangle. j 额定 The pre-determined, such as Figure 2 As shown. Figure 2 The Reuleaux triangle shown is advantageously presented to the user of the fluorescence microscope 100 as a GUI ("Graphical User Interface") for operating the microscope. At the vertices of the arc-shaped triangle are the origins of three parameters: "Speed," "Image Quality," and "Sample Protection." The values ​​of these parameters are... Figure 3The values ​​1, 2, or 3 are used to represent these values ​​and are located on arcs equidistant from their respective origins. In the example shown, the parameter "Speed" is fixedly predetermined by a predetermined value of 1 for the exposure time τ and / or a predetermined value for the gain. The parameter "Sample Protection" is labeled with a minimum value of 3, indicating that the assigned bleaching value must always be less than or equal to the maximum bleaching value. In other words, the assigned value for the parameter "Sample Protection" is not allowed to be lower than 3. Therefore, for the parameter "Image Quality," only values ​​located on the upper arc segment d2 remain. Value 2 is the maximum possible value here. It should be noted that the value 3 of the parameter "Sample Protection" can be dynamically changed during imaging by decreasing the region d2.

[0084] Therefore, the Reuleaux triangle constructed for the GUI allows users to intuitively input the desired nominal value of the image quality, where the input is located in region d2 (in... Figure 2 The value 2 (shown in the figure) derives the nominal signal-to-noise ratio. If the system knows which fluorophores in sample 110 are particularly sensitive to bleaching, the system can automatically select a lower value 2 for image quality. The system and the calculation unit 150 determine the signal-to-noise ratio (SNR) per fluorophore based on the predetermined value 2. j The cumulative detected intensity value is used as the illumination luminance P to be determined. k The rated value.

[0085] Next, for example, using the iterative method described above, the optimal illumination settings for the K light sources are determined through n steps, starting with the initial illumination settings. In each step, the image intensity I of each fluorophore is also determined or measured. j (Accumulated on the detector). Iterative steps 2 through 8 explained above are performed until one of the interruption criteria mentioned in point 9 is met. These include: the difference between the measured image intensity of each fluorophore and the predetermined nominal value is less than a permissible value. Furthermore: the difference between the illumination brightness to be newly set for the light source and the illumination brightness set in the previous step is less than a permissible value. Furthermore: the Jacobi-Matrix given in equation (8) is irreversible. This results in an error and advantageously leads to the output of possible error causes to the user. Furthermore: a predetermined number of iterations has been performed. Finally: the sample is bleached too much; in other words, the value range d2 shrinks during the dynamic development of the value 3 of the parameter “sample protection”, making it necessary to reduce the illumination brightness to eliminate damage to the sample. In this case, another interruption criterion can be introduced, wherein the corresponding illumination brightness of the relevant light source is reduced, and advantageously, the gain of the relevant detector is increased, or at least the gain of the detector directly detecting the fluorophore involved is increased.

[0086] Figure 3An embodiment of the inventive concept for optimizing the illumination brightness is illustrated in the form of a flow chart.

[0087] Figure 3 An overview of an embodiment of the inventive concept for setting the illumination brightness of K existing light sources 120 k in a fluorescence microscope 100 is shown, which is used for examining a sample 110 having J distinguishable fluorophores 130 j , wherein each fluorophore is detected by one of I detectors 140 i in the form of an image of the sample. Reference is made for this to the above-described embodiments in connection with Figure 1 and Figure 2 In step S1, the calculation unit 150 (see Figure 1 ) is pre-provided, in particular by the user, with an initial value for the illumination brightness, for example according to a manufacturer's specification or a learned algorithm. The coupling matrix M is calculated from the light spectrum of the light source, the detection spectrum of the detector and the fluorophore data. The target value for the image intensity I j per fluorophore accumulated on the detector is generated, for example from the mentioned Loo input (see Figure 2 ) and / or by corresponding system preset parameters (data relating to the pigments). Furthermore, the above-mentioned interruption criteria are defined.

[0088] After the initialization step S1, the system begins the image acquisition of the individual detector images, which each detect the image intensity I i of the microimaged sample 110 for each fluorophore.

[0089] In step S3, the concentration c j of the individual fluorophores 130 j in the sample 110 is estimated and the accumulated brightness I j is calculated according to equation (7). This results from the fluorophore concentration c j , the set exposure time τ, the accordingly set illumination brightness P k and the mentioned coupling matrix M. Details for this have been set out above.

[0090] In the mentioned step S3, it is also advantageous to calculate the bleaching factor κ j . As already emphasized above, this is an aspect which can be used in addition to but also independently of the "cross excitation" or "cross emission" in order to determine the optimum illumination brightness. This can be done by comparing the detected c jwhich are evaluated according to a mathematical model. Thus, depending on the construction of the model, for example from two successive image acquisitions and the illumination intensity associated therewith, the c j relative to each other or from N successive image acquisitions and the illumination intensity parameterize an N-2. order model. Linear models are of particular interest, which can be parameterized by means of three successive image acquisitions and the illumination intensity by means of common mathematical methods. After the final setting of the illumination intensity, the specific bleaching kinetics of each fluorophore can be calculated from the corresponding determined bleaching coefficients, from which in turn the maximum bleaching value can be determined, the parameter "sample protection" being based on this maximum bleaching value.

[0091] In a subsequent step S4, the Jacobian matrix of the accumulated intensities of each fluorophore is calculated according to equation (8). Subsequently, in step S5 it is checked whether the Jacobian matrix has empty rows (the respective fluorophore is not in the image) or empty columns (the respective light source has no influence on the number of detected photoelectrons). If this is the case, the respective row (fluorophore) or column (light source) is eliminated, that is to say, they are no longer taken into account further in order to simplify the calculation effort. If step S5 does not result in a result, step S7 is continued, in which a new setting of the illumination intensity is carried out according to equation (9).

[0092] Then, in step S8, the proposed interruption criteria are checked, as has been explained in detail above. If the interruption criteria are not met, the next step n+1 of the iterative method is entered, that is to say, a return is made to step S2 according to Figure 3 the flowchart of equation (8). If one of the interruption criteria is met, the main image acquisition is started in step S9 with the determined and set illumination intensity P k of the light sources 120 k .

[0093] The method proposed here for setting the mentioned parameters in a fluorescence microscope is very user-friendly, since the method works intuitively and does not require technical knowledge and the respective settings are made automatically. The method also presents a technical parameter space of the adjustment parameters based on the parameters important to the user of a biological experiment.

[0094] The term "and / or" includes all combinations of one or more of the associated listed items and can be abbreviated as " / ".

[0095] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or apparatus corresponds to a method step or a function of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or corresponding apparatus for performing the method steps.

[0096] Some embodiments relate to a microscope comprising a system, as described in connection with one or more of the accompanying drawings. Alternatively, the microscope can be part of the system or connected to the system. Figure 1 A schematic of a system configured for performing the methods described herein is shown. The system comprises a microscope 100 and a computer system or computing unit 150. The microscope 100 is configured for taking images and is connected to the computer system 150. The computer system is configured for performing at least part of the methods described herein. The computer system can be configured for performing a machine learning algorithm. The computer system and the microscope can be separate units, but can also be integrated together in a common housing. The computer system can be part of a central processing system of the microscope, and / or the computer system can be part of a subcomponent of the microscope, such as a sensor, actuator, camera or illumination unit of the microscope, etc.

[0097] The computer system can be a local computer device (e.g., a personal computer, a laptop, a tablet computer, or a mobile phone) having one or more processors and one or more storage devices, or can be a distributed computer system (e.g., a cloud computing system having one or more processors or one or more storage devices distributed at different locations, e.g., at a local client and / or one or more remote server farms and / or data centers). The computer system can include any circuit or combination of circuits. In an embodiment, the computer system can include one or more processors, which can be of any type. According to context, by processor can be meant any type of computing circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set microprocessor (CISC), a reduced instruction set microprocessor (RISC), a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multiple core processor, a field programmable gate array (FPGA), such as a microscope or microscope component (e.g., a camera), or any other type of processor or processing circuit. Other types of circuit that can be included in the computer system can be custom circuit, an application-specific integrated circuit (ASIC), etc., such as one or more circuits (e.g., a communication circuit) for a wireless device (e.g., a mobile phone, a tablet computer, a laptop, a radio transceiver, and similar electronic systems). The computer system can include one or more storage devices, which can include one or more storage elements suitable for the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media (such as CD-ROM, flash storage card, DVD, etc.). The computer system can further include a display device, one or more speakers, and a keyboard and / or controller, which can include a mouse, a trackball, a touch screen, voice recognition device, or any other device that allows a system user to input information to and receive information from the computer system.

[0098] Some or all of the method steps can be performed by a hardware device or in a case of using a hardware device, e.g., a hardware device can be, e.g., a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps can be performed by such a device.

[0099] Depending on certain implementation requirements of the inventive methods, embodiments of the application can be implemented in hardware or in software. The implementation can be performed using a non-transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium can be computer readable.

[0100] Some embodiments according to the application comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0101] Generally, embodiments of the present application can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code can for example be stored on a machine readable carrier.

[0102] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0103] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0104] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary. A further embodiment of the inventive method is, therefore, a product (or article of manufacture) comprising a processor and the computer program for performing one of the methods described herein when the computer program runs on the processor.

[0105] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be configured to be transferred via a data communication connection, e.g. a modem, a cellular communication link, PSTN or PSTN infrastructure, or Internet infrastructure.

[0106] A further embodiment comprises a processing means, such as a computer, or a programmable logic device, configured to or adapted for performing one of the methods described herein.

[0107] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0108] Another embodiment according to the application comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a data server.

[0109] In some embodiments, a programmable logic device (for example, a field programmable gate array, FPGA) can be used to perform some or all functions of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor in order to perform one of the methods described herein. Generally, these methods are preferably performed by each hardware device.

[0110] List of reference signs

[0111] 100 fluorescence microscope

[0112] 110 sample

[0113] 120k light source; k = 0,..., K-1

[0114] 130j fluorophore; j = 0,..., J-1

[0115] 140 i detector; i = 0,..., I-1

[0116] 150 calculation unit

[0117] 160 objective

[0118] 162 detection light path

[0119] 164 illumination light path

[0120] 166 spectral separation element

[0121] P k illumination intensity

[0122] c j fluorophore concentration; j = 0,..., J-1

[0123] d2 region of the parameter "image quality"

[0124] S1-S9 method steps

Claims

1. A method for automatically determining an illumination brightness (Pk) to be set for at least two light sources (120k), the at least two light sources (120k) being used to excite at least one fluorophore (130j) in a sample (110) to be imaged in a fluorescence microscope (100). in, Each of at least two light sources (120k) is individually controllable in terms of its illuminance (Pk), and Among them, at least two detectors (140i) detect the image intensity (Ii) of the sample (110) being imaged by microscopy. Specifically, the illumination luminance (Pk) of at least two light sources (120k) is automatically determined to achieve a predetermined rated signal-to-noise ratio per fluorophore (130j). In order to determine the illumination brightness (Pk) of at least two light sources (120k), crosstalk of detectors for different emission spectra of fluorophores (130j) and / or cross-excitation of fluorophores (130j) for different illumination spectra of light sources (120k) are considered. Specifically, based on the excitation of corresponding fluorophores (130j) by at least two light sources (120k), the signal-to-noise ratio of each fluorophore (130j) is determined according to the image intensity (Ij) of each fluorophore (130j) detected by at least one of at least two detectors (140i); and / or, The signal-to-noise ratio of each fluorophore (130j) is determined based on the image intensity (Ij) detected by at least two detectors (140i) for each fluorophore (130j).

2. The method according to claim 1, wherein, To determine the illumination luminance (Pk) of at least two light sources (120k), an initial value (Pk°) of illumination luminance is first predetermined for each light source (120k), and the associated image intensity (Ij°) is measured, and the associated signal-to-noise ratio for each fluorophore is calculated. Then, the value of illumination luminance (Pk) is gradually changed in an iterative method until a predetermined nominal value of the signal-to-noise ratio for each fluorophore is reached.

3. The method according to claim 1, wherein, Image enhancement of at least two detectors (140i) is considered when determining the illumination luminance (Pk) of at least two light sources (120k).

4. The method according to claim 1, wherein, To determine the illumination luminance (Pk) of at least two light sources (120k), the exposure time (τ) for each detector (140i) is considered.

5. The method according to claim 4, wherein, The exposure time (τ) for each detector (140i) remains constant, and / or the exposure time (τ) is determined based on the image enhancement of the detector (140i).

6. The method according to claim 1, wherein, Simultaneously read the detector (140i).

7. The method according to claim 1, wherein, The bleaching factor (κ) per fluorophore (130 J) was determined during the determination of the illumination luminance (Pk) of at least two light sources (120 K). j ).

8. The method according to claim 7, wherein, The determined bleaching coefficient (κ) j Determine the maximum bleaching value and / or the duration until the maximum bleaching value per fluorophore (130j) is reached.

9. The method according to claim 8, wherein, Based on the determined maximum bleaching value, an interruption criterion is established for imaging of sample (110) with respect to the corresponding fluorophore (130j); The interruption standard stipulates that the illumination brightness (Pk) of the corresponding light source (120k) should be reduced.

10. The method according to claim 9, wherein, Improve the image enhancement of the corresponding detector (140i).

11. The method according to claim 1, wherein, During the determination of illumination luminance (Pk), fluorophores (130j) and / or light sources (120k) that do not affect the detected image intensity (Ij) are identified.

12. The method according to claim 1, wherein, To determine the illumination brightness (Pk) of at least two light sources (120k) as a result of objective lens replacement in fluorescence microscope (100), the change in detected image intensity (Ii) due to the objective lens replacement is taken into account.

13. A computing unit (150) configured to perform the method according to any one of claims 1 to 12.

14. A computer program product having program code, wherein when executed on a processor, the computer program is used to perform the method according to any one of claims 1 to 12.

15. A fluorescence microscope (100), comprising: At least two light sources (120k) are used to excite at least one fluorophore (130j) in a sample (110) to be imaged by means of a fluorescence microscope (100), wherein each of the at least two light sources (120k) is individually controllable with respect to its illumination intensity (Pk). At least two detectors (140i) are used to detect the image intensity (Ij) of the sample (110) being imaged under a microscope, and The computing unit (150) according to claim 13 is used to determine the illumination brightness (Pk) to be set so as to achieve a predetermined rated value of the signal-to-noise ratio per fluorophore (130j), wherein, in order to determine the illumination brightness (Pk) of at least two light sources (120k), crosstalk of detectors for different emission spectra of fluorophores (130j) and / or cross-excitation of fluorophores (130j) for different illumination spectra of light sources (120k) are taken into account.

16. The fluorescence microscope (100) according to claim 15, comprising: A means for setting the illumination luminance (Pk) of at least two light sources (120k). The computing unit (150) is in communication connection with at least two detectors (140i) and means for setting the illumination brightness (Pk) so as to set the determined illumination brightness (Pk) of at least two light sources (120k).

Images