Method for adjusting illumination in a fluorescence microscope and corresponding fluorescence microscope

Abstract

The present invention provides a method for detecting at least one fluorophore (130) in a sample (110) to be imaged in a fluorescence microscope (100). j At least two light sources (120 k ) to be adjusted. k ) is a method for automatically calculating the light intensity of at least two light sources (120 k ) is the illumination luminance (P k ) can be individually driven and controlled, and at least two detectors (140 i ) is the respective image intensity (I i ) and detect at least two light sources (120 k ) to be adjusted. k ) with phosphor (130 j ) is automatically calculated to achieve the target signal-to-noise ratio for each light source (120 k ) lighting brightness (P k ) for each phosphor (130 j ) and / or the crosstalk of the detector due to the different emission spectra of each light source (120 k ) for different illumination spectra of phosphor (130 j ) and a corresponding fluorescence microscope (100) in which cross-excitation is taken into account.

Description

[Technical field]

[0001] The invention relates to a method for automatically calculating the adjusted illumination intensity of at least two light sources which each excite at least one fluorophore in a sample to be imaged in a fluorescence microscope, as well as to a corresponding fluorescence microscope. [Background technology]

[0002] In fluorescence microscopy, a sample stained with fluorophores is imaged by microscopy. Fluorophores are dyes that are excited by an excitation light frequency and emit radiation themselves. Typically, for each fluorophore, a light source of the appropriate wavelength or a light source from whose emission spectrum the appropriate wavelengths are filtered is required. The fluorescence emitted by the sample is imaged by a suitable detector. Here, the fluorescence radiation of each fluorophore is typically detected separately. Again, this can be done by individual detectors with a correspondingly narrowband sensitivity or by a broadband detector connected upstream with a filter that passes only the wavelength of the fluorescence radiation in question. For further details regarding the structure and function of the fluorescence microscope, the relevant literature is referred to. Fluorescence microscopes are used in particular, in particular for examining living cells.

[0003] From US 10200625 a system and method are known for imaging biological samples in culture medium. First, a sample image is taken with pre-adjusted values. The system then calculates the saturated pixels and the signal-to-noise ratio for each pixel. If the number of saturated pixels exceeds a pre-defined threshold or the signal-to-noise ratio for the corresponding pixel falls below a pre-defined threshold, a new image is taken, during which new values ​​of the photon flow and / or the exposure time are adjusted. The method is repeated until a pre-defined threshold of the signal-to-noise ratio for the unsaturated pixels is reached or the set maximum imaging time has elapsed. Photon flow is referred to in the above-mentioned document as "Light Intensity", which represents the number of photons that reach the camera sensor per unit area and unit time. "Exposure Time" is the time during which the camera sensor integrates the signal.

[0004] Using this or other known methods, the user can then adjust the illumination intensity of an existing light source that excites the corresponding fluorophores in the fluorescence microscope, which methods prove to be laborious and time-consuming, especially for untrained users, and often do not result in the desired high image quality.

[0005] There is therefore a need for a user-friendly, and in particular automatic, progression of adjustment of microscope parameters in a fluorescence microscope, such as the illumination intensity of an excitation light source. Summary of the Invention [Problem to be solved by the invention]

[0006] The inventors have found that one of the causes of the difficulty in optimally adjusting the microscope parameters in fluorescence microscopy is the existence of crosstalk when detecting multiple dyes / fluorophores in various detection channels and cross-excitation of multiple dyes / fluorophores by various light sources. In the following, crosstalk in various detection channels is also referred to as "cross-emission" (sometimes also referred to as "cross-talk" of detectors), and cross-excitation of fluorophores by various light sources is also referred to as "cross-excitation". [Means for solving the problem]

[0007] An embodiment of the concept of the present invention includes a method for automatically calculating the illumination brightness to be adjusted of at least two light sources which each excite at least one fluorophore in a sample to be imaged in a fluorescence microscope, in which each of the at least two light sources can be individually driven and controlled in terms of its illumination brightness, at least two detectors detect respective image intensities of the sample to be imaged by microscopy, and the illumination brightness to be adjusted of the at least two light sources is automatically calculated such that a set target value of the signal-to-noise ratio for each fluorophore is achieved, and for calculating the illumination brightness of the at least two light sources, cross-talk of the detectors for different emission spectra of each fluorophore and / or cross-excitation of fluorophores for different illumination spectra of each light source are taken into account.

[0008] In this concept of the invention, the term "light source" includes any light-emitting device suitable for exciting fluorophores, also called dyes. In order to excite a fluorophore, it is necessary that a certain excitation wavelength is present in the spectrum of the light source. Thus, the "light source" in this application can include a broadband light source including the excitation wavelength, a narrowband light source including the excitation wavelength, or a light source with a filter connected downstream that filters the spectrum of the light source including the excitation wavelength. The same applies in reverse to the "detector" in this application. The detector must be able to detect the wavelength of the fluorescent radiation emitted by the relevant fluorophores. For this purpose, a corresponding broadband detector can be used, or a corresponding narrowband detector with sufficient sensitivity for the corresponding wavelength of the fluorescent radiation can be used. Furthermore, a broadband detector with a filter connected upstream can be used, which filters the relevant wavelength of the fluorescent radiation to reach the sensor of the detector. The above-mentioned filters connected downstream of the light source or upstream of the detector can be configured as filter wheels or filter sliders, as spectral separation layers, spectrometer devices or monochromator devices, or for example in the form of an acousto-optical system or a liquid crystal-based system.

[0009] Each light source according to the present application is provided for the direct excitation of a fluorophore that the user assumes to be present in the sample to be imaged. Each of the light sources is individually adjustable in terms of its illumination brightness. Thus, the photon flow impinging on the directly assigned fluorophore and thus the photon flow resulting from the fluorescence emission and finally impinging on the detector can be adjusted. According to the inventive concept, during such adjustment, it is taken into account that a given light source not only excites the fluorophore directly assigned to it based on the emitted spectrum, but also excites (to a given extent) other fluorophores present in the sample. Furthermore, as will be explained below, such cross-excitation can be taken into account by being calculated during the adjustment process of the illumination brightness.

[0010] Alternatively or additionally, in order to calculate the illumination brightness of at least two light sources according to the present application, it is taken into account that a detector associated with a given phosphor according to the present application also detects (to a given extent) the fluorescent radiation of another phosphor based on its sensitivity spectrum. Such "cross-emission" can be taken into account by calculating the degree of crosstalk here during adjustment of the illumination brightness of the light sources, as described below.

[0011] In summary, therefore, the inventive concept adjusts the illumination intensity of the light source in such a way that a set target value of the signal-to-noise ratio for each fluorophore is achieved, taking into account "cross-emission" and / or "cross-excitation". The inventive concept therefore makes it unnecessary for the user to make extensive and complex adjustments of the microscope parameters in fluorescence microscopy. The realization of the concept with a fast-converging algorithm minimizes the number of images to be taken and thus also the light load of samples sensitive to bleaching as well as phototoxicity in the microscopy of living cells.

[0012] It should be noted that the term "estimating" includes "calculating" and "experimentally determining" as well as mixtures thereof. It can also include "estimating" when the mathematical model is so complex that an estimation can arrive at the goal more quickly. "Estimating" in the sense of a statistical estimation is also contemplated.

[0013] In fluorescence microscopy, for example, it is possible to work with different parameters whose target values ​​are specified by the user and / or the system and which are ultimately adjustable by the system itself. A possible parameter is the "speed", i.e. the imaging speed, which is controlled in particular by adjusting the exposure time. In addition, there is the "image quality", which is decisively controlled by the signal-to-noise ratio of the image. The main role here is played by the number of detected photons or the number of generated photoelectrons, which can be assigned to one fluorophore. Since the photons follow Poisson statistics, the signal-to-noise ratio is practically proportional to the square root of the detected photoelectrons, which can be assigned to a fluorophore, and thus to the square root of the image intensity per detected fluorophore. Another possible parameter is the "specimen load". An indicator of the specimen load due to the illumination of the specimen to be examined is the bleaching.

[0014] A particularly advantageous approach for the inventive concept is to optimize image quality when the imaging speed is constant and to utilize sample load as a cutoff criterion. Further embodiments are described in the following description. First, the optimization of image quality is discussed in detail.

[0015] It is advantageous to calculate the signal-to-noise ratio per fluorophore as a function of the image intensity per at least one fluorophore detected by at least one of the detectors, whereby it is further advantageous to utilize as many image intensities per fluorophore as possible, in particular all image intensities, detected by the detectors, depending on the crosstalk present in the detectors, so that, in other words, the image intensities per particularly detected fluorophore are integrated over all present detectors.

[0016] Furthermore, it is particularly advantageous that, instead of or better still in addition to the above-mentioned crosstalk of the detectors, the cross-excitation of the fluorophores is taken into account when calculating the signal-to-noise ratio for each fluorophore, by calculating the image intensity for each fluorophore detected by one of the detectors based on the excitation of each fluorophore by the various light sources present. In this way, not only the excitation light source directly associated with the fluorophore in question is taken into account, but also the spectrum of the remaining light sources. In other words, the image intensity for each detected fluorophore is integrated over all the light sources present.

[0017] In the following, an advantageous implementation of the concept according to the invention is mathematically explained.

[0018] The system includes K light sources k [k=0,...,K-1], I detectors i [i=0,...,I-1], and J fluorophores j [j=0,...,J-1]. The number of channels l, i.e., the number of all possible connections between light sources and detectors, is L=K * l[l=0,…,L-1], where the intensity I of the pixel in channel l l teeth,

number

number

number

number

number

[0019] Simultaneous detection of illumination should be realized, i.e. the number of channels that can be read out simultaneously is limited. Note that the number of channels is determined by the number of detectors, since a detector can only read out all channels that reach it at the same time, so we can identify channel l with detector i, and therefore change the notation to I l ≡I i and M. i(l)jk ≡M ijk Here, simultaneous illuminations should be observed (sequential illuminations are always possible and are included here, and of course combinations of channels l that cannot be read out simultaneously can also be observed - this makes it easier to adapt the model, but in this case complicates the understanding of the notation), so P kl Remove the index l in P k ≡P kl We can put it as follows: M ijk This makes it possible to calculate Equation (5) as follows:

number

[0020] In this case, the integral of the detected intensity per fluorophore (across all detectors) is

number

[0021] Therefore, according to equation (7), the measured image intensity I j is the set value I, which corresponds to the desired photoelectrons per pixel, which is set based on the desired signal-to-noise ratio j Soll The lighting luminance P k should be selected. As is evident, in equation (7) the integration is done over all k light sources and over all i detectors, so that in this embodiment both detector crosstalk and fluorophore cross-excitation are taken into account. In principle it is also possible to only take into account one of the two effects. In this case one can integrate only over all k light sources or only over all i detectors.

[0022] According to the concept of the invention, the selection of the optimal illumination intensity for all light sources can be performed jointly, in particular during simultaneous readout of the detectors present. However, the concept of the invention can also be realized during sequential adjustment of the various light sources and / or during sequential readout of the various detectors, in which case the calculated illumination intensity values ​​can simply be applied to the light sources sequentially. A mixture of sequential illumination and different superpositions of illumination intensities is also possible.

[0023] In an advantageous optimization method, the illumination brightness of at least two light sources is calculated by first setting an initial value of the illumination brightness for each of the light sources, measuring the image intensity for each corresponding fluorophore, and calculating the corresponding signal-to-noise ratio, and then, in an iterative process, the illumination brightness value is successively changed until the set target value of the signal-to-noise ratio for each fluorophore is achieved. Changing the illumination brightness in such an iterative process includes increasing the illumination brightness or also includes decreasing the illumination brightness. It is noted that in such an optimization method, the initial value of the brightness may be 0, but may also be changed to set it to 0.

[0024] In an advantageous embodiment, the image amplification of at least two detectors is taken into account when calculating the illumination brightness of the at least two light sources. For this purpose, the set image amplification of the detector is taken as the basis for calculating the illumination brightness, or conversely, the appropriate image amplification of the detector is output or adjusted when calculating the illumination brightness. The same applies analogously to the exposure time of each detector. These can in particular be set to a fixed value and can be kept constant. The above-mentioned parameter "speed" of image formation corresponds to a fixed set value. The value of the exposure time can also be determined depending on the image amplification of the detector. Conversely, the image amplification of the detector can also be determined depending on the set exposure time. It is generally considered that a small image amplification can be selected for a long exposure time and, conversely, a short exposure time can be selected for a large image amplification. In general, these two adjustments determine the parameter "speed" of image formation. In particular, the selection of the image amplification can be made depending on the dynamic range of the analog-to-digital conversion of the detector and the photoelectrons that are expected based on the setting of the signal-to-noise ratio.

[0025] A third possible parameter of imaging according to the concept according to the invention is the "sample load" as mentioned above. Since imaging is generally performed on biological samples, care must be taken to avoid thermal or photochemical damage due to excessively high radiation intensities. A measure of the sample load of a sample colored by a fluorophore is the bleaching of the dye.

[0026] According to another aspect, particularly as a configuration of the above-mentioned concept of the invention, and independent of the above-mentioned concept, a bleaching coefficient for each fluorophore is determined during the calculation of the illumination intensity of at least two light sources. It should be noted here that such an aspect may be independent of the aspect mentioned at the beginning considering "cross-excitation" and "cross-emission" and is therefore an aspect to be protected independently. Thus, within the scope of the present disclosure, this second aspect relates to a method for automatically calculating the illumination intensity to be adjusted of at least two light sources exciting at least one fluorophore in a sample to be imaged in a fluorescence microscope, each of the at least two light sources being individually controllable with respect to its illumination intensity, at least two detectors detecting respective image intensities of the sample to be imaged by microscopy, automatically calculating the illumination intensity to be adjusted of the at least two light sources such that a set target value of the signal-to-noise ratio for each fluorophore is achieved, and determining the illumination coefficient for each fluorophore during the calculation of the illumination intensity of the at least two light sources. In particular, in an iterative process for optimizing the adjustment value of the illumination brightness (with or without taking into account cross-excitation and cross-emission), the bleaching coefficient for each phosphor can be mathematically determined (at least approximately). From the bleaching coefficients thus determined, the respective phosphor concentrations c j The fading rate for a given concentration can be shown. The "maximum fading value" is obtained when the concentration drops to a predefined minimum value. With a known fading rate, the minimum value is associated with a predefined period of time until the maximum fading value is reached. A possible fading rate starts, for example, from a time-exponential drop in the fluorophore concentration. In this case, the maximum fading value can be calculated from the fading coefficient, the calculated illumination luminance and the intensity for each detected fluorophore.

[0027] Based on the above mentioned relationship, the calculated maximum bleaching value or the time period corresponding to this maximum bleaching value creates an interruption criterion for the imaging of the sample associated with the corresponding fluorophore. Such an interruption criterion can be provided in such a way that, for example, when the maximum bleaching value or the time period corresponding to this maximum bleaching value is reached or correspondingly in advance or earlier, the illumination intensity of the corresponding light source can be reduced and, as an option, the image amplification of the corresponding detector can be increased. For the observed fluorophore, this is usually sufficient to reduce the directly associated excitation light source, in which case the image amplification and / or the exposure time of the directly associated corresponding detector can be increased to compensate for the image brightness reduction. However, in practice, especially with short-wave excitation light sources, other fluorophores whose excitation is further in the red spectral region also bleach ("Stokes shift"). In this case, a reduction in the illumination intensity of such light sources can also be advantageous. Taking into account cross-excitation, an effect matrix of the intensities of the individual light sources exists for the excitation of the individual fluorophores, and the effect matrix for their bleaching coefficients can be determined experimentally either separately or in the course of adjusting the signal-to-noise ratio, which can be done quite as intended.

[0028] With the thus newly adjusted illumination intensity, the signal-to-noise ratio and the new maximum bleaching value for each fluorophore can be calculated again. An interruption criterion can further be provided such that if a further maximum bleaching value is reached, the illumination intensity of the corresponding excitation light source or the corresponding associated light source is reduced to 0 in order to prevent damage to the sample.

[0029] In the present concept according to the invention, it is preferred to determine fluorophores that do not affect the detected image intensity and / or light sources that do not affect the detected image intensity during the calculation of the illumination brightness. This may be the case, for example, if a certain fluorophore is not present in the sample or in the selected field of view, despite instructions to the contrary from the user and / or the system, or if for this or other reasons a certain light source has no effect on the detected image intensity.

[0030] It is further particularly advantageous to take into account the change in the image intensity detected as a result of the objective change in the fluorescence microscope in order to calculate the illumination brightness of the at least two light sources as a result of the objective change by considering the imaging ratio into the sample, which relates to the geometrical flux of the illumination light and the geometrical flux of the fluorescence imaged from the system onto the detector element, which in particular relates to the system-dependent illumination intensity of the illumination pupil, the magnification of the field stop, the detection cone of the objective and the size of the detector element imaged into the sample.

[0031] In the following, a brief mathematical description will be given of a possible optimization method for optimally calculating the illumination brightness of at least two light sources.

[0032] The optimization is performed by an iterative process, for example by the Newton-Raphson method, i.e. 1.Initial lighting adjustment

number

number

number

number

number

number

number

number

number

number

number

number

[0033] The concept according to the invention further relates to a computing unit configured to execute the method according to the above-mentioned embodiment of the concept according to the invention. The method can be executed in this embodiment fully automatically on the computing unit. Thus, the corresponding microscope parameters can be calculated in a highly user-friendly manner. The calculated values ​​can be displayed on corresponding elements of the fluorescence microscope and / or can be adjusted in the same way.

[0034] In yet another embodiment, the inventive concept relates to a computer program comprising a program code for performing the method according to the inventive concept as described above, when the program is executed on a processor, in particular on a computing unit as described above.

[0035] Finally, the concept according to the present invention relates to a fluorescence microscope comprising at least two light sources which excite at least one fluorophore in a sample to be imaged using the fluorescence microscope, each of the at least two light sources being individually controllable in terms of its illumination brightness, and further comprising at least two detectors which detect the respective image intensities of the sample to be imaged by microscopy, and a computing unit configured to execute the above-mentioned method according to the concept of the present invention in order to automatically calculate the illumination brightness to be adjusted of the at least two light sources.

[0036] If the signal-to-noise ratio for each fluorophore is calculated depending on the image intensity for each detected fluorophore, it is preferred if the calculation unit is communicatively connected to a detector of the fluorescence microscope.

[0037] Furthermore, it is preferred that the calculation unit is communicatively connected to a means for automatically adjusting the illumination intensity of the at least two light sources in order to adjust the illumination intensity so that a set target value of the signal-to-noise ratio for each fluorophore is achieved, wherein in order to calculate the illumination intensity of the at least two light sources, the cross-talk of the detector for the different emission spectra of each fluorophore and / or the cross-excitation of the fluorophore for the different illumination spectra of each light source are taken into account.

[0038] Otherwise, the method-related embodiments of the present invention apply equally to the fluorescence microscope of the present invention as well as to the configurations thereof and advantages arising therefrom.

[0039] The means for inputting the three above mentioned parameters "speed", "image quality" and "sample protection" are provided by the so-called Reuleaux element already mentioned. Here, the input surface generally takes the form of a polygon, the coordinate origin of which can be a vertex or a side midpoint between two adjacent vertices, respectively. Preferably, it is a curvilinear polygon, i.e. a polygon in which each side is not a straight line but a circular arc centered on the opposite vertex. For the three parameters considered here, this element is in particular (without limiting its generality) constructed as a curvilinear or Reuleaux triangle. In the case of a Reuleaux triangle, the distance from the opposite vertices to each point of a side is constant.

[0040] The selection of qualitative parameters like "speed", "image quality" and "specimen protection" allows intuitive adjustments without the user having to know or understand the technical background. The Reuleaux triangle expresses three demands in two dimensions, but here it is shown that these demands cannot be met simultaneously. According to this concept, "image quality" is quantified in terms of the signal-to-noise ratio, while "speed" is given by the exposure time (and / or amplification) used. Since the number of detected photons strongly depends on the specimen and on the fluorophores and marking densities used, the observable is optimized to achieve the desired value. In this case, the exposure time is preferably known and the illumination is adapted. The "specimen protection" or "specimen-friendly imaging" variables are taken into account as interruption criteria to ensure that a certain specimen load is not exceeded.

[0041] The setting of the signal-to-noise ratio can be done directly by the user. However, it is also possible for the user to define the signal-to-noise ratio within a set range. Such a set range can be generated or parameterized from a priori information, for example the bleaching sensitivity of a fluorophore known in other ways, a set exposure time or taking into account another criterion. In general, the setting of the signal-to-noise ratio always involves a compromise between the exposure time and the bleaching behavior which should primarily be maximized in the frame of this exposure time.

[0042] Thus, the user has a predefined choice of a set "speed", i.e. here for example "image quality" at a set exposure time, while at the same time the dynamically changing sample load during illumination of the sample must remain below a limit value. Additionally, the system can automatically reduce the target SNR value, particularly in the presence of fluorophores that are highly sensitive to bleaching, and / or perform other adaptations. Further embodiments of the Reuleaux triangle, which is advantageously realized as a GUI ("Graphical User Interface") for operating a fluorescence microscope, can be seen from the corresponding examples.

[0043] Further advantages and embodiments of the present invention will become apparent from the following description and accompanying drawings.

[0044] It is evident that the features mentioned above and those to be described hereinafter can be used not only in the respective combinations mentioned, but also in other combinations, or alone, without departing from the scope of the invention.

[0045] The invention is illustrated diagrammatically by way of an embodiment in the drawings and is explained further below with reference to these drawings. [Brief description of the drawings]

[0046] [Figure 1] FIG. 1 illustrates a schematic diagram of a fluorescence microscope according to an embodiment of the present invention. [Diagram 2] FIG. 13 is a diagram showing a parameter adjustment means using Reuleaux control. [Diagram 3] FIG. 2 illustrates an embodiment of a method flow of a method according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0047] FIG. 1 illustrates K light sources 120 emitting K different excitation wavelengths. kA fluorescence microscope 100 with [k=0,...,K-1] is shown very diagrammatically. Advantageously, these light sources are LEDs or lasers with corresponding spectra, where filters can be connected downstream if necessary. Basically, broadband light sources can also be used, from whose spectra the desired excitation wavelengths can be filtered out in sequence by means of a filter wheel or filter slider. However, one of the great advantages of the concept according to the invention is that in the case of multiple light sources, they can be adjusted simultaneously in terms of their respective illumination brightness, so that sequential operation or sequential adjustment is not necessary.

[0048] As shown in FIG. k 1 emits an illumination beam path 164, which is guided via a spectral separation element 166 to the objective lens 160 of the microscope 100. The spectral separation element 166 is a dichroic element which deflects the relevant excitation wavelengths and is transparent to the corresponding fluorescence radiation. The details of the optical system of a fluorescence microscope are known in sufficient detail from the prior art and are therefore only shown very diagrammatically in FIG. 1. The illumination beam path 164 is deflected via the objective lens 160 towards the sample 110 and deflects the fluorophores 130 present in the sample. j [j=0,...,J-1] are excited to emit fluorescent radiation. The emitted fluorescent radiation is imaged onto a corresponding detector 140i [i=0,...,I-1] via an objective lens 160 and possibly further optical imaging elements. Each light source 120 emitting an excitation wavelength is referred to as a k The corresponding phosphor 130 j are associated with each phosphor 130 j emits fluorescent radiation which is then detected by a corresponding detector 140 i. I detectors are therefore respectively assigned to the J distinguishable fluorophores. In particular, since the inventive concept allows for simultaneous readout of the I detectors, the detectors (like the light sources) can be several individual detectors, with filters connected upstream of these individual detectors. In principle, however, it is also possible to operate one or several broadband detectors with corresponding filter wheels or filter sliders, so that the detectors are operated entirely or partially sequentially. The detection beam path entering the detector is designated with the reference number 162.

[0049] Although FIG. 1 illustrates an epifluorescence microscope by way of example, it should be understood that the concepts of the present invention can also be implemented in other fluorescence microscope systems, such as light sheet microscopes, confocal microscopes, multiphoton microscopes, etc.

[0050] Lighting luminance P k ("Lighting Output") and thus each light source 120 k The power emitted from each light source 120 k In particular, the concept according to the invention can be adjusted individually for the illumination luminance P k For this purpose, in the embodiment considered here, the illumination brightness P k A calculation unit 150 is provided that is communicatively connected to or operatively connected to the means for adjusting the detector 140. i represents the distribution of each phosphor in the captured image of the sample 110, and thus the respective image intensities I i The calculation unit 150 calculates the intensity of the light emitted by the detector 140 so as to be able to capture signals corresponding to the individual image intensities. i or a detector 140 i The inventive concept allows for a simultaneous readout of I detectors.

[0051] Here, the illumination luminance P to be adjusted so that the set target values ​​of the signal-to-noise ratio for the different phosphors are adjusted.k is automatically calculated by the calculation unit 150, taking into account the crosstalk of the detectors due to other emission spectra of fluorophores that do not correspond directly and the cross-excitation of fluorophores by other illumination spectra of the light source that do not correspond directly. The measure for the signal-to-noise ratio per fluorophore is now the square root of the intensity per fluorophore detected by integration over all detectors according to equation (7):

[0052] As shown in Figure 2, the signal-to-noise ratio setting I j Soll The setting of is performed in this embodiment by means of a Reuleaux triangle. The Reuleaux triangle shown in FIG. 2 is advantageously presented to the user of the fluorescence microscope 100 as a GUI ("Graphical User Interface") for operating the microscope. At each vertex of the arc-shaped triangle there is a coordinate origin for the three parameters "speed", "image quality" and "specimen protection". The values ​​of these parameters are indicated in FIG. 2 with the reference characters 1, 2, 3 and lie on arcs equidistant from the respective coordinate origins. In the illustrated example, the parameter "speed" is fixed by a setting value 1 of the "exposure time" τ and / or a predefined value of the "gain". Here, the parameter "specimen protection" is filled in with a minimum value of 3, which indicates that the corresponding bleaching value must always be less than or equal to the maximum bleaching value. In other words, the corresponding value of the parameter "specimen protection" must not fall below the value 3. Therefore, for the value of the parameter "image quality", only the values ​​lying on the upper arc portion d2 remain. Here, the value 2 is the maximum possible value. It should be noted that the value 3 of the parameter "sample protection" can be changed dynamically so that the area d2 decreases during the imaging process.

[0053] The Reuleaux triangle configured as a GUI thus allows the user to intuitively input the desired target value of the image quality, whereby from the input value 2 (maximum value 2 is shown in FIG. 2) lying in the region d2, the target value of the signal-to-noise ratio is derived. The system is itself able to select from the lower values ​​2 for the image quality, especially if fluorophores in the sample 110 that are sensitive to bleaching are known to the system. The system and the calculation unit 150 thus derives from the set value 2 the target value of the detected fluorophores I by integration. j The corresponding value of the intensity for each is calculated, and this value is the lighting luminance P k is used as a target value for

[0054] Then, starting from the initial illumination adjustment, optimal illumination adjustments for the K light sources are calculated through n steps, for example by the iterative process already described above. At each step, the image intensities I j is also calculated or measured (integrated over all detectors). The above mentioned iterative steps 2 to 8 are carried out until one of the interruption criteria listed in point 9 is met. This includes that the measured image intensity per fluorophore differs from the set target value by less than a tolerance value. Furthermore, it also includes that the difference between the newly adjusted illumination intensity of the light source and the illumination intensity adjusted in the previous step is smaller than a tolerance value. Furthermore, the Jacobian matrix shown in formula (8) is not invertible. In this case, an error message is formed and preferably possible causes of the error are output to the user. A predefined number of iterations has then been achieved. Finally, the bleaching of the specimen was too strong. In other words, the value range d2 can be reduced by the dynamic evolution of the value 3 of the parameter "specimen protection", so that the illumination intensity must be reduced in order to prevent damage to the specimen. In this case, a different interruption criterion can also be used, in which the corresponding illumination intensity or intensities of the relevant light sources are reduced and, preferably in parallel, the amplification of the involved detector or at least the amplification of the detector which directly detects the corresponding fluorophore is increased.

[0055] In FIG. 3 an embodiment of the inventive concept for optimizing illumination brightness is shown in the form of a flow chart.

[0056] FIG. 3 shows J different phosphors 130 j In order to inspect a sample 110 including k 1 shows an overview of an embodiment of the inventive concept for adjusting the illumination intensity of I detectors 140, where each phosphor is i in the form of a sample image by means of the light source spectrum, see the above description in relation to Figs. 1 and 2. In step S1, in particular the calculation unit 150 (see Fig. 1) is set partly by the system and partly by the user: an initial value for the illumination brightness is set, for example based on a manufacturer setting or a learning algorithm; a combination matrix M is calculated from the light source spectrum, the detection spectrum of the detector and the fluorophore data; the image intensities I per fluorophore are calculated, j A target value of is generated, integrated over the entire detector, for example from the above-mentioned Reuleaux input (see FIG. 2) and / or by corresponding system settings (dye-specific data). Furthermore, the above-mentioned interruption criterion is defined.

[0057] After an initialization step S1, the system calculates the respective image intensity I i Start image capture of individual detector images detecting each fluorophore.

[0058] In step S3, each phosphor 130 in the sample 110 j Concentration of c j is estimated, and the integral luminance I j is calculated according to equation (7), which is the phosphor concentration c j , the adjusted exposure time τ, and the adjusted illumination brightness P k , and the above-mentioned coupling matrix M. Details on this have already been given above.

[0059] In the aforementioned step S3, the fading coefficient κ j is also calculated. This is an aspect that can be used in addition to or independently of the "cross-excitation" or "cross-emission" to calculate the optimal illumination brightness, as already emphasized above. This is based on the c j This can be done by comparing the successive images, which are evaluated according to a mathematical model. j Based on the mutually relative measurements, depending on the model configuration, for example from two successive image acquisitions and the corresponding illumination intensities, a model 0 is obtained. An order of magnitude parameterization is possible, or from N sequentially successive image acquisitions and illumination intensities a model of order N-2 is possible. Of particular importance are linear models that can be parameterized using common mathematical methods from three successive image acquisitions and illumination intensities. The bleaching rate for a given fluorophore can then be calculated according to the final adjustment of the illumination intensities from the respective given bleaching coefficients, from which a maximum bleaching value can again be determined, which maximum bleaching value is the basis for the parameter "sample protection".

[0060] In the following step S4, the Jacobian matrix of the integrated intensities per fluorophore is calculated according to equation (8). Then, in step S5, it is checked whether said Jacobian matrix has an empty row (the corresponding fluorophore is not present in the image) or an empty column (the corresponding light source has no influence on the number of detected photoelectrons). If so, the corresponding row (fluorophore) or column (light source) is removed, i.e. is no longer considered in order to facilitate the computational costs. If step S5 does not produce this result, step S7 is followed, in which a new adjustment of the illumination brightness is performed according to equation (9).

[0061] Then, in step S8, the established interruption criteria are checked, as already explained in detail above. If the interruption criteria are not met, the next step n+1 of the iterative process is proceeded to, i.e. the flow chart of FIG. 3 returns to step S2. On the other hand, if any of the interruption criteria are met, the calculated and adjusted light source 120 is adjusted in step S9. k Illumination luminance P k The main image capture starts.

[0062] The method of adjusting parameters in the above-mentioned fluorescence microscope presented herein is extremely user-friendly since it works intuitively, does not require technical prior knowledge, and performs the corresponding adjustments automatically, and it reflects the technical parameter space of the adjustment amounts for the amount of biological experiment that is important to the user.

[0063] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0064] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step, and similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0065] Some embodiments relate to a microscope including a system as described in relation to one or more of the figures. Alternatively, the microscope may be part of the system or may be connected to the system. FIG. 1 shows a schematic diagram of a system configured to perform the methods described herein. The system includes a microscope 100 and a computer system or computation unit 150. The microscope 100 is configured to image and is connected to the computer system 150. The computer system is configured to perform at least a part of the methods described herein. The computer system may be configured to execute a machine learning algorithm. The computer system and the microscope may be separate entities, but may be integrated in one common housing. The computer system may be part of a central processing system of the microscope and / or the computer system may be part of a subordinate part of the microscope, such as a sensor, actor, camera or lighting unit of the microscope.

[0066] A computer system may be a local computer device (e.g., a personal computer, laptop, tablet computer, or mobile phone) with one or more processors and one or more storage devices, or may be a distributed computer system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, such as local clients and / or one or more remote server farms and / or data centers). A computer system may include any circuit or combination of circuits. In one embodiment, a computer system may include one or more processors, which may be of any type. As used herein, a processor may contemplate any type of computational circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit, for example, of a microscope or a microscope component (e.g., a camera). Other types of circuits that may be included in a computer system may be custom circuits, application specific integrated circuits (ASICs), such as one or more circuits (such as communications circuits) used in wireless devices such as cell phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. A computer system may also include one or more storage devices, which may include one or more memory elements suitable for a particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives handling removable media, such as compact discs (CDs), flash memory cards, digital video discs (DVDs), and the like.The computer system may also include a display device, one or more speakers and a keyboard and / or controller which may include a mouse, a trackball, a touch screen, a voice recognition device, or any other device that enables a user of the system to input information to and receive information from the computer system.

[0067] Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, any one or more of the crucial steps may be performed by such an apparatus.

[0068] Depending on certain implementation requirements, the embodiments of the present invention can be implemented in hardware or software. The implementation can be performed by a non-transitory recording medium, such as a digital recording medium, for example a floppy disk, a DVD, a Blu-ray, a CD, a ROM, a PROM and EPROM, an EEPROM or a FLASH memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to implement the respective methods. Thus, the digital recording medium can be computer readable.

[0069] Some embodiments of the present invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform any of the methods described herein.

[0070] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code which is operable to perform any of the methods when the computer program product is run on a computer, the program code may for example be stored on a machine readable carrier.

[0071] Another embodiment comprises the computer program for performing any of the methods described herein, stored on a machine readable carrier.

[0072] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing any of the methods described herein, when the computer program runs on a computer.

[0073] Therefore, another embodiment of the present invention is a recording medium (or data carrier or computer readable medium) containing a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital recording medium or recording medium is typically tangible and / or non-transitory. Another embodiment of the present invention is an apparatus as described herein, including a processor and a recording medium.

[0074] A further embodiment of the invention is therefore also a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or the sequence of signals being for example adapted to be transmitted via a data communication connection, for example the Internet.

[0075] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.

[0076] Another embodiment comprises a computer having the computer program installed thereon for performing any of the methods described herein.

[0077] Another embodiment of the invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, include a file server to transfer the computer program to the receiver.

[0078] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus. [Explanation of symbols]

[0079] 100 Fluorescence Microscope 110 Samples 120 k Light source;k=0,…,K-1 130 j Phosphor; j=0,…,J-1 140 i detector; i=0,…,I-1 150 Computational Units 160 Objective Lens 162 Detection Beam Path 164 Lighting Beam Path 166 Spectral Separation Element P k Lighting brightness c j Phosphor concentration; j=0,…,J-1 Range for d2 parameter "image quality" S1~S9 Method steps

Claims

1. At least one fluorophore (130) in each sample (110) to be imaged in a fluorescence microscope (100) ｊ At least two light sources (120) for exciting the ｋ ) to be adjusted. ｋ ) A method for automatically calculating The at least two light sources (120 ｋ ) is the illumination luminance (P ｋ ) can be driven and controlled individually, At least two detectors (140 ｉ ) is the image intensity (I) of each one of the samples (110) imaged by microscopy. ｉ ) is detected, The at least two light sources (120 ｋ ) to be adjusted. ｋ ) with phosphor (130 ｊ ) to automatically calculate the signal-to-noise ratio so that a set target value for each signal-to-noise ratio is achieved; The at least two light sources (120 ｋ ) lighting brightness (P ｋ In order to calculate ｊ ) and / or the crosstalk of the detector due to the different emission spectra of each light source (120 ｋ ) phosphors (130) for different illumination spectra ｊ ) considering cross excitation, method.

2. The phosphor (130 ｊ The signal-to-noise ratio for each detector (140) ｉ ) detected by at least one of the ｊ At least one image intensity (I ｊ ) The method of claim 1.

3. The phosphor (130 ｊ ) for each of the at least two light sources (120 ｋ ) for each phosphor (130 ｊ ) based on the excitation of the at least two detectors (140 ｉ ) detected by at least one of the ｊ ) for each image intensity (I ｊ ) The method of claim 2.

4. The phosphor (130 ｊ ) for each of the at least two detectors (140 ｉ ) detected by the fluorescent substance (130 ｊ ) for each image intensity (I ｊ ) The method according to claim 2 or 3.

5. The at least two light sources (120 ｋ ) lighting brightness (P ｋ ) in order to calculate First, the initial value of the illumination luminance (P ｋ 120°) to the light source ｋ ) and the corresponding image intensity (I ｊ ) and calculate the corresponding signal-to-noise ratio for each fluorophore. Then, in an iterative process, the illumination brightness (P ｋ ) is continuously varied until a set target value of the signal-to-noise ratio for each fluorophore is achieved; 5. The method according to any one of claims 2 to 4.

6. The at least two light sources (120 ｋ ) lighting brightness (P ｋ ) when calculating the at least two detectors (140 ｉ ) image amplification degree is taken into account, 6. The method according to any one of claims 1 to 5.

7. The at least two light sources (120 ｋ ) lighting brightness (P ｋ In order to calculate ｉ ) considering the exposure time (τ) for each of 7. The method according to any one of claims 1 to 6.

8. The detector (140 ｉ ) holding the exposure time (τ) constant; The method of claim 7.

9. The exposure time (τ) is set to the detector (140 ｉ ) depending on the image amplification degree, 9. The method according to claim 7 or 8.

10. The detector (140 ｉ ) are read out simultaneously.

10. The method according to any one of claims 1 to 9.

11. The at least two light sources (120 ｋ ) lighting brightness (P ｋ While calculating the phosphor (130 ｊ ) for each fading coefficient (κ ｊ ) to determine 11. The method according to any one of claims 1 to 10.

12. The determined fading coefficient (κ ｊ ) to phosphor (130 ｊ Calculate the maximum color fade value and / or the time to reach the maximum color fade value for each sample; The method of claim 11.

13. Based on the calculated maximum bleaching value, the corresponding phosphor (130 ｊ creating an interruption criterion for imaging the sample (110) related to 13. The method of claim 12.

14. The interruption criterion is the corresponding light source (120 ｋ ) lighting brightness (P ｋ ) is reduced, and optionally the corresponding detector (140 ｉ ) is assumed to have an increased image amplification degree, 14. The method of claim 13.

15. The detected image intensity (I ｊ ) is not affected by the phosphor (130 ｊ ) and / or a light source (120 ｋ ) to the illumination luminance (P ｋ ) is determined while calculating 15. The method according to any one of claims 3 to 14 when claim 2 is recited.

16. The at least two light sources (120 ｋ ) lighting brightness (P ｋ In order to calculate the image intensity (I) detected based on the objective lens change by changing the objective lens in the fluorescence microscope (100), ｉ ) changes, 16. The method according to any one of claims 3 to 15 when claim 2 is recited.

17. The illumination luminance (P k ) is calculated based on a mixing matrix M lj determined from crosstalk and cross excitation; [0010] Em j (λ) is the emission spectrum of phosphor j; E xc j (λ′) is the excitation spectrum of fluorophore j; Sens i(l) (λ) is the spectral sensitivity of detector i(l) associated with channel l; Ill l (λ′) is the illumination spectrum of channel l; 17. The method according to any one of claims 1 to 16.

18. A computing unit (150) configured to perform the method according to any one of claims 1 to 17.

19. 19. A computer program having a program code for performing the method according to one of claims 1 to 17, when the computer program is executed on a processor, in particular on a computing unit (150) according to claim 18.

20. At least two light sources (120 ｋ )and, At least two detectors (140 ｉ )and, A computing unit (150) according to claim 18, A fluorescence microscope (100) comprising: The at least two light sources (120 ｋ ) respectively detects at least one fluorophore (130) in a sample (110) to be imaged using the fluorescence microscope (100). ｊ ), each of said at least two light sources (120 k ) having its illumination brightness (P ｋ ) can be driven and controlled individually, The at least two detectors (140 ｉ ) is the image intensity (I) of each one of the samples (110) imaged by microscopy. ｊ ) is detected, The calculation unit (150) is a phosphor (130 ｊ The illumination brightness (P) should be adjusted so that the set target value of the signal-to-noise ratio for each ｋ ) is calculated, The at least two light sources (120 ｋ ) lighting brightness (P ｋ In order to calculate ｊ ) and / or the crosstalk of the detector due to the different emission spectra of each light source (120 ｋ ) phosphors (130) for different illumination spectra ｊ ) cross excitation is taken into account, Fluorescence microscope (100).

21. The fluorescence microscope (100) includes the at least two light sources (120 ｋ ) lighting brightness (P ｋ ) means for adjusting the The calculation unit (150) is configured to calculate the at least two light sources (120 ｋ ) calculated lighting luminance (P ｋ ) to adjust the at least two detectors (140 ｉ ) and the illumination brightness (P ｋ ) communicatively connected to a means for adjusting 21. The fluorescence microscope (100) of claim 20.

22. The illumination luminance (P k ) is calculated based on a mixing matrix M lj determined from crosstalk and cross excitation; [0025] Em j (λ) is the emission spectrum of phosphor j; E xc j (λ′) is the excitation spectrum of fluorophore j; Sens i(l) (λ) is the spectral sensitivity of detector i(l) associated with channel l; Ill l (λ′) is the illumination spectrum of channel l; 22. A fluorescence microscope (100) according to claim 20 or 21.

Images