Microscope and related devices, methods and computer programs

Abstract

The embodiments relate to a microscope and an apparatus, method, and computer program for a microscope. The microscope includes a light emitting module that provides illumination to an organic tissue sample in multiple wavelength bands. The microscope includes one or more imaging sensor modules configured to independently detect light in multiple, mutually separated wavelength bands of the multiple wavelength bands. The microscope includes a processing module configured to control the light emitting module such that, in a first operating mode, light in a first subset of the multiple wavelength bands is emitted toward the organic tissue sample, and in a second operating mode, light in a second subset of the multiple wavelength bands is emitted toward the organic tissue sample. The wavelength bands of the first subset and the wavelength bands of the second subset are at least partially different. The processing module is configured to perform reflection imaging and fluorescence imaging in each of the multiple, mutually separated wavelength bands based on the light emitted in the first operating mode and the second operating mode, using the one or more imaging sensor modules.

Description

[Technical Field]

[0001] The embodiments relate to a microscope and an apparatus, method and computer program for a microscope. [Background technology]

[0002] Microscopes can be used to capture images in various imaging modes. For example, a microscope can include two sensors, one of which is used for fluorescence imaging and the other for reflection imaging. The two imaging sensor modules can be used for imaging in different wavelength bands. Such microscopes usually include a light source that is used to emit light that can be captured by each sensor, for example, as reflection (i.e., at the same wavelength as that emitted) or as fluorescence (i.e., at a wavelength different from that of the emitted light). Summary of the Invention [Problem to be solved by the invention]

[0003] There may be a need for improved microscope concepts in which sensors are used more versatilely. [Means for solving the problem]

[0004] This need is addressed by embodiments of the present disclosure.

[0005] An embodiment of the present disclosure provides a microscope. The microscope includes a light emitting module that provides illumination to an organic tissue sample in multiple wavelength bands. The microscope includes one or more imaging sensor modules configured to independently detect light in multiple, mutually separated wavelength bands of the multiple wavelength bands. The microscope includes a processing module configured to control the light emitting module such that, in a first operating mode, light in a first subset of the multiple wavelength bands is emitted toward the organic tissue sample, and in a second operating mode, light in a second subset of the multiple wavelength bands is emitted toward the organic tissue sample. The wavelength bands of the first subset and the wavelength bands of the second subset are at least partially different. The processing module is configured to perform reflection imaging and fluorescence imaging in each of the multiple, mutually separated wavelength bands based on the light emitted in the first operating mode and the second operating mode, using the one or more imaging sensor modules.

[0006] Utilizing one or more imaging sensor modules that are sensitive to light in multiple, mutually separated wavelength bands, and with a light emitting module providing the light in an appropriate manner, embodiments can perform reflection and fluorescence imaging in the same wavelength band using a single imaging sensor, improving imaging versatility. Furthermore, when multiple imaging sensor modules are used, different combinations of wavelength bands can be used to, for example, cover all of the wavelength bands in both fluorescence and reflection imaging simultaneously, further improving versatility. This is made possible by light emitting modules that can emit light in different wavelength bands in different operating modes.

[0007] One embodiment of the present disclosure relates to an apparatus for a microscope. The apparatus can be used to control the microscope, for example, with respect to illumination and imaging. The apparatus includes an interface for communication with a light emitting module that provides illumination of an organic tissue sample in multiple wavelength bands. The interface is further suitable for communication with one or more imaging sensor modules configured to independently detect light in multiple, mutually separated wavelength bands of the microscope's multiple wavelength bands. The apparatus includes a processing module configured to control the light emitting module such that, in a first operating mode, light in a first subset of the multiple wavelength bands is emitted toward the organic tissue sample, and in a second operating mode, light in a second subset of the multiple wavelength bands is emitted toward the organic tissue sample. The first subset of wavelength bands and the second subset of wavelength bands are at least partially different. The processing module is configured to perform reflection imaging and fluorescence imaging in each of the multiple, mutually separated wavelength bands using the one or more imaging sensor modules based on the light emitted in the first operating mode and the second operating mode.

[0008] One embodiment of the present disclosure relates to a corresponding method for a microscope. The method includes controlling a light emitting module such that light in a first subset of a plurality of wavelength bands is emitted toward an organic tissue sample in a first operating mode and light in a second subset of the plurality of wavelength bands is emitted toward the organic tissue sample in a second operating mode. The first subset of wavelength bands and the second subset of wavelength bands are at least partially different. The light emitting module is adapted to provide illumination of the organic tissue sample in the plurality of wavelength bands. The method includes using one or more imaging sensor modules to perform reflection imaging and fluorescence imaging in each of the plurality of mutually separated wavelength bands based on the light emitted in the first operating mode and the second operating mode. The one or more imaging sensor modules are configured to be independently sensitive to light in the plurality of mutually separated wavelength bands (e.g., a subset of the plurality of wavelength bands of the microscope). One embodiment of the present disclosure relates to a computer program having program code for performing the method when executed on a processor.

[0009] Some embodiments of the apparatus and / or methods will now be described, by way of example only, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0010] [Figure 1a] FIG. 1 shows a block diagram of an embodiment of a microscope and a microscope apparatus. [Figure 1b] FIG. 1 shows a block diagram of an embodiment of a microscope and a microscope apparatus. [Figure 2] FIG. 1 shows a flowchart of one embodiment of a method for microscopy. [Figure 3] FIG. 1 shows a schematic diagram of an optical imaging system such as a microscope. [Figure 4] FIG. 1 illustrates a table of different illumination spectrums and imaging modes according to one embodiment. [Figure 5] FIG. 1 shows a schematic diagram of a system including a microscope and a computer system. DETAILED DESCRIPTION OF THE INVENTION

[0011] Various embodiments will now be described more fully with reference to the accompanying drawings, in which several embodiments are shown, in which the thickness of lines, layers and / or regions may be exaggerated for clarity.

[0012] Therefore, while various modifications and alternatives are possible for the further embodiments, several specific embodiments are shown in the drawings and described in detail below. However, this detailed description is not intended to limit the further embodiments to the particular forms described. The further embodiments may cover all modifications, equivalents, and alternatives falling within the scope of this disclosure. Identical or similar numbers refer to identical or similar elements throughout the description of the figures, which may be implemented in identical or modified forms when compared to one another while providing the same or similar functionality.

[0013] When an element is referred to as being "connected" or "coupled" to another element, it will be understood that the elements may be directly connected or coupled, or may be connected or coupled via one or more intervening elements. When two elements A and B are connected using "or," this should be interpreted as disclosing all possible combinations, i.e., A only, B only, and A and B, unless otherwise expressly or implicitly defined. Alternative expressions for the same combination are "at least one of A and B" or "A and / or B." The same applies, mutatis mutandis, to combinations of more than two elements.

[0014] The terms used herein for the purpose of describing particular embodiments are not intended to limit further embodiments. Whenever singular forms, such as “a,” “an,” and “the,” are used and are not explicitly or implicitly defined as requiring the use of only a single element, multiple elements can be used to implement the same functionality in further embodiments. Similarly, when a function is described below as being implemented using multiple elements, a single element or processing entity can be used in further embodiments to implement the same functionality. It will be further understood that when the terms “comprises,” “comprising,” “includes,” and / or “including” are used, the presence of stated features, integers, steps, operations, processes, acts, elements, and / or components is specified, but the presence or addition of one or more other features, integers, steps, operations, processes, acts, elements, components, and / or any groups thereof is not excluded.

[0015] Unless otherwise defined, all terms (including technical and scientific terms) are used herein in the ordinary sense of the art to which the examples belong.

[0016] 1a and 1b illustrate block diagrams of an embodiment of a microscope 100. The microscope 100 includes a light emitting module 120 that provides illumination to an organic tissue sample 150 in multiple wavelength bands. The microscope 100 further includes one or more imaging sensor modules 130a and 130b configured to independently detect light in multiple, mutually separated wavelength bands of the multiple wavelength bands. The microscope 100 further includes a processing module 114 coupled to the light emitting module 120 and the one or more imaging sensor modules 130a and 130b, for example, via an interface 112. FIGS. 1a and 1b also illustrate an embodiment of an apparatus 110 that includes the processing module 114 and an interface 112 coupled to the processing module 114. The interface 112 is suitable for communication with the light emitting module 120 and the one or more imaging sensor modules 130a and 130b. The processing module is configured to control the light emitting module 120 so that in a first operating mode, light in a first subset of the plurality of wavelength bands is emitted toward the organic tissue sample 150, and in a second operating mode, light in a second subset of the plurality of wavelength bands is emitted toward the organic tissue sample 150. The first subset of wavelength bands and the second subset of wavelength bands are at least partially different. The processing module 114 is configured to use one or more imaging sensor modules 130a; 130b to perform reflection imaging and fluorescence imaging in each of the plurality of mutually separated wavelength bands based on the light emitted in the first operating mode and the second operating mode. FIG. 1b illustrates a possible implementation of the microscope and / or apparatus of FIG. 1a, using an optical element 140, such as a polychroic mirror, and two imaging sensor modules 130a; 130b.

[0017] Embodiments relate to a microscope or an apparatus, method, or computer program for a microscope. Generally, a microscope is an optical instrument suitable for examining objects too small to be visible to the human eye. For example, a microscope can optically magnify an object, such as an organic tissue sample. In modern microscopes, optical magnification is often provided for a camera or imaging sensor, such as one or more imaging sensor modules 130a, 130b of the microscope 100 of FIG. 1a and / or FIG. 1b. In other words, the microscope 100 can further include one or more optical magnification components used to magnify the view on the organic tissue sample. For example, the microscope 100 can be a microscope for use in a laboratory, such as a microscope that can be used to examine an organic tissue sample in a Petri dish. Alternatively, the microscope 100 can be part of a surgical microscope system, such as a microscope used during a surgical procedure. While the embodiments are described in relation to a microscope, they may also apply more generally to any optical device that includes a light-emitting module and one or more imaging sensor modules. Thus, the apparatus 110, in more general terms, is also suitable for such optical devices.

[0018] The microscope includes a light emitting module 120. The light emitting module 120 is suitable for and / or configured to provide illumination of an organic tissue sample in multiple wavelength bands. For example, the multiple wavelength bands may be defined as mutually separated, i.e., non-overlapping, and / or individual wavelength bands, rather than as a continuous wavelength band consisting of two or more wavelength bands of the multiple wavelength bands. For example, the light emitting module 120 may be configured to block emission at wavelengths outside the multiple wavelength bands, such as emission at wavelengths between the multiple wavelength bands. In other words, the light emitting module 120 may be a multispectral light emitting module 120 configured to emit multiple wavelength bands separated from one another.

[0019] Similar considerations apply to multiple, mutually separated wavelength bands. For example, the multiple, mutually separated wavelength bands may be non-overlapping wavelength bands and / or may be defined as individual wavelength bands rather than as a continuous wavelength band including two or more of the multiple, mutually separated wavelength bands. For example, one or more imaging sensor modules may be configured to ignore light with wavelengths outside the multiple, mutually separated wavelength bands. In at least some embodiments, one or more imaging sensor modules may include two or more optical (bandpass) filters for filtering light outside the multiple, mutually separated wavelength bands. Each filter may be part of a respective filter module or may be disposed between all imaging sensor modules and the organic tissue sample. For example, the two or more bandpass filters may be implemented by a polychroic mirror 140 disposed between the organic tissue sample 150 and one or more imaging sensor modules 130a; 130b.

[0020] For example, the light emitting module may include (exactly) one or multiple individual light sources for generating emitted light. For example, the light emitting module may include one, e.g., exactly one, broadband light source and multiple bandpass filters, which, in combination, are suitable for providing illumination to an organic tissue sample in multiple wavelength bands. For example, the light emitting module 120 may be configured to emit multiple wavelength bands in isolation from one another using two or more optical filters (i.e., multiple optical filters in a number corresponding to the number of wavelength bands among the multiple wavelength bands). For example, the two or more optical filters may be optical bandpass filters, and the passbands of the optical bandpass filters correspond to the multiple wavelength bands. For example, the two or more filters may be part of the light emitting module 120.

[0021] Alternatively, the light emitting module 120 may be configured to use multiple light sources to emit multiple wavelength bands in a mutually separated manner. For example, the multiple individual light sources may be multiple LEDs (light emitting diodes), each configured to emit light in (only) a wavelength band of the multiple wavelength bands. Each of the multiple individual light sources may be tuned to a wavelength band of the multiple wavelength bands. In some embodiments, in addition to the multiple individual light sources, two or more optical filters may be used to emit multiple wavelength bands in a mutually separated manner, e.g., to more precisely define the multiple wavelength bands.

[0022] In the embodiment, multiple different operating modes are used that can affect the light emitted by the light emitting module. For example, the multiple different operating modes include at least the first and second operating modes described above. The processing module 114 is configured to control the light emitting module 120 so that in the first operating mode, light in a first subset of the multiple wavelength bands is emitted toward the organic tissue sample 150, and in the second operating mode, light in a second subset of the multiple wavelength bands is emitted toward the organic tissue sample 150. For example, each operating mode can cause the light emitting module to emit a different subset (or combination / permutation) of the wavelength bands to be emitted. In this regard, a subset of wavelength bands can include one or more wavelength bands. In other words, each subset of wavelength bands can include (or consist of) light from one or more of the multiple wavelength bands (and does not include light of wavelengths outside the one or more wavelength bands). The different subsets of wavelength bands can be at least partially different, e.g., partially different or completely different. Two subsets of wavelength bands are partially different if they share at least one wavelength band included in both subsets. Two subsets of wavelength bands are completely different if they do not share any wavelength bands, i.e., the wavelength bands of the two subsets do not overlap. In embodiments, the wavelength bands of a first subset and the wavelength bands of a second subset may be partially different or completely different.

[0023] For example, the following describes a combination of a first operating mode and a second operating mode in which the first and second subsets are completely different. In one of the operating modes, e.g., in the first operating mode, the processing module 114 may be configured to control the light emitting module 120 so that (only) light in a wavelength band outside the plurality of mutually separated wavelength bands is emitted by the light emitting module 120. For example, this wavelength band may be shown as wavelength band "0" in FIGS. 3 and 4. Because this wavelength band is not one of the plurality of mutually separated wavelength bands, it may be used only for fluorescence imaging, i.e., to perform fluorescence imaging in each of the plurality of mutually separated wavelength bands. Additionally (or alternatively), the processing module 114 may be configured to control the light emitting module 120 so that (only) light in each of the plurality of mutually separated wavelength bands is emitted by the light emitting module 120 in one of the operating modes, e.g., in the second operating mode. For example, in this operating mode, reflection imaging in each of the plurality of mutually separated wavelength bands may be performed. These two modes of operation represent edge cases where reflection or fluorescence imaging can be performed simultaneously in each of multiple wavelength bands that are separated from one another (see, for example, reference numerals 460 and 480 in FIG. 4).

[0024] In some operating modes, a mixed mode may be applied in which one or more of the multiple wavelength bands separated from one another are used for reflectance imaging and one or more other wavelength bands are used for fluorescence imaging. For example, the multiple wavelength bands separated from one another may include a third subset of wavelength bands and a fourth subset of wavelength bands. The third subset and the fourth subset may be discontinuous with one another or may include the entire multiple wavelength bands separated from one another. In other words, each wavelength band of the multiple different wavelength bands may be part of the third subset or the fourth subset. In some embodiments, either the third subset or the fourth subset may be used in conjunction with a wavelength band outside the multiple wavelength bands separated from one another. The processing module 114 may be configured to control the light emitting module 120 such that in one of the operating modes, e.g., the first operating mode or the third operating mode, light in (only) the third subset of wavelength bands is emitted by the light emitting module 120, and in one of the operating modes, e.g., the second operating mode or the fourth operating mode, light in (only) the fourth subset of wavelength bands is emitted by the light emitting module 120. In some embodiments, the third subset of wavelength bands and the fourth subset of wavelength bands may each include two or more wavelength bands enabling simultaneous recording of different wavelength bands.

[0025] In some embodiments, the first subset may be equal to or include the third subset, and / or the second subset may be equal to or include the fourth subset of wavelength bands. In other words, the first subset of the plurality of wavelength bands may include (or correspond to) a wavelength band of the third subset of the plurality of mutually separated wavelength bands. The second subset of the plurality of wavelength bands may include a wavelength band of the fourth subset of the plurality of mutually separated wavelength bands. This may provide an association between the first / third subset of wavelength bands and the second / fourth subset of wavelength bands, which may be used to perform imaging using one or more imaging sensor modules.

[0026] In summary, the light emitting module can: (1) All light in multiple wavelength bands separated from each other; (2) Light in only a wavelength band outside multiple mutually separated wavelength bands; (3) light in a third subset of wavelength bands and optionally wavelength bands outside the plurality of mutually separated wavelength bands; (4) light in a fourth subset of wavelength bands and optionally wavelength bands outside the plurality of mutually separated wavelength bands; Four different modes of operation can be distinguished, in which the radiation is controlled to emit

[0027] For example, the control module may be configured to apply two or more of the above operating modes. That is, the first operating mode and the second operating mode may be selected from the above operating modes (1) to (4). Furthermore, a third operating mode, or the third operating mode and the fourth operating mode, may be used. For example, when three operating modes are used, (3) may be used as the first operating mode, (4) may be used as the second operating mode, and (2) may be used as the third operating mode. When four operating modes are used, all of the operating modes (1) to (4) may be used, as shown in FIG. 4.

[0028] Generally, each operating mode can be used to perform a single recording using one or more imaging sensor modules. For example, within one operating mode, multiple wavelength bands separated from one another can all be recorded (substantially) simultaneously using one or more imaging sensor modules, i.e., within a maximum of 1 second (or a maximum of 0.5 seconds, a maximum of 0.3 seconds, a maximum of 0.1 seconds). Different operating modes can be used consecutively, i.e., sequentially. For example, a first operating mode, a second operating mode, and optionally a third operating mode and a fourth operating mode can be used in that order. Alternatively, another order can be selected.

[0029] The microscope includes one or more imaging sensor modules 130a; 130b. The one or more imaging sensor modules are configured to be independently sensitive to light in multiple, mutually separated wavelength bands of the multiple wavelength bands. In other words, the one or more imaging sensor modules can be configured to be sensitive to light in multiple, mutually separated wavelength bands such that each wavelength band of the multiple, mutually separated wavelength bands is distinguishable (or separately output) at the output of the one or more imaging sensor modules. For example, the one or more imaging sensor modules can be capable of outputting separate sensor measurements for each of the multiple, mutually separated wavelength bands.

[0030] In some embodiments, as shown in FIG. 1b, the one or more imaging sensor modules may include a first imaging sensor module 130a and a second imaging sensor module 130b. Alternatively, as shown in FIG. 1a, the microscope 100 may include a single imaging sensor module 130a. The processing module 114 may be configured to use the first imaging sensor module and the second imaging sensor module to perform reflection imaging and fluorescence imaging in each of a plurality of mutually separated wavelength bands based on light emitted in the first and second operating modes. By using two sensor modules, each sensor module need only be sensitive to a subset of the mutually separated wavelength bands, allowing for greater flexibility and / or facilitating construction of each imaging sensor module. For example, the first imaging sensor module and the second imaging sensor module may be configured to be sensitive to different wavelength bands of the mutually separated wavelength bands. Therefore, circuitry used to redundantly measure light in the same wavelength band need not be included, allowing for greater flexibility and / or facilitating construction of each imaging sensor module. For example, each of the first imaging sensor module and the second imaging sensor module may be configured to be sensitive only to light in one or more wavelength bands to which the other imaging sensor module is not sensitive.

[0031] For example, the first imaging sensor module may be configured to be sensitive to light in (only) a third subset of the plurality of mutually separated wavelength bands. The second imaging sensor module may be configured to be sensitive to light in (only) a fourth subset of the plurality of mutually separated wavelength bands. The first imaging sensor module and the second imaging sensor module may together cover the entire plurality of mutually separated wavelength bands without overlapping the wavelength bands they measure. For example, the first imaging sensor module may be configured to output a separate sensor measurement value for each wavelength band in the third subset of wavelength bands. The second imaging sensor module may be configured to output a separate sensor measurement value for each wavelength band in the fourth subset of wavelength bands. Generally, one or more imaging sensor modules may be APS (active pixel sensor)-based or CCD (charge-coupled device)-based imaging sensor modules. For example, in an APS-based imaging sensor module, light is recorded at each pixel using a photodetector and an active amplifier in the pixel. APS-based imaging sensor modules are often based on CMOS (Complementary Metal Oxide Semiconductor) or S-CMOS (Scientific CMOS) technology. In CCD-based imaging sensor modules, incident photons are converted into electronic charges at the semiconductor-oxide interface, and then the electronic charges are transferred between capacitive bins within the imaging sensor module by the sensor imaging module's control circuit to perform imaging.

[0032] The processing module is configured to use one or more imaging sensor modules 130a; 130b to perform reflection imaging and fluorescence imaging in each of a plurality of mutually separated wavelength bands based on light emitted in the first and second operating modes. In some embodiments, light emitted in the first operating mode is used to perform reflection imaging, and light emitted in the second operating mode is used to perform fluorescence imaging in (all of) a plurality of mutually separated wavelength bands, for example, when the operating modes (1) and (2) introduced above are selected. For example, the processing module 114 may be configured to use the first imaging sensor module 130a and the second imaging sensor module 130b to perform reflection imaging in the first operating mode and fluorescence imaging in the second operating mode (or vice versa). This may result in reflection and fluorescence images being captured simultaneously, with a slight delay between the reflection and fluorescence images. Alternatively, a mixed mode may be used as introduced above, in which performance and reflection imaging are performed in both the first and second operating modes. In this case, there may be a slight delay in the reflectance / fluorescence images, but no overall delay in one or the other. For example, the processing module 114 may be configured to use the first imaging sensor module 130a to perform reflectance imaging in a first operating mode and fluorescence imaging in a second operating mode. The processing module 114 may be configured to use the second imaging sensor module 130b to perform fluorescence imaging in the first operating mode and reflectance imaging in the second operating mode (or vice versa).

[0033] As introduced above, both approaches can be selected, for example, to increase the number of reflection / fluorescence images captured at each wavelength. For example, the processing module 114 may be configured to perform reflection imaging in a first operational mode using the first imaging sensor module 130a and the second imaging sensor module 130b, perform fluorescence imaging in a second operational mode using the first imaging sensor module 130a, perform reflection imaging in a third operational mode using the first imaging sensor module 130a, perform fluorescence imaging in a fourth operational mode using the second imaging sensor module 130b, and perform fluorescence imaging in the third operational mode and reflection imaging in the fourth operational mode using the second imaging sensor module 130b. In either case, the numbering of the operational modes is used only to distinguish between different operational modes and may be changed, for example, if a different order of the operational modes is desired.

[0034] When two (or more) different imaging sensor modules are used, light emanating from the organic tissue sample can be distributed between the sensor modules using a polychroic mirror. Generally, a polychroic mirror is an optical element configured to selectively reflect or transmit light of multiple wavelength bands (hence the term "polychroic" meaning "multiple colors"). In this case, the microscope can include a polychroic 140 configured to separate light used by a first imaging sensor module from light used by a second imaging sensor module. For example, the polychroic mirror 140 can be configured to separate light having a wavelength in a third subset of wavelength bands from light having a wavelength in a fourth subset of wavelength bands of the multiple mutually separated wavelength bands. The polychroic mirror can be configured and arranged such that light having a wavelength in the third subset of wavelength bands is incident on (or reflected from) the polychroic mirror and (re)directed to the first imaging sensor module. The polychroic mirror may be constructed and arranged such that light having a wavelength in one of the wavelength bands of the fourth subset is reflected by (or incident on) the polychroic mirror and (re)directed towards the second imaging sensor module. Alternatively, the polychroic mirror may be replaced by a semi-transparent mirror suitable for partially reflecting light emanating from the organic tissue sample, regardless of the wavelength of the light, such that the full spectrum is directed towards both imaging sensor modules.

[0035] The interface 112 may correspond to one or more inputs and / or outputs for receiving and / or transmitting information, which may be digital (bit) values ​​in a specified code within a module, between modules, or between modules of different entities. For example, the interface 12 may include an interface circuit configured to receive and / or transmit information. In some embodiments, the processing module 114 may be implemented using any processing means, such as one or more processing units, one or more processing devices, processors, computers, or programmable hardware components operable with correspondingly adapted software. In other words, the above-described functionality of the processing module 114 may be implemented in software, which may then be executed on one or more programmable hardware components. Such hardware components may include general-purpose processors, digital signal processors (DSPs), microcontrollers, etc.

[0036] Further details and aspects of the microscope or apparatus for microscopy are set forth in relation to the proposed concept or one or more of the embodiments described above or below (e.g., Figures 2-4). The microscope or apparatus for microscopy may include one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more of the embodiments described above or below.

[0037] FIG. 2 shows a flowchart of one embodiment of a corresponding method, e.g., a computer-implemented method, for a microscope, e.g., the microscope 100 of FIG. 1a and / or FIG. 1b. While aspects of the microscope have been described in the context of an apparatus, these aspects also represent a description of the corresponding method, and it is clear that properties or functional features of the apparatus correspond to method steps or features of the method steps. The method includes controlling (210) the light-emitting module 120 so that in a first mode of operation, light in a first subset of wavelength bands is emitted toward the organic tissue sample 150, and in a second mode of operation, light in a second subset of wavelength bands is emitted toward the organic tissue sample 150. The first subset of wavelength bands and the second subset of wavelength bands are at least partially different. The light-emitting module 120 is adapted to provide illumination of the organic tissue sample 150 in the multiple wavelength bands. The method includes performing (220) reflection imaging and fluorescence imaging in each of the multiple wavelength bands, separated from one another, based on the light emitted in the first mode of operation and the second mode of operation, using one or more imaging sensor modules. One or more imaging sensor modules 130a; 130b are configured to be independently sensitive to light in multiple, mutually separated wavelength bands that are a subset of the wavelength bands of microscope 100. The features described in relation to microscope 100 of Figures 1a / 1b above are equally applicable to the method of Figure 2.

[0038] Further details and aspects of the method are referred to in relation to the proposed concept or one or more of the examples above or below (e.g., Figures 1a / 1b, 3 or 4). The method may include one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more of the examples above or below.

[0039] At least some embodiments relate to the use of multiple imaging sensor modules to capture multiple images with different parameters, for example in a multi-sensor camera. An aim of at least some embodiments may be to improve utilization of the presence of multiple imaging sensor modules in imaging devices such as surgical microscopes.

[0040] Some imaging devices may employ multiple imaging sensor modules, such as CCD, CMOS, and SCMOS. This is because it is useful to simultaneously capture multiple images with different properties, such as reflection or fluorescence. For example, color and visible pseudocolor can be achieved by simultaneously capturing multiple images, such as reflectance and fluorescence images. In this case, for example, each sensor can be specialized to capture one type of image without changing the sensor's function. However, the functions performed by some sensors may not necessarily be required. For example, if fluorescence imaging is not required, the specialized sensors remain unused. In embodiments, a technique is proposed that utilizes imaging sensor modules installed to perform different functions in a tailored manner.

[0041] FIG. 3 shows a schematic diagram of an optical imaging system, such as a microscope, including a light emitting module 120 configured to provide illumination to an object in a subset of wavelength bands 0 through 6, as shown in graph 310 (where the x-axis represents wavelength and the y-axis represents amplitude), and two imaging sensor modules 130a and 130b for recording light arriving from tissue and split by a polychroic mirror 140. Graph 320 shows the sensitivity of sensor 130a, and graph 330 shows the sensitivity of sensor 130b (where the x-axis represents wavelength and the y-axis represents sensitivity to incident light at each wavelength). As shown in FIG. 3, sensor 130a is sensitive in wavelength bands 1, 3, and 5, and sensor 130b is sensitive in wavelength bands 2, 4, and 6. When the optical imaging system is used in medical applications, each wavelength band may represent a specific type or state of tissue (e.g., different tissue types, healthy tissue, pathological tissue). Thus, FIG. 3 may show the spectral bands of a multispectral imaging sensor. In Figure 3, light emitting module 120 is configured to emit light in wavelength bands 1, 3, and 5 (which match the wavelength bands of sensor 130a). In Figure 3, sensor 130a receives light in the same bands as the illumination light and is therefore able to detect reflected light. Sensor 130b has a sensitivity curve complementary to the illumination spectrum and therefore detects only fluorescent light.

[0042] By changing the illumination spectrum and illuminating with different spectral bands, sensors 130a and 130b can still be sensitive to the same spectral band, but the light in that band is of a different nature (reflected / fluorescent). Adjusting the illumination spectral band can be done, for example, by using interference multiple bandpass filters (in light emitting module 120) or by using a multispectral light source consisting of multiple (in this case seven) monochromatic light sources, such as LEDs (light emitting diodes).

[0043] In embodiments, varying the light emitted toward tissue can improve the use of the imaging sensor module. FIG. 4 shows a table of different illumination spectra that can be implemented using the light emitting module 120 and that can result in different imaging modes (i.e., resulting properties of the measured spectral bands) in the imaging sensor modules 130a; 130b. In FIG. 4, the left side 410 of the table shows wavelength bands 0-6 440 emitted by the light emitting module. For example, when used in conjunction with FIGS. 1a / 1b, light emitted in wavelength band 0 can be light in a wavelength band outside of multiple mutually separated wavelength bands. Wavelength bands 1-6 can correspond to multiple mutually separated wavelength bands. The right side 420 shows the imaging modes (R = reflectance, F = fluorescence) used by each sensor 430 (A = 130a; B = 130b) when tissue is illuminated using the spectrum shown on the left side 410. As shown in connection with FIGS. 1a / 1b, the imaging sensor modules 130a and 130b cover (only) a plurality of mutually separated wavelength bands 1 through 6. Reference numerals 450 through 480 indicate different spectra obtainable in different operating modes of the light emitting module. For example, the spectrum indicated by reference numeral 450 may correspond to light emitted in operating mode (3) of FIG. 1a / 1b, the spectrum indicated by reference numeral 460 may correspond to light emitted in operating mode (1) of FIG. 1a / 1b, the spectrum indicated by reference numeral 470 may correspond to light emitted in operating mode (4) plus wavelength band 0 outside the plurality of mutually separated wavelength bands, and the spectrum indicated by reference numeral 480 may correspond to light emitted in operating mode (2). The table below summarizes, under reference numeral 410, four different illumination spectra (at least 460; 470 and 480 have not been used previously) and, under reference numeral 420, whether reflection or fluorescence is used.In the example of FIG. 4, the spectrum of reference number 450 yields three spectral bands measured in reflectance and three spectral bands measured in fluorescence, the spectrum of reference number 460 yields six spectral bands measured in reflectance, the spectrum of reference number 470 yields three spectral bands measured in reflectance and three spectral bands measured in fluorescence, and the spectrum of reference number 480 yields six spectral bands measured in fluorescence.

[0044] Further details and aspects of the concepts illustrated in connection with Figures 3 and / or 4 are described in connection with the proposed concepts or one or more examples above or below (e.g., Figures 1a / 1b and / or 2). The concepts illustrated in connection with Figures 3 and / or 4 may include one or more additional optional features corresponding to one or more aspects of the proposed concepts or one or more examples above or below.

[0045] Some embodiments relate to a microscope including a system described in connection with one or more of FIGS. 1-4. Alternatively, the microscope may be part of or connected to a system such as described in connection with one or more of FIGS. 1-4. FIG. 5 shows a schematic diagram of a system 500 configured to perform methods described herein. The system 500 includes a microscope 510 and a computer system 520. The microscope 510 is configured to capture images and is connected to the computer system 520. The computer system 520 is configured to perform at least some of the methods described herein. The computer system 520 may be configured to execute machine learning algorithms. The computer system 520 and the microscope 510 may be separate entities or may be integrated within a common housing. The computer system 520 may be part of a central processing system of the microscope 510 and / or part of a subsidiary component of the microscope 510, such as a sensor, actor, camera, or lighting unit of the microscope 510.

[0046] The computer system 520 may be a local computing device (e.g., a personal computer, laptop, tablet computer, or mobile phone) with one or more processors and one or more storage devices, or may be a distributed computing system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, such as local clients and / or one or more remote server farms and / or data centers). The computer system 520 may include any circuit or combination of circuits. In one embodiment, the computer system 520 may include one or more processors, which may be of any type. As used herein, a processor may contemplate any type of computing circuit, such as, but not limited to, a microprocessor of a microscope or microscope component (e.g., a camera), a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit. Other types of circuitry that may be included in computer system 520 may be custom circuitry, application specific integrated circuits (ASICs), etc., such as one or more circuits (e.g., communications circuits) used in wireless devices such as cell phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. Computer system 520 may also include one or more storage devices, which may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives that handle removable media, such as compact discs (CDs), flash memory cards, digital video discs (DVDs), etc.Computer system 520 may also include a display device, one or more speakers and a controller which may include a keyboard and / or mouse, trackball, touch screen, voice recognition device, or any other device that allows a user of the system to input information to and receive information from computer system 520.

[0047] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0048] While some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step. Similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, microprocessor, programmable computer, or electronic circuitry. In some embodiments, any one or more of the critical steps may be performed by such an apparatus.

[0049] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be performed by a non-transitory storage medium, such as a digital storage medium, for example, a floppy disk, a DVD, a Blu-ray, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a FLASH memory, on which electronically readable control signals are stored that cooperate (or can cooperate) with a programmable computer system to implement the respective methods. Therefore, the digital storage medium may be computer-readable.

[0050] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system to perform any of the methods described herein.

[0051] Generally, embodiments of the present invention may be implemented as a computer program product comprising program code that operates to perform any of the methods when the computer program product is run on a computer, and that may be stored, for example, on a machine-readable carrier.

[0052] Further embodiments comprise the computer program for performing any of the methods described herein, stored on a machine readable carrier.

[0053] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing any of the methods described herein, when the computer program runs on a computer.

[0054] Therefore, another embodiment of the invention is a recording medium (or data carrier or computer readable medium) comprising a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital recording medium or recording medium is typically tangible and / or non-transitory. Another embodiment of the invention is an apparatus as described herein, comprising a processor and a recording medium.

[0055] A further embodiment of the present invention is, therefore, a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or sequence of signals being for example adapted to be transmitted via a data communication connection, for example the Internet.

[0056] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.

[0057] Another embodiment comprises a computer having installed thereon the computer program for performing any of the methods described herein.

[0058] Another embodiment of the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0059] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus. [Explanation of symbols]

[0060] 100 microscopes 110 Apparatus for microscopes 112 Interface 114 Processing Module 120 Light Emitting Module 130a Image sensor module 130b Image sensor module 140 Polychroic Mirror 150 organic tissue samples 210 Controlling the light-emitting module 220 Use one or more image sensor modules 310 Lighting Graph 320 Sensor sensitivity graph for sensor A 330 Sensor sensitivity graph for sensor B 410 Wavelength band of light emitted by the light-emitting module Imaging modes of sensors A / B in the 420 wavelength band 430 Sensor A or Sensor B 440 wavelength band 450,460,470,480 Light spectrum and corresponding imaging modes 500 Systems 510 Microscope 520 Computer Systems

Claims

1. A microscope (100; 510) comprising: a light emitting module (120) that provides illumination to an organic tissue sample (150) in multiple wavelength bands; a first imaging sensor module (130a) and a second imaging sensor module (130b) configured to be independently sensitive to light in a plurality of mutually separated wavelength bands of the plurality of wavelength bands, wherein the first imaging sensor module is configured to be sensitive to light in a third subset of wavelength bands of the mutually separated plurality of wavelength bands, and the second imaging sensor module is configured to be sensitive to light in a fourth subset of wavelength bands of the mutually separated plurality of wavelength bands, each of the third subset of wavelength bands and the fourth subset of wavelength bands including two or more wavelength bands; a processing module (114); Equipped with The processing module (114) controlling the light emitting module (120) such that in a first operating mode, light in a first subset of the plurality of wavelength bands is emitted toward the organic tissue sample (150); in a second operating mode, light in a second subset of the plurality of wavelength bands is emitted toward the organic tissue sample (150); and in a third operating mode, light in a fifth subset of only wavelength bands outside the plurality of mutually separated wavelength bands is emitted toward the organic tissue sample (150), the wavelength bands of the first subset and the wavelength bands of the second subset being at least partially different; performing reflection imaging and fluorescence imaging in each of the plurality of mutually separated wavelength bands based on light emitted in the first and second operational modes using the first and second imaging sensor modules; It is configured as follows: Microscope (100; 510).

2. the processing module (114) is configured to use the first imaging sensor module (130a) to perform reflectance imaging in the first mode of operation and to perform fluorescence imaging in the second mode of operation. The microscope according to claim 1.

3. the processing module (114) is configured to use the second imaging sensor module (130b) to perform fluorescence imaging in the first mode of operation and to perform reflectance imaging in the second mode of operation.

3. The microscope according to claim 1 or 2.

4. the microscope comprises a polychroic mirror (140) configured to separate light having a wavelength in the third subset of wavelength bands from light having a wavelength in the fourth subset of wavelength bands of the plurality of mutually separated wavelength bands; 4. The microscope according to claim 1.

5. the first subset of the plurality of wavelength bands includes wavelength bands of the third subset of the plurality of mutually separated wavelength bands, and the second subset of the plurality of wavelength bands includes wavelength bands of the fourth subset of the plurality of mutually separated wavelength bands; 5. A microscope according to any one of claims 1 to 4.

6. the first imaging sensor module and the second imaging sensor module are configured to be sensitive to different wavelength bands among the plurality of mutually separated wavelength bands; 6. A microscope according to any one of claims 1 to 5.

7. the processing module (114) is configured to control the light emitting module (120) so that light in a wavelength band outside the plurality of mutually separated wavelength bands is emitted by the light emitting module (120) in one of the operation modes.

7. A microscope according to any one of claims 1 to 6.

8. the processing module (114) is configured to control the light emitting module (120) so that light in each of the plurality of mutually separated wavelength bands is emitted by the light emitting module (120) in one of the operation modes.

8. A microscope according to any one of claims 1 to 7.

9. the light emitting module (120) is a multispectral light emitting module (120) configured to emit the plurality of wavelength bands separately from one another; 9. A microscope according to any one of claims 1 to 8.

10. The light emitting module (120) is configured to emit the plurality of wavelength bands in a mutually isolated manner using two or more optical filters.

10. The microscope according to claim 9.

11. The light emitting module (120) is configured to emit the plurality of wavelength bands separately from one another using a plurality of individual light sources.

10. The microscope according to claim 9.

12. An apparatus (110) for a microscope (100; 510), said apparatus (110) comprising: an interface (112); a processing module (114); Equipped with the interface (112) is in communication with a light emitting module (120) that provides illumination of an organic tissue sample (150) in a plurality of wavelength bands, and is in communication with a first imaging sensor module (130a) and a second imaging sensor module (130b) that are configured to be independently sensitive to light in a plurality of mutually separated wavelength bands of the plurality of wavelength bands of the microscope, the first imaging sensor module being configured to be sensitive to light in a third subset of wavelength bands of the mutually separated plurality of wavelength bands, and the second imaging sensor module being configured to be sensitive to light in a fourth subset of wavelength bands of the mutually separated plurality of wavelength bands, each of the third subset of wavelength bands and the fourth subset of wavelength bands including two or more wavelength bands; The processing module (114) controlling the light emitting module (120) such that in a first operating mode, light in a first subset of the plurality of wavelength bands is emitted toward the organic tissue sample (150); in a second operating mode, light in a second subset of the plurality of wavelength bands is emitted toward the organic tissue sample (150); and in a third operating mode, light in a fifth subset of only wavelength bands outside the plurality of mutually separated wavelength bands is emitted toward the organic tissue sample (150), the wavelength bands of the first subset and the wavelength bands of the second subset being at least partially different; performing reflection imaging and fluorescence imaging in each of the plurality of mutually separated wavelength bands based on light emitted in the first and second operational modes using the first and second imaging sensor modules; It is configured as follows: Apparatus (110).

13. A method for a microscope (100; 510), said method comprising: controlling (210) the light emitting module (120) such that in a first mode of operation, light in a first subset of a plurality of wavelength bands is emitted toward the organic tissue sample (150), in a second mode of operation, light in a second subset of the plurality of wavelength bands is emitted toward the organic tissue sample (150), and in a third mode of operation, light in a fifth subset of only wavelength bands outside the plurality of mutually separated wavelength bands is emitted toward the organic tissue sample (150), the wavelength bands of the first subset and the wavelength bands of the second subset being at least partially different, and the light emitting module (120) is adapted to provide illumination of the organic tissue sample (150) in the plurality of wavelength bands; performing (220) reflection imaging and fluorescence imaging in each of the plurality of mutually separated wavelength bands based on light emitted in the first and second operational modes using a first imaging sensor module (130a) and a second imaging sensor module (130b); Including, the first imaging sensor module (130a) and the second imaging sensor module (130b) are configured to be independently sensitive to light in the plurality of mutually separated wavelength bands, the first imaging sensor module is configured to be sensitive to light in a third subset of wavelength bands of the plurality of mutually separated wavelength bands, and the second imaging sensor module is configured to be sensitive to light in a fourth subset of wavelength bands of the plurality of mutually separated wavelength bands, and each of the third subset of wavelength bands and the fourth subset of wavelength bands includes two or more wavelength bands. method.

14. 14. A computer program having a program code for performing the method of claim 13, when the computer program runs on a processor.

Images