Microscopes and related apparatus, methods and computer programs

By introducing multiple imaging sensor modules and light emission modules into the microscope, light emission control of different wavelength bands can be achieved, solving the problem that existing microscopes cannot simultaneously perform reflection and fluorescence imaging, and improving imaging flexibility and the utilization efficiency of sensor modules.

CN114556184BActive Publication Date: 2024-11-22LEICA INSTRUMENTS (SINGAPORE) PTE LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080070739.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-10-10
Filing Date
2020-10-07
Publication Date
2024-11-22
Estimated Expiration
2040-10-07

AI Technical Summary

Technical Problem

Existing microscopes have limitations in imaging modes, making it difficult to efficiently achieve reflectance imaging and fluorescence imaging in the same device, and the use of sensor modules is not flexible enough.

Method used

Design a microscope equipped with multiple imaging sensor modules and a light emission module, capable of emitting light of different wavelength bands in different operating modes. The light emission module and the imaging sensor module are controlled by a processing module to achieve switching between reflection imaging and fluorescence imaging, and to perform imaging using multiple mutually separate wavelength bands.

Benefits of technology

It improves the imaging versatility and flexibility of the microscope, enabling simultaneous or sequential reflection and fluorescence imaging in the same device, and enhances the utilization efficiency of the sensor module.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114556184B_ABST
    Figure CN114556184B_ABST
Patent Text Reader

Abstract

Examples relate to microscopes and apparatus, methods, and computer programs for microscopes. The microscope includes a light emission module for providing illumination of an organic tissue sample in multiple wavelength bands. The microscope includes one or more imaging sensor modules configured to independently sense light in multiple mutually separated wavelength bands in the multiple wavelength bands. The microscope includes a processing module configured to control the light emission module so that light in a first subset of the multiple wavelength bands is emitted to the organic tissue sample in a first operating mode, and light in a second subset of the multiple wavelength bands is emitted to the organic tissue sample in a second operating mode. The first and second wavelength band subsets are at least partially different. The processing module is configured to perform reflectance imaging and fluorescence imaging in each of the multiple mutually separated wavelength bands using one or more imaging sensor modules based on the light emitted in the first operating mode and the second operating mode.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Examples relate to microscopes, and apparatus, methods and computer programs for microscopes. Background Art

[0002] The microscope can be used to perform imaging in various imaging modes. For example, the microscope can include two sensors, one for fluorescence imaging and one for reflection imaging. The two imaging sensor modules may be used for imaging in different wavelength bands. Such microscopes typically include a light source for emitting light that can be picked up by the corresponding sensor, such as reflected light (i.e., the same wavelength as emitted) or fluorescent light (i.e., a wavelength different from the wavelength of the emitted light). SUMMARY OF THE INVENTION

[0004] It may be desirable to improve the concept of microscopy in which sensors are used in a more general way.

[0005] Embodiments of the present disclosure address this desire.

[0006] An embodiment of the present disclosure provides a microscope. The microscope includes a light emitting module for providing illumination for an organic tissue sample in multiple wavelength bands. The microscope includes one or more imaging sensor modules configured to independently sense light in multiple mutually separated wavelength bands in the multiple wavelength bands. The microscope includes a processing module configured to control the light emitting module so that light in a first subset of multiple wavelength bands is emitted to the organic tissue sample in a first operating mode, and light in a second subset of multiple wavelength bands is emitted to the organic tissue sample in a second operating mode. The first and second wavelength band subsets are at least partially different. The processing module is configured to perform reflectance imaging and fluorescence imaging in each of the multiple mutually separated wavelength bands using one or more imaging sensor modules based on the light emitted in the first operating mode and the second operating mode.

[0007] By using one or more imaging sensor modules capable of sensing light in multiple mutually separated wavelength bands, and if the light emitting module provides light in a suitable manner, embodiments can use a single imaging sensor to perform both reflectance imaging and fluorescence imaging in the same wavelength band, improving the versatility of imaging. In addition, if multiple imaging sensor modules are used, versatility can be further improved because different wavelength band combinations can be used to cover all wavelength bands in fluorescence and reflectance imaging, such as simultaneously. This is achieved by light emitting modules capable of emitting light in different wavelength bands in different operating modes.

[0008] Embodiments of the present disclosure relate to a device for a microscope. The device can be used to control a microscope, such as controlling illumination and imaging. The device includes an interface for communicating with a light emitting module that provides illumination of an organic tissue sample in multiple wavelength bands. The interface is further adapted to communicate with one or more imaging sensor modules, which are configured to independently sense light in multiple mutually separated wavelength bands of the microscope's multiple wavelength bands. The device includes a processing module configured to control the light emitting module so that light in a first subset of multiple wavelength bands is emitted to the organic tissue sample in a first operating mode, and light in a second subset of multiple wavelength bands is emitted to the organic tissue sample in a second operating mode. The first and second subsets of wavelength bands are at least partially different. The processing module is configured to perform reflectance imaging and fluorescence imaging in each of the multiple mutually separated wavelength bands using one or more imaging sensor modules based on the light emitted in the first operating mode and the second operating mode.

[0009] Embodiments of the present disclosure relate to a corresponding method for a microscope. The method includes controlling a light emission module so that light in a first subset of multiple wavelength bands is emitted to an organic tissue sample in a first operating mode, and light in a second subset of multiple wavelength bands is emitted to the organic tissue sample in a second operating mode. The first and second subsets of wavelength bands are at least partially different. The light emission module is suitable for providing illumination of the organic tissue sample in multiple wavelength bands. The method includes performing reflective imaging and fluorescent imaging in each of a plurality of mutually separated wavelength bands using one or more imaging sensor modules based on the light emitted in the first operating mode and the second operating mode. The one or more imaging sensor modules are configured to independently sense light in a plurality of mutually separated wavelength bands (e.g., as a subset of a plurality of wavelength bands of a microscope). Embodiments of the present disclosure relate to a computer program having a program code, the program code being used to execute the method when the computer program is executed on a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Some examples of the apparatus and / or method will now be described, by way of example only, with reference to the accompanying drawings, in which

[0011] Figure 1a and 1b A block diagram showing an embodiment of a microscope and an apparatus for a microscope;

[0012] Figure 2 A flow chart illustrating an embodiment of a method for microscopy;

[0013] Figure 3 shows a schematic diagram of an optical imaging system, such as a microscope;

[0014] Figure 4a table showing different illumination spectra and imaging modes according to embodiments; and

[0015] Figure 5 A schematic diagram of a system including a microscope and a computer system is shown. DETAILED DESCRIPTION

[0016] Various examples will now be described more fully with reference to the accompanying drawings, in which some examples are illustrated. In the drawings, the thickness of lines, layers and / or regions may be exaggerated for clarity.

[0017] Therefore, although further examples are capable of various modifications and alternative forms, some specific examples are shown in the figures and will be described in detail later. However, this detailed description does not limit the further examples to the specific forms described. Further examples may include all modifications, equivalents and alternatives within the scope of this disclosure. The same or similar numbers indicate similar or similar elements throughout the description of the figures, and these elements can be implemented in the same or modified form when providing the same or similar functions.

[0018] It is understood that when an element is referred to as being "connected" or "coupled" to another element, these elements may be directly connected or coupled, or connected or coupled via one or more intermediate elements. If two elements A and B are combined with "or", this is understood to disclose all possible combinations, if there is no explicit or implicit other definition, i.e. only A, only B, and A and B. Alternative statements for the same combination are "at least one of A and B" or "A and / or B". The same applies to combinations of more than two elements.

[0019] The terms used to describe specific examples herein are not intended to limit further examples. When using singular forms such as "one", "an" and "this", and only using a single element, when there is neither an explicit definition nor an implicit definition as a mandatory requirement, further examples may also use plural elements to implement the same function. Similarly, when the function is subsequently described as being implemented using multiple elements, further examples may implement the same function using a single element or processing entity. It should be further understood that the terms "include", "include", "include" and / or "include" indicate the features, integers, steps, operations, processes, behaviors, elements and / or components when used, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, behaviors, elements, components and / or any groups thereof. Unless otherwise defined, all terms (including technical and scientific terms) are used in this article.

[0020] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same common meaning as those in the art to which examples belong.

[0021] Figure 1 and Figure 1b A block diagram of an embodiment of a microscope 100 is shown. The microscope 100 includes a light emission module 120 for providing illumination for an organic tissue sample 150 in a plurality of wavelength bands. The microscope 100 further includes one or more imaging sensor modules 130a, 130b, which are configured to independently sense light in a plurality of mutually separated wavelength bands in the plurality of wavelength bands. The microscope 100 further includes a processing module 114, which is coupled to the light emission module 120 and the one or more imaging sensor modules 130a, 130b, for example, via an interface 112. Figure 1a and 1b Further illustrating an embodiment of the device 110, the device 110 includes a processing module 114 and an interface 112, the interface 112 being coupled to the processing module 114. The interface 112 is adapted to communicate with the light emission module 120 and with one or more imaging sensor modules 130a, 130b. The processing module is configured to control the light emission module 120 so that light in a first subset of a plurality of wavelength bands is emitted to the organic tissue sample 150 in a first operating mode, and light in a second subset of a plurality of wavelength bands is emitted to the organic tissue sample 150 in a second operating mode. The first and second subsets of wavelength bands are at least partially different. The processing module 114 is configured to perform reflectance imaging and fluorescence imaging in each of a plurality of mutually separated wavelength bands using the one or more imaging sensor modules 130a, 130b based on the light emitted in the first operating mode and the second operating mode. Figure 1b One possible implementation of the microscope and / or device of FIG. 1a is shown, wherein an optical element 140, such as a polychromatic mirror, is used, and two imaging sensor modules 130a, 140a.

[0022] Embodiments relate to microscopes, or apparatus, methods, or computer programs for microscopes. In general, a microscope is an optical instrument useful for examining objects that are too small to be examined by the naked eye (alone). For example, a microscope can provide optical magnification of an object, such as an organic tissue sample. In modern microscopes, the optical magnification is typically provided to a camera or imaging sensor, such as a Figure 1aand / or 1b of one or more imaging sensor modules 130a / b of the microscope 100. In other words, the microscope 100 may further include one or more optical magnification components for magnifying the view of the organic tissue sample. For example, the microscope 100 may be a laboratory microscope, such as a microscope that can be used to examine organic tissue samples in a culture dish. Alternatively, the microscope 100 may be part of a surgical microscope system, such as a microscope used during a surgical procedure. Although embodiments related to microscopes are described, they may also be applied in a more general manner to any optical device that includes a light emission module and one or more imaging sensor modules. Therefore, in more general terms, the device 110 is also applicable to such optical devices.

[0023] The microscope includes a light emission module 120. The light emission module 120 is suitable for, and / or configured to, provide illumination for an organic tissue sample in a plurality of wavelength bands. For example, the plurality of wavelength bands may be a plurality of mutually separated wavelength bands, i.e., non-overlapping and / or defined as a single wavelength band, i.e., not as a continuous wavelength band including two or more of the plurality of wavelength bands. For example, the light emission module 120 may be configured to block light emission at wavelengths outside the plurality of wavelength bands, such as light emission at wavelengths between the plurality of wavelength bands. In other words, the light emission module 120 may be a multi-spectral light emission module 120 configured to emit a plurality of wavelength bands in a mutually separated manner.

[0024] Similar considerations apply to multiple mutually separated wavelength bands. For example, multiple mutually separated wavelength bands may be non-overlapping wavelength bands and / or may be defined as a single wavelength band, i.e., not a continuous wavelength band including two or more of the multiple mutually separated wavelength bands. For example, one or more imaging sensor modules may be configured to ignore light at wavelengths outside the multiple mutually separated wavelength bands. In at least some embodiments, one or more imaging sensor modules may include two or more optical (bandpass) optical filters for filtering out light outside the multiple mutually separated wavelength bands. Each filter may be part of a respective filter module or may be arranged between all imaging sensor modules and the organic tissue sample. For example, two or more bandpass filters may be implemented by a polychromatic mirror 140, which is arranged between the organic tissue sample 150 and one or more imaging sensor modules 130a, 130b.

[0025] For example, the light emission module may include (precisely) one or more single light sources for generating the light to be emitted. For example, the light emission module may include one (e.g., precisely one) broadband light source, and a plurality of bandpass filters, which are combined to provide illumination for an organic tissue sample in a plurality of wavelength bands. For example, the light emission module 120 may be configured to use two or more optical filters (i.e., a plurality of optical filters, the number of filters matching the number of wavelength bands of the plurality of wavelength bands) to emit a plurality of wavelength bands in a mutually separated manner. For example, the two or more optical filters may be optical bandpass filters, the passbands of the optical bandpass filters matching the plurality of wavelength bands. For example, the two or more optical filters may be part of the light emission module 120.

[0026] Alternatively, the light emitting module 120 can be configured to use a plurality of separate light sources to emit a plurality of wavelength bands in a mutually separated manner. For example, the plurality of separate light sources can be a plurality of LEDs (light emitting diodes), each of which is configured to (only) emit light in a wavelength band of a plurality of wavelength bands. Each of the plurality of separate light sources can be tuned to a wavelength band of a plurality of wavelength bands. In some embodiments, in addition to a plurality of separate light sources, two or more optical filters can be used to emit a plurality of wavelength bands in a mutually separated manner, for example, in order to more accurately define a plurality of wavelength bands.

[0027] In an embodiment, a plurality of different operating modes are used, which can affect the light emitted by the light emission module. For example, the plurality of different operating modes include at least the first operating mode and the second operating mode described above. The processing module 114 is configured to control the light emission module 120 so that in the first operating mode, light in a first subset of a plurality of wavelength bands is emitted to the organic tissue sample 150, and in the second operating mode, light in a second subset of a plurality of wavelength bands is emitted to the organic tissue sample 150. For example, each operating mode may cause the light emission module to emit a different subset (or combination / arrangement) of wavelength bands to be emitted. In this case, the subset of wavelength bands may include one or more wavelength bands. In other words, each subset of wavelength bands may include (or consist of) light in one or more wavelength bands of the plurality of wavelength bands (and there is no light at a wavelength outside the one or more wavelength bands). Different subsets of wavelength bands may be at least partially different, such as partially different or completely different. If they share at least one wavelength band included in both subsets, the subsets of two wavelength bands are partially different. If the subsets of two wavelength bands do not have a shared wavelength band, that is, the wavelength bands of the two subsets do not overlap, they are completely different. In an embodiment, the first and second subsets of wavelength bands may be partially different or completely different.

[0028] For example, a combination of a first operating mode and a second operating mode is shown below, wherein the first and second subsets are completely different. The processing module 114 can be configured to control the optical transmission module 120 so that in one of the operating modes, for example, in the first operating mode, light in a wavelength band outside of a plurality of mutually separated wavelength bands is emitted by the optical transmission module 120. For example, this wavelength band can be represented as Figure 3 and Figure 4 The wavelength band “0” in the image processing module 114 is a wavelength band “0” in the image processing module 114 . Since this wavelength band is not one of the multiple mutually separated wavelength bands, it can be used only for fluorescence imaging, that is, fluorescence imaging is performed at each of the multiple mutually separated wavelength bands. Additionally (or alternatively), the processing module 114 can be configured to control the light emitting module 120 so that in one of the operating modes, for example, in the second operating mode, light is emitted by the light emitting module 120 at each of the multiple mutually separated wavelength bands. For example, in this operating mode, reflectance imaging can be performed at each of the multiple mutually separated wavelength bands. These two operating modes represent edge cases, where reflectance imaging or fluorescence imaging can be performed simultaneously at each of the multiple mutually separated wavelength bands (see Figure 4 460 and 480).

[0029] In some operating modes, a hybrid mode may be applied, in which one or more wavelength bands of a plurality of mutually separated wavelength bands are used for reflective imaging, and one or more other wavelength bands are used for fluorescence imaging. For example, a plurality of mutually separated wavelength bands may include a third subset of wavelength bands and a fourth subset of wavelength bands. The third and fourth subsets may be disjoint and may include all of the plurality of mutually separated wavelength bands. In other words, each of the plurality of different wavelength bands may be part of the third or fourth subset. In some embodiments, the third or fourth subset may be used in combination with wavelength bands outside the plurality of mutually separated wavelength bands. The processing module 114 may be configured to control the light emitting module 120 so that in one of the operating modes, for example, in the first operating mode or in the third operating mode, the light emitting module 120 emits (only) light in the third subset of wavelength bands, and in one of the operating modes, for example, in the second or fourth operating mode, the light emitting module 120 emits (only) light in the fourth subset of wavelength bands. In some embodiments, each of the third and fourth subsets of wavelength bands may include two or more wavelength bands, so that different wavelength bands can be recorded simultaneously.

[0030] In some embodiments, the first subset may be equal to or include the third subset, and / or the second subset may be equal to or include the fourth subset of wavelength bands. In other words, the first subset of the plurality of wavelength bands may include (or correspond to) a subset of the third wavelength band of the plurality of mutually separated wavelength bands. The second subset of the plurality of wavelength bands may include a subset of the fourth wavelength band of the plurality of mutually separated wavelength bands. This may provide an association between the first / third and second / fourth subsets of wavelength bands, which may be used to perform imaging using one or more imaging sensor modules.

[0031] In summary, four different operating modes can be distinguished, in which the optical transmission module is controlled to transmit:

[0032] (1) All light in multiple mutually separated wavelength bands

[0033] (2) Light in a wavelength band other than a plurality of mutually separated wavelength bands

[0034] (3) light in a subset of the third wavelength band, and optionally in wavelength bands outside of the plurality of mutually separated wavelength bands;

[0035] (4) Light in a subset of the fourth wavelength band, and optionally in wavelength bands outside of the plurality of mutually separated wavelength bands.

[0036] For example, the control module may be configured to apply two or more of the above operation modes, i.e., the first operation mode and the second operation mode may be selected from the above operation modes (1)-(4). In addition, a third operation mode or the third and fourth operation modes may be used. For example, when three operation modes are used, (3) may be used as the first operation mode, (4) may be used as the second operation mode, and (2) may be used as the third operation mode. When four operation modes are used, as Figure 4 As shown in , all operating modes (1) to (4) can be used.

[0037] Typically, each operation mode can be used to perform a single recording using one or more imaging sensor modules. For example, within one operation mode, all multiple mutually separated wavelength bands can be recorded using one or more imaging sensor modules (substantially) simultaneously, i.e. within at most one second (or at most 0.5 seconds, at most 0.3 seconds, at most 0.1 seconds). The different operation modes can be used in a sequential manner, i.e. one after another. For example, the first operation mode, the second operation mode and optionally the third operation mode and the fourth operation mode can be used in this order. Alternatively, other orders can be selected.

[0038] The microscope includes one or more imaging sensor modules 130a, 130b. The one or more imaging sensor modules are configured to independently sense light in a plurality of mutually separated wavelength bands in a plurality of wavelength bands. In other words, the one or more imaging sensor modules can be configured to sense light in a plurality of mutually separated wavelength bands such that each of the plurality of mutually separated wavelength bands is distinguishable (or output separately) in the output of the one or more imaging sensor modules. For example, the one or more imaging sensor modules can output a separate sensor measurement for each of the plurality of mutually separated wavelength bands.

[0039] In some embodiments, Figure 1b As shown in , the one or more imaging sensor modules may include a first imaging sensor module 130a and a second imaging sensor module 130b. Alternatively, as shown in FIG. 1 , the microscope 100 may include a single imaging sensor module 130a. The processing module 114 may be configured to use the first imaging sensor module and the second imaging sensor module to perform reflectance imaging and fluorescence imaging in each of a plurality of mutually separated wavelength bands based on light emitted in the first operating mode and the second operating mode. By using two sensor modules, each sensor module may only need to sense a subset of a plurality of mutually separated wavelength bands, thereby having higher flexibility and / or facilitating the construction of corresponding imaging sensor modules. For example, the first imaging sensor module and the second imaging sensor module may be configured to sense different wavelength bands in a plurality of mutually separated wavelength bands. Therefore, no circuit for redundantly measuring light in the same wavelength band is included, thereby having higher flexibility and / or facilitating the construction of individual imaging sensor modules. For example, each of the first imaging sensor module and the second imaging sensor module may be configured to sense only light in one or more wavelength bands that are not sensed by the corresponding other imaging sensor modules.

[0040] For example, the first imaging sensor module may be configured to sense light in (only) a subset of a third wavelength band of a plurality of mutually separated wavelength bands. The second imaging sensor module may be configured to sense light in (only) a subset of a fourth wavelength band of a plurality of mutually separated wavelength bands. The first imaging sensor module and the second imaging sensor module may together cover all of the plurality of mutually separated wavelength bands, e.g., without overlap in the measured wavelength bands. For example, the first imaging sensor module may be configured to output a separate sensor measurement for each wavelength band of the subset of the third wavelength band. The second imaging sensor module may be configured to output a separate sensor measurement for each wavelength band of the subset of the fourth wavelength band. In general, one or more imaging sensor modules may be imaging sensor modules based on APS (active pixel sensor) or CCD (charge coupled device). For example, in an APS-based imaging sensor module, light is recorded at each pixel by using a photodetector and an active amplifier of the pixel. APS-based imaging sensor modules are typically based on CMOS (complementary metal oxide semiconductor) or S-CMOS (Scientific CMOS) technology. In a CCD-based imaging sensor module, incident photons are converted into electronic charges at the semiconductor-oxide interface, which are then moved between capacitive bins in the imaging sensor module by the sensor imaging module's control circuitry to perform imaging.

[0041] The processing module is configured to perform reflectance imaging and fluorescence imaging in each of a plurality of mutually separated wavelength bands using one or more imaging sensor modules 130a, 130b based on the light emitted in the first operating mode and the second operating mode. In some embodiments, the light emitted in the first operating mode is used to perform reflectance imaging, while the light emitted in the second operating mode is used to perform fluorescence imaging in (all of) the plurality of mutually separated wavelength bands, for example, when the operating modes (1) and (2) described above are selected. For example, the processing module 114 can be configured to perform reflectance imaging in the first operating mode using the first imaging sensor module 130a and the second imaging sensor module 130b, and to perform fluorescence imaging in the second operating mode (or vice versa). This can produce a reflectance image and a fluorescence image acquired simultaneously, respectively, with a slight delay between the reflectance image and the fluorescence image. Alternatively, as described above, a hybrid mode can be used in which both performance and reflectance imaging are achieved in the first operating mode and the second operating mode. In this case, there may be a slight delay in the reflectance / fluorescence image, but it is not a general delay that is either one or the other. For example, processing module 114 may be configured to perform reflectance imaging in a first operating mode using first imaging sensor module 130a and to perform fluorescence imaging in a second operating mode. Processing module 114 may be configured to perform fluorescence imaging in a first operating mode using second imaging sensor module 130b and to perform reflectance imaging in a second operating mode (or vice versa).

[0042] As described above, both methods may be selected, for example, to increase the number of reflectance / fluorescence images acquired at each wavelength. For example, processing module 114 may be configured to perform reflectance imaging in a first operating mode and fluorescence imaging in a second operating mode using first imaging sensor module 130a and second imaging sensor module 130b, to perform reflectance imaging in a third operating mode and fluorescence imaging in a fourth operating mode using first imaging sensor module 130a, and to perform fluorescence imaging in a third operating mode and reflectance imaging in a fourth operating mode using second imaging sensor module 130b. In any case, the numbering of the operating modes is used only to distinguish between different operating modes and may be changed, for example, if another order of the operating modes is desired.

[0043] When two (or more) different imaging sensor modules are used, a polychromatic mirror may be used to distribute light from the organic tissue sample between the sensor modules. In general, a polychromatic mirror is an optical element that is configured to selectively reflect or receive light of multiple wavelength bands (hence the term "polychromatic" means "having multiple colors"). In this case, the microscope may include a polychromatic mirror 140 that is configured to separate light used by a first imaging sensor module from light used by a second imaging sensor module. For example, the polychromatic mirror 140 may be configured to separate light having a wavelength in one wavelength band of a subset of a third wavelength band from light having a wavelength in one wavelength band of a subset of a fourth wavelength band of a plurality of mutually separated wavelength bands. The polychromatic mirror may be configured and arranged so that light having a wavelength in one wavelength band of a subset of the third wavelength band is admitted (or reflected) by the polychromatic mirror and (re)directed to the first imaging sensor module. The polychromatic mirror may be configured and arranged so that light having a wavelength in one wavelength band of a subset of the fourth wavelength band is reflected (or admitted) by the polychromatic mirror and (re)directed to the second imaging sensor module. Alternatively, a semi-transparent mirror may be used instead of the polychromatic mirror, which is adapted to partially reflect light from the organic tissue sample regardless of the wavelength of the light so that the full spectrum is directed to both imaging sensor modules.

[0044] The interface 112 may correspond to one or more inputs and / or outputs for receiving and / or sending information within a module, between modules, or between modules of different entities, which may be digital (bit) values ​​according to a specified code. For example, the interface 12 may include an interface circuit configured to receive and / or send information. In an embodiment, one or more processing units, one or more processing devices, or any device for processing may be used to implement the processing module 114, such as a processor, a computer, or a programmable hardware component that can be operated with correspondingly adapted software. In other words, the functions of the processing module 114 described may also be implemented in software, which is then executed on one or more programmable hardware components. These hardware components may include a general-purpose processor, a digital signal processor (DSP), a microcontroller, etc.

[0045] In combination with the proposed concept or one or more examples described above or below (e.g. Figures 2 to 4 ) mentions more details and aspects of the microscope or the device for microscopy. The microscope or the device for microscopy may include one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

[0046] Figure 2 shows a microscope (e.g. Figure 1a1 and / or 1b) of the microscope 100). Although various aspects of the microscope have been described in the context of an apparatus, it is clear that these aspects also represent descriptions of corresponding methods, where properties or functional features of the apparatus correspond to method steps or features of method steps. The method includes controlling 210 a light emission module 120 so that light in a first subset of a plurality of wavelength bands is emitted to an organic tissue sample 150 in a first operating mode, and light in a second subset of a plurality of wavelength bands is emitted to the organic tissue sample 150 in a second operating mode. The first and second subsets of wavelength bands are at least partially different. The light emission module 120 is adapted to provide illumination of the organic tissue sample 150 in a plurality of wavelength bands. The method includes performing reflectance imaging and fluorescence imaging in each of a plurality of mutually separated wavelength bands using 220 one or more imaging sensor modules based on the light emitted in the first operating mode and the second operating mode. The one or more imaging sensor modules 130a, 130b are configured to independently sense light in a plurality of mutually separated wavelength bands that are subsets of the plurality of wavelength bands of the microscope 100. As described above, the features described in conjunction with the microscope 100 of FIG. 1a / b may also be applied to Figure 2 method.

[0047] In combination with the proposed concept or one or more examples described above or below (e.g. Figure 1a / b, 3 or 4) mention more details and aspects of the method. The method may include one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

[0048] At least some embodiments relate to the use of multiple imaging sensor modules to capture multiple images with different parameters, such as in a multi-sensor camera. At least some embodiments may be directed to improving the utilization of multiple imaging sensor modules in an imaging device such as a surgical microscope.

[0049] Some imaging devices may employ multiple imaging sensor modules, such as CCD, CMOS, and SCMOS. This is because it is useful to capture multiple images at the same time, each with different properties, i.e., reflective or fluorescent. For example, by capturing reflective and fluorescent images simultaneously, color and visible pseudo-color can be achieved using multiple images captured simultaneously. For example, in this case, each sensor may be specialized to capture one type of image without being able to change the function of the sensor. However, the functions served by some sensors are not always necessary. For example, when fluorescent imaging is not required, the dedicated sensor remains unused. In an embodiment, a method of reusing installed imaging sensor modules to perform different functions is presented.

[0050] Figure 3 An overview diagram of an optical imaging system (such as a microscope) is shown, including a light emitting module 120 configured to provide illumination for an object in a subset of wavelength bands 0 to 6, as shown in Figure 310 (where the x-axis represents wavelength and the y-axis represents amplitude), and two imaging sensor modules 130a, 130b for recording light from tissue that is separated at a polychromatic mirror 140. Figure 320 shows the sensitivity of sensor 130a, and Figure 330 shows the sensitivity of sensor 130b (where the x-axis represents wavelength and the y-axis represents sensitivity to incident light at each wavelength). Figure 3 As shown, sensor 130a is sensitive at wavelength bands 1, 3, and 5, while sensor 130b is sensitive at wavelength bands 2, 4, and 6. If the optical imaging system is used for medical applications, each wavelength band may indicate a specific type or tissue condition (e.g., different tissue types, healthy tissue, pathological tissue). Figure 3 The spectral wavelength bands of the multispectral imaging sensor can be shown. Figure 3 In FIG. 1 , the light emitting module 120 is configured to emit light in wavelength bands 1, 3, and 5 (matching the wavelength band of the sensor 130a). Figure 3 In FIG. 1 , sensor 130a receives the same wavelength band as the illumination light and can therefore detect reflected light. Sensor 130b has a sensitivity curve complementary to the illumination spectrum and therefore detects only fluorescence.

[0051] By changing the illumination spectrum, illuminating at different spectral wavelength bands, the sensors 130a, 130b can still sense the same spectral wavelength band, but the light in their wavelength bands will have different properties (reflection / fluorescence). For example, the illumination spectral wavelength band can be adjusted by adopting an interferometric multi-bandpass filter (at the light emitting module 120), or by using a multi-spectral light source composed of multiple (in this case: 7) monochromatic light sources, such as LEDs (light emitting diodes).

[0052] In embodiments, the use of an imaging sensor module may be improved by altering the light emitted toward tissue. Figure 4 Different illumination spectra that can be achieved using the light emitting module 120 are shown, and can result in different imaging modes at the imaging sensor modules 130a, 130b (i.e., the resulting properties of the measured spectral wavelength bands). Figure 4 In the table, the left side 410 represents the wavelength band 0 to 6 440 emitted by the optical transmission module. Figure 1aIn the terms related to wavelength band 0, light emitted in wavelength band 0 may be light in a wavelength band outside of a plurality of mutually separated wavelength bands. Wavelength bands 1 to 6 may correspond to a plurality of mutually separated wavelength bands. The right side 420 shows the imaging mode (R=reflection, F=fluorescence) used at the corresponding sensor 430 (A=130a; B=130b) when the tissue is illuminated using the spectrum shown in the left side 410. Figure 1a As shown in Figure 1a / b, the imaging sensor modules 130a, 130b cover (only) multiple mutually separated wavelength bands 1 to 6. Reference numerals 450 to 480 show different spectra that can be obtained in different operating modes of the light emitting module. For example, the spectrum of reference numeral 450 can correspond to the light emitted in operating mode (3) of Figure 1a / b, the spectrum of reference numeral 460 can correspond to the light emitted in operating mode (1) of Figure 1a / b, the spectrum of reference numeral 470 can correspond to the light emitted in operating mode (4) + wavelength band 0 outside of the multiple mutually separated wavelength bands, and the spectrum of reference numeral 480 can correspond to the light emitted in operating mode (2). The table below summarizes at reference numeral 410 the four different illumination spectra at reference numeral 420 (of which at least 460, 470 and 480 are previously unused), and whether reflection or fluorescence is used. Figure 4 In the example, the spectrum marked with reference numeral 450 produces three spectral wavelength bands measured in reflection and three spectral wavelength bands measured in fluorescence, the spectrum marked with reference numeral 460 produces six spectral wavelength bands measured in reflection, the spectrum marked with reference numeral 470 produces three spectral wavelength bands measured in reflection and three spectral wavelength bands measured in fluorescence, and the spectrum marked with reference numeral 480 produces six spectral wavelength bands measured in fluorescence.

[0053] In combination with the proposed concept or one or more examples described above or below (e.g. Figure 1a / b and / or 2) mentioned the combination Figure 3 and / or Figure 4 More details and aspects of the concepts shown. Figure 3 and / or Figure 4 The illustrated concepts may include one or more additional optional features corresponding to one or more aspects of the presented concepts or one or more examples described above or below.

[0054] Some embodiments relate to a microscope comprising the Figure 4 Alternatively, the microscope may be a combination of FIG. 1 to FIG. Figure 4 One or more of the described systems are part of or connected to the system. Figure 5A schematic diagram of a system 500 configured to perform the methods herein is shown. System 500 includes a microscope 510 and a computer system 520. Microscope 510 is configured to capture images and is connected to computer system 520. Computer system 520 is configured to perform at least a portion of the methods described herein. Computer system 520 may be configured to execute a machine learning algorithm. Computer system 520 and microscope 510 may be separate entities, but may also be integrated in a common housing. Computer system 520 may be part of a central processing system of microscope 510 and / or computer system 520 may be part of a subcomponent of microscope 510, such as a sensor, participant, camera, or lighting unit of microscope 510.

[0055] The computer system 520 can be a local computer device (e.g., a personal computer, laptop, tablet, or mobile phone) having one or more processors and one or more storage devices or can be a distributed computer system (e.g., a cloud computing system having one or more processors and one or more storage devices distributed in different locations, for example, at a local client and / or one or more remote server farms and / or data centers). The computer system 520 may include any circuit or combination of circuits. In one embodiment, the computer system 520 may include one or more processors that may be of any type. A processor as used herein may mean any type of computing circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computer (CISC) microprocessor, a reduced instruction set computer (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), such as a microscope or a microscope component (e.g., a camera), or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 520 may be custom circuits, application specific integrated circuits (ASICs), or such as, for example, one or more circuits (such as communication circuits) used in wireless devices such as mobile phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 520 may include one or more storage devices, which may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard disk drives, and / or one or more drives that handle removable media, such as compact disks (CDs), flash memory cards, digital video disks (DVDs), etc. The computer system 520 may also include a display device, one or more speakers, and a keyboard and / or controller, which may include a mouse, trackball, touch screen, voice recognition device, or any other device that allows a system user to input information to and receive information from the computer system 520.

[0056] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0057] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, the aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as a processor, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some of the most important method steps may be performed by such a device.

[0058] According to certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be performed using a non-temporary storage medium, such as a digital storage medium, for example a floppy disk, DVD, Blu-ray, CD, ROM, PROM and EPROM, EEPROM or FLASH memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to perform the corresponding method. Therefore, the digital storage medium may be computer readable.

[0059] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0060] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, when the computer program product runs on a computer, the program code is operative for performing one of the methods.For example, the program code can be stored on a machine readable carrier.

[0061] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0062] In other words, therefore, an embodiment of the inventive method is a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0063] Therefore, a further embodiment of the present invention is a storage medium (or a data carrier, or a computer-readable medium) comprising a computer program stored thereon for performing one of the methods herein when executed by a processor. The data carrier, the digital storage medium or the recorded medium is typically tangible and / or non-transitory. A further embodiment of the present invention is an apparatus as described herein, comprising a processor and a storage medium.

[0064] A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection, for example via the Internet.

[0065] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0066] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods herein.

[0067] A further embodiment according to the invention comprises an apparatus or system configured to transmit (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. For example, the receiver may be a computer, a mobile device, a memory device, etc. For example, the apparatus or system may comprise a file server for transmitting the computer program to the receiver.

[0068] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all functions of the methods described herein. In some embodiments, a field programmable gate array can collaborate with a microprocessor to perform one of the methods described herein. Typically, these methods are preferably performed by any hardware device.

[0069] Reference numerals list

[0070] 100 Microscope

[0071] 110 Device for microscope

[0072] 112 Interface

[0073] 114 Processing Module

[0074] 120 optical transmitter module

[0075] 130a Imaging Sensor Module

[0076] 130b Imaging Sensor Module

[0077] 140 Polychromatic Mirror

[0078] 150 organic tissue samples

[0079] 210 Control optical transmission module

[0080] 220 Using one or more imaging sensor modules

[0081] 310 Lighting Diagram

[0082] 320 Sensor sensitivity diagram for sensor A

[0083] 330 Sensor sensitivity diagram for sensor B

[0084] 410 wavelength band of light emitted by the optical transmission module

[0085] 420 Imaging mode of sensor A / B in wavelength band

[0086] 430 Sensor A or B

[0087] 440 wavelength band

[0088] 450 Spectra and corresponding imaging modes

[0089] 460 Spectra and corresponding imaging modes

[0090] 470 Spectra and corresponding imaging modes

[0091] 480 Spectra and corresponding imaging modes

[0092] 500 System

[0093] 510 Microscope

[0094] 520 Computer Systems

Claims

1. A microscope (100; 510), comprising: A light emitting module (120) for providing illumination for an organic tissue sample (150) in a plurality of wavelength bands; a first imaging sensor module (130a) and a second imaging sensor module (130b) configured to independently sense light in a plurality of mutually separated wavelength bands among the plurality of wavelength bands, wherein the first imaging sensor module is configured to sense light in a third wavelength band subset of the plurality of mutually separated wavelength bands, and wherein the second imaging sensor module is configured to sense light in a fourth wavelength band subset of the plurality of mutually separated wavelength bands, wherein each of the third wavelength band subset and the fourth wavelength band subset includes two or more wavelength bands; The processing module (114) is configured to: The light emitting module (120) is controlled so that light in a first subset of the plurality of wavelength bands is emitted to the organic tissue sample (150) in a first operation mode, and light in a second subset of the plurality of wavelength bands is emitted to the organic tissue sample (150) in a second operation mode, the first subset and the second subset of wavelength bands being at least partially different, and based on the light emitted in the first operation mode and the second operation mode, reflectance imaging and fluorescence imaging are performed in each of the plurality of mutually separated wavelength bands using the first imaging sensor module and the second imaging sensor module.

2. The microscope according to claim 1, wherein: The processing module (114) is configured to perform reflectance imaging in the first operating mode and to perform fluorescence imaging in the second operating mode using the first imaging sensor module (130a).

3. The microscope according to claim 1 , wherein: The processing module (114) is configured to perform fluorescence imaging in the first operating mode and reflectance imaging in the second operating mode using the second imaging sensor module (130b).

4. The microscope according to claim 1 or 2 comprises a polychroic mirror (140), which is configured to separate light having a wavelength in one wavelength band in the third wavelength band subset from light having a wavelength in one wavelength band in the fourth wavelength band subset of the plurality of mutually separated wavelength bands.

5. The microscope according to claim 1 or 2, wherein: The first subset of the plurality of wavelength bands includes the third subset of the plurality of mutually separated wavelength bands, and wherein the second subset of the plurality of wavelength bands includes the fourth subset of the plurality of mutually separated wavelength bands.

6. The microscope according to claim 1 or 2, wherein: The first imaging sensor module and the second imaging sensor module are configured to sense different wavelength bands among the plurality of mutually separated wavelength bands.

7. The microscope according to claim 1 or 2, wherein: The processing module (114) is configured to control the light emitting module (120) so that in one of the operation modes, light in a wavelength band other than the plurality of mutually separated wavelength bands is emitted by the light emitting module (120).

8. The microscope according to claim 1 or 2, wherein: The processing module (114) is configured to control the light emitting module (120) so that in one of the operation modes, light at each of the plurality of mutually separated wavelength bands is emitted by the light emitting module (120).

9. The microscope according to claim 1 or 2, wherein: The light emitting module (120) is a multi-spectral light emitting module (120) configured to emit a plurality of wavelength bands in a mutually separated manner.

10. The microscope according to claim 9, wherein: The optical transmission module (120) is configured to transmit the plurality of wavelength bands using two or more optical filters in a mutually separated manner.

11. The microscope according to claim 9, or wherein: The light emitting module (120) is configured to emit the plurality of wavelength bands in a mutually separated manner using a plurality of separate light sources.

12. A device (110) for a microscope (100; 510), the device comprising: Interface (112) for: communicating with a light emitting module (120) for providing illumination of an organic tissue sample (150) in a plurality of wavelength bands, and communicating with a first imaging sensor module (130a) and a second imaging sensor module (130b), the first imaging sensor module (130a) and the second imaging sensor module (130b) being configured to independently sense light in a plurality of mutually separated wavelength bands among the plurality of wavelength bands of the microscope, wherein the first imaging sensor module is configured to sense light in a third wavelength band subset of the plurality of mutually separated wavelength bands, and wherein the second imaging sensor module is configured to sense light in a fourth wavelength band subset of the plurality of mutually separated wavelength bands, wherein each of the third wavelength band subset and the fourth wavelength band subset includes two or more wavelength bands; as well as The processing module (114) is configured to: The light emitting module (120) is controlled so that light in a first subset of the plurality of wavelength bands is emitted to the organic tissue sample (150) in a first operation mode, and light in a second subset of the plurality of wavelength bands is emitted to the organic tissue sample (150) in a second operation mode, the first subset and the second subset of wavelength bands being at least partially different, and based on the light emitted in the first operation mode and the second operation mode, reflectance imaging and fluorescence imaging are performed in each of the plurality of mutually separated wavelength bands using the first imaging sensor module and the second imaging sensor module.

13. A method for use with a microscope (100; 510), the method comprising: controlling (210) a light emission module (120) such that in a first operating mode light in a first subset of a plurality of wavelength bands is emitted to an organic tissue sample (150), and in a second operating mode light in a second subset of the plurality of wavelength bands is emitted to the organic tissue sample (150), the first and second subsets of wavelength bands being at least partially different, the light emission module (120) being adapted to provide illumination of the organic tissue sample (150) in the plurality of wavelength bands, and performing reflectance imaging and fluorescence imaging in each of a plurality of mutually separated wavelength bands using (220) a first imaging sensor module (130a) and a second imaging sensor module (130b) based on light emitted in a first operating mode and a second operating mode, The first imaging sensor module (130a) and the second imaging sensor module (130b) are configured to independently sense light in the plurality of mutually separated wavelength bands, wherein the first imaging sensor module is configured to sense light in a third wavelength band subset of the plurality of mutually separated wavelength bands, and wherein the second imaging sensor module is configured to sense light in a fourth wavelength band subset of the plurality of mutually separated wavelength bands, Each of the third wavelength band subset and the fourth wavelength band subset includes two or more wavelength bands.

14. A computer program having a program code for performing the method according to claim 13 when the computer program is executed on a processor.