Method, processor and medical observation device using two color images and color cameras for fluorescent and white light

Abstract

The present application relates to an image processor (170) and a computer-implemented image processing method for generating a digital output color image (160) of an object (106) using a medical viewing device (100), such as a microscope or endoscope. The method comprises the following steps: recording or retrieving a digital white light color image (114) of the object (106) in a first imaging spectrum (202) using a white light color camera (110); recording or retrieving a digital fluorescent color image (112) of the object (106) in a second imaging spectrum (222) using the fluorescent color camera (111), the second imaging spectrum (222) overlapping the fluorescent emission spectrum (226) of the at least one fluorophore (116), the second imaging spectrum (222) being different from the first imaging spectrum (202), the first and second imaging spectra (202, 222) both overlapping the visible spectrum (212); and generating a digital output color image (160) by combining the digital fluorescent color image (112) and the digital white color image (114). An image processor (170) is configured to perform these steps. The application further relates to a medical viewing device (100) comprising such an image processor (170) and to a method of operating such a device comprising the above computer-implemented method.

Description

Technical Field

[0001] The claimed subject matter relates to a computer-implemented image processing method and to an image processor for generating a digital output color image of an object using a medical viewing device such as a microscope or an endoscope. The claimed subject matter also relates to a method for operating such a medical viewing device and to the medical viewing device itself. Background Art

[0002] In existing medical observation devices, fluorescence images and white light images are recorded by a single camera. However, the problem is that the fluorescence intensity is much lower than the white light reflectance. Therefore, the recorded fluorescence is difficult to detect because it is drowned out by the white light reflectance.

[0003] Other medical viewing devices use a grayscale camera to record fluorescence, and a color camera to record white light images. In such a setup, the fluorescence image and the white light image are digitally mixed, and the fluorescence is labeled in pseudo-color. Here, the disadvantage is that the pseudo-colors cannot accurately represent the colors of the fluorescence as they would be perceived by the human eye. Summary of the invention

[0004] Therefore, there is a need to provide a medical observation device that generates natural-looking images of the fluorescence spectrum and reflectance spectrum of an object, that is, true color images.

[0005] This need is addressed by a computer-implemented image processing method, in particular using a medical observation device such as a microscope or an endoscope, for generating a digital output color image of an object, the method comprising the following steps: retrieving a digital white light color image of the object recorded in a first imaging spectrum, for example using a white light color camera; retrieving a digital fluorescence color image of the object recorded in a second imaging spectrum, for example using a fluorescence color camera, the second imaging spectrum overlapping with the fluorescence emission spectrum of at least one fluorophore, the second imaging spectrum being different from the first imaging spectrum, and both the first and second imaging spectra overlapping with the visible spectrum; and generating a digital output color image by combining the digital fluorescence color image and the digital white light color image.

[0006] This need is further addressed by an image processor for a medical observation device such as a microscope or an endoscope, the image processor being configured to: retrieve a digital white light color image of an object recorded in a first imaging spectrum, for example using a white light color camera; retrieve a digital fluorescence color image of an object recorded in a second imaging spectrum, for example using a fluorescence color camera, the first imaging spectrum overlapping with a fluorescence emission spectrum of at least one fluorophore, the first imaging spectrum being different from the second imaging spectrum, and both the first and second imaging spectra overlapping with the visible spectrum; and generate a digital output color image by combining the digital fluorescence color image and the digital white light color image.

[0007] Finally, a portion of the spectrum of the digital output color image generated by the medical viewing device is recorded by using a fluorescent color camera in the medical viewing device, which also includes a white light color camera configured to record another portion of the spectrum of the digital output color image.

[0008] By using a color camera for both fluorescent and white light images, a true color output image with more accurate colors can be provided.

[0009] The above solution can be further improved by the following features, which can be added and combined independently of each other and each feature has its own beneficial technical effect.

[0010] Each of the following features can be used to improve one of the methods described above and / or to improve one of the devices described above, regardless of whether a particular feature is mentioned only in relation to the method or in relation to the device.

[0011] The computer-implemented image processing method may include the step of demosaicing at least one of the digital white light color image and the digital fluorescent color image. By demosaicing, color artifacts may be avoided.

[0012] The computer-implemented image processing method may further include the step of normalizing at least one of the digital white light color image and the digital fluorescent color image. Normalization facilitates common processing of the digital white light color image and the digital fluorescent color image.

[0013] The computer-implemented image processing method may further include the step of registering at least one of the digital white light color image and the digital fluorescent color image. By registering, image features present in the digital white light color image and the digital fluorescent color image are represented in each image at the same size and orientation. After the registration, the digital white light color image and the digital fluorescent color image match each other.

[0014] The image processor may be configured to perform any of the above processing steps.

[0015] The fluorescence color camera is not limited to recording only fluorescence images. In one embodiment, the fluorescence color camera may record additional digital white light color images that may supplement the digital white light color images recorded by the white light color camera. If the second imaging spectrum needs to be changed, i.e. the fluorescence color camera is required to record images in a different second imaging spectrum, the optical setup may need to be changed, for example by requiring replacement of one or more filters. This may be the case if different fluorophores are used in the object.

[0016] According to another advantageous embodiment of the computer-implemented image processing method and / or the image processor, the first imaging spectrum and the second imaging spectrum are complementary to each other. In such a configuration, crosstalk between fluorescence and white light is avoided and an accurate reproduction of the fluorescence of at least one fluorophore on the one hand and the reflectivity of the object on the other hand is maintained. Further, by having complementary spectra in the digital fluorescence color image and the digital white light color image, respectively, a common processing of these two images to form a multispectral image is facilitated.

[0017] The digital white light color image may be represented in a first color space using at least three first color bands.

[0018] The digital fluorescent color image can be represented in a second color space comprising at least three second color bands. The first and second color spaces are preferably identical, so that the first and second color bands are also identical. However, this is not necessarily the case. The first color space and the second color space can also be different.

[0019] The digital output color image may be generated in a third color space comprising at least three third color bands. The third color space may be the same color space as the first color space and / or the second color space, or a different color space.

[0020] Any of the first, second and / or third color spaces may be an RGB color space, which includes three color bands R, G, B, or any other color space, such as a multispectral color space or a hyperspectral color space, i.e., using more than three color bands. The expression "color band" is used in the text as a synonym for "color space coordinates". The same color is represented by different color space coordinates in different color spaces.

[0021] According to one embodiment of a computer-implemented image processing method and / or image processor, a first color band and a second color band are mapped to a third color band of a digital output color image using a linear transformation. The linear transformation may include a color conversion matrix or a linear function. In order to map the first and second color bands to the third color band, a color conversion matrix may be applied to the first color band and the second color band. Preferably, each color band of the first color band and each color band of the second color band are input to the linear transformation as separate color bands. Specifically, each color band of the first color band and each color band of the second color band may be input to the linear transformation simultaneously.

[0022] The color conversion matrix or linear function may reside in the memory of the medical viewing device or image processor.

[0023] The color conversion matrix may have a dimension that is a number X in one direction multiplied by a number Y in another direction. The number X may be the sum of the number of first color bands (i.e., the number of color bands in the first color band) and the number of second color bands (i.e., the number of color bands in the second color band). In other words, one dimension of the color conversion matrix may be the sum of the dimension of the color space of the digital fluorescent color image and the dimension of the color space of the digital white light color image. The number Y may be the number of color bands in the third color space. In other words, another dimension of the color conversion matrix may correspond to the dimension of the color space of the digital output color image. For example, if the first, second, and third color spaces are each RGB color spaces, the color conversion matrix has a dimension of 6×3 or 3×6.

[0024] Thus, the third color band, ie, the color band of the digital output color image, can be generated by a linear combination of the color band of the digital white light color image and the color band of the digital fluorescent color image.

[0025] According to one aspect, the digital fluorescent color image and the digital white light color image are processed together as a multispectral image, the color bands of the multispectral image being formed or composed of the complementary color bands of the digital fluorescent color image and the digital white light color image. This results in improved color reproduction, particularly if the fluorescent image represents reflected white light that has been recorded in a second imaging spectrum and can therefore complement the white light information in the white light color image. Using this approach, the white light information is contained in the color bands of the digital fluorescent color image and therefore has a finer color granularity than can be provided by the digital white light color image alone. If the digital fluorescent color image represents fluorescent emission, treating the digital white light color image and the digital fluorescent color image as a multispectral image represents a true representation of the object under reflected white light and fluorescence.

[0026] According to another advantageous embodiment, the first imaging spectrum may comprise a first sub-band in a color band of a color space and the second imaging spectrum may comprise a second sub-band in this color band, wherein the first and second sub-bands are preferably complementary to each other. The color space may be a color space of a digital white light color image, a color space of a digital fluorescent color image or, preferably, a color space of a digital output color image. Color bands with such subdivision facilitate a linear transformation for mapping the color bands of the fluorescent and white light color images to the color bands of the digital output color image. Preferably, a plurality of sub-bands of a color band together complete the respective color band. This ensures that no useful spectral information is lost.

[0027] Of course, there may be first and second sub-bands in more than one color band of the corresponding color space. The more sub-bands used in different color bands, the more accurate the spectral information recorded in the corresponding white light or fluorescent color image. The sub-bands in a color band may themselves be divided into multiple bands of different wavelengths separated from each other.

[0028] Preferably, the second imaging spectrum includes IR wavelengths. This allows fluorescent emissions near the infrared to be captured.

[0029] When performing the method for operating a medical viewing device, the above computer-implemented image processing method may be performed. The image processor as described above may be part of a medical viewing device such as a microscope or an endoscope, in particular a microscope or an endoscope configured for use in, for example, neurosurgery, plastic surgery and / or ophthalmic surgery. The microscope may also be a laboratory microscope used, for example, for biopsy.

[0030] The method for operating a medical viewing device may include the steps of recording a digital white light color image using a white light color camera and recording a digital fluorescent color image using a digital fluorescent color camera. The medical viewing device may include a digital fluorescent color camera and a white light color camera.

[0031] To reduce post-processing, the fluorescence color camera and the white light color camera may have overlapping, coaxial fields of view, and / or preferably the same fields of view.

[0032] The medical viewing device may be a stereoscopic or monoscopic device. In one embodiment, a digital fluorescent color camera and a white light color camera may be provided in each stereoscopic channel. Alternatively, in a monoscopic medical viewing device, only a single white light color camera and a single digital fluorescent color camera may be provided.

[0033] In an alternative arrangement of a stereoscopic medical viewing device, a fluorescence color camera may be provided in one stereo channel, whereas a white light color camera may be provided in the other stereo channel. In such an arrangement, the fluorescence color camera may be configured to record a (white light) reflectance image in the second imaging spectrum. For the (white light) reflectance image, this arrangement provides a stereoscopic view, although strictly speaking the stereoscopic view is limited to the overlapping portion of the first and second imaging spectra, although this is rarely noticed by a human observer. In the case where fluorescence emissions are recorded, the fluorescence color camera provides a monoscopic view of the fluorescence emissions of the object, whereas the white light color camera provides a monoscopic view of the reflectance of the object.

[0034] The medical observation device may include an optical color separation component configured to separate light entering the color separation component into a first imaging spectrum and a second imaging spectrum. The color separation component may include an optical element, such as a beam splitter, in particular a dichroic beam splitter, and / or an optical filter, such as a fluorescence filter that blocks light other than fluorescent emission and a white light filter that blocks fluorescent emission.

[0035] The medical observation device may also include a lighting assembly, which is preferably adjustable, for example by including a plurality of differently colored light emitting diodes (LEDs) or organic light emitting diodes (OLEDs) or other light sources that emit light in a plurality of different spectral bands and can be turned on and off individually.

[0036] In order to facilitate post-processing and to ensure that the intensity values ​​of the recorded digital white light image and the digital fluorescent color image are comparable to each other, it is preferred that the digital fluorescent color image and the digital white light color image are recorded simultaneously and / or using the same exposure time and / or using the same gain and / or the same white balance and / or color compensation. In addition, the gain of the digital fluorescent color camera and the gain of the digital white light color camera can be kept at a constant ratio, but can also be allowed to adapt automatically. Among these possibilities, it is most preferred that the white light color camera and the fluorescent color camera are synchronized with respect to at least one of the exposure time and gain. The white light color camera and / or the fluorescent color camera are preferably a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) camera. The fluorescent camera and the white light color camera are preferably identical.

[0037] According to another aspect, the medical observation device may include a memory in which a plurality of different color conversion matrices are stored. Each color conversion matrix represents a different combination of a first imaging spectrum and a second imaging spectrum, i.e., if different fluorophores or a combination of different fluorophores are used, different sets of filters used in the color separation component are necessary. Examples of fluorophores used in this case are 5-aminolevulinic acid (ALA), fluorescein, and indocyanine green (ICG). Each of these fluorophores requires excitation at a different wavelength and also emits fluorescence at a different wavelength. Therefore, in a surgical environment using ICG, the first imaging spectrum and the second imaging spectrum will be different from a surgical environment using 5-ALA, requiring different color conversion matrices. Therefore, for each case of the fluorophore, the sub-bands of the first imaging spectrum and the second imaging spectrum in the color band of the corresponding color space are different.

[0038] Depending on the fluorophore used, the color conversion matrix may be automatically or manually selectable from a plurality of color conversion matrices. For example, the medical viewing device may be configured to automatically detect the configuration or setting of the color separation component and select the color conversion matrix according to this setting. For example, if the optical filter of the color separation component is replaced for use with 5-ALA, the medical viewing device may automatically select the color conversion matrix for this 5-ALA. Of course, this selection may also be done manually, or the automatic selection may be manually overridden.

[0039] The claimed subject matter also relates to a computer-readable medium and a computer program comprising instructions for causing a computer to perform the computer-implemented image processing of any of the above embodiments.

[0040] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0041] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0042] Hereinafter, the present invention is exemplarily described with reference to the embodiments and the accompanying drawings. The combination of features shown in the embodiments should not be considered as limiting. For example, features of the embodiments having technical effects as described above that are not required in a specific application may be omitted. On the contrary, if a technical effect associated with a specific feature is required in a specific application, the features described above that are not part of the embodiments described below may be added.

[0043] Throughout the description and the drawings, the same reference numerals are used for elements that correspond to each other in terms of function and / or structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In the accompanying drawings,

[0045] Figure 1 A schematic diagram of a medical observation device is shown;

[0046] Figure 2 A schematic diagram showing the combination of a digital white light color image and a digital fluorescent spectrum color image to obtain a digital output color image;

[0047] Figure 3 A schematic diagram showing jointly processing a digital white light color image and a fluorescent color image to obtain a digital output color image;

[0048] Figure 4 A schematic diagram showing a method for obtaining a digital output color image from a combination of a digital white light color image and a digital fluorescent spectrum color image; and

[0049] Figure 5 A schematic diagram of a system configured to perform a method of obtaining a digital output color image from a combination of a digital white light color image and a digital fluorescent spectrum color image is shown. DETAILED DESCRIPTION

[0050] First, refer to Figure 1 Exemplary embodiments of the present invention are described.

[0051] Figure 1 A medical observation device 100 is schematically shown. The medical observation device 100 may be a microscope or an endoscope, the difference between a microscope and an endoscope being that in an endoscope (not shown) the object 106 is observed by means of optical fibers which are brought into the vicinity of the object 106 to be investigated, for example by being inserted into a body containing the object, whereas in a microscope the objective lens 174 is directed directly onto the object. Figure 1 The medical observation device 100 is a microscope, but the following description is also applicable to an endoscope. The medical observation device 100 may be a medical observation device used in a surgical operation. The medical observation device 100 may also be a medical observation device used in a laboratory, such as a laboratory microscope. The object 106 to be studied may consist of or include a biological tissue 107.

[0052] The object 106 may include one or more fluorophores 116, 118. At least one fluorophore may be a fluorophore naturally contained in the object. For example, bones and blood contain fluorophores. At least one fluorophore may also be added to the object 106, for example by injecting it into the biological tissue 107. Examples of fluorophores that may be added to the object 106 are ICG, fluorescein and / or 5-ALA.

[0053] The medical viewing device 100 is a fluorescence imaging device. This means that the medical viewing device is configured to observe and preferably also stimulate the fluorescence of one or more fluorophores 116 , 118 .

[0054] The medical observation device 100 may be Figure 1 Thus, the medical observation device may include two identical subassemblies 101L and 101R for each of the two stereo channels. Since the two subassemblies 101L, 101R are identical in function and structure, the following description focuses on the right subassembly 101R, but is equally applicable to the left stereo channel 101L.

[0055] The medical observation device 100 may alternatively be a monoscopic device. In this case, only one of the two subassemblies 101L, 101R may be present. Therefore, for the monoscopic medical observation device 100, the following description also applies.

[0056] The medical viewing device 100 may be used to generate at least one digital white light color image 114, which preferably represents a reflected image of the object 106 in the visible light range. The visible light range or visible spectrum includes wavelengths from about 310 nanometers to about 1100 nanometers, or from about 380 nanometers to 750 nanometers, or from about 450 nanometers to about 700 nanometers. The fluorescence spectrum, or if more than one fluorophore is used, the fluorescence spectrum of the fluorophores 116, 118 is preferably omitted from the spectrum recorded in the digital white light color image 114. This ensures that if fluorescence is present, only reflected light is included in the digital white light color image 114. For example, if 5-ALA is used as a fluorophore, the fluorescence spectrum from about 625 nanometers to about 650 nanometers may not be recorded in the digital white light color image 114.

[0057] If specific wavelengths of light are used to excite fluorescence, spectra containing or consisting of those wavelengths may also not be recorded or represented in the digital white light color image 114. For example, if 5-ALA is used as a fluorophore, fluorescence may be excited by illuminating the object 106 with a wavelength between about 380 nanometers and about 450 nanometers. Different but known ranges for the excitation and emission spectra apply to fluorescein and ICG as well as other fluorophores than for 5-ALA.

[0058] By not recording the fluorescence excitation and emission spectra, digital white light color image 114 represents the reflectance of object 106, i.e., (as it would be seen by a human observer) represents a white light image of object 106. Therefore, digital white light color image 114 may be considered a natural color or true color image.

[0059] The digital imaging system 102 can further be used to generate a digital fluorescence color image 112 of the object 106. The digital fluorescence color image 112 represents the fluorescence emission of the one or more fluorophores 116, 118. Therefore, the digital fluorescence color image 112 preferably does not record the emission spectrum of the one or more fluorophores or any wavelengths outside of the emission spectrum.

[0060] Both the digital white light color image 114 and the digital fluorescent color image 112 are color images. They are recorded using at least three color bands or, equivalently, the primary colors of a color space. For example, the digital white light color image 114 and the digital fluorescent color image 112 may be recorded in an RGB color space using three primary colors or color bands R (red), G (green), B (blue). Alternatively, the digital white light color image 114 and the digital fluorescent color image 112 may be recorded in different color spaces, respectively, and / or represent multi-spectral or hyper-spectral color images. The digital white light color image 114 and the digital fluorescent color image 112 do not need to be recorded in the same color space, although this is preferred.

[0061] In a color space, such as the RGB color space, each color is represented by a triple of three integers, where each integer represents the intensity of one of the primary colors R, G, and B. Each number represents a color space coordinate, and each color band represents a coordinate axis. For example, the strongest red is represented by the triple [255, 0, 0]. The strongest green is represented by [0, 255, 0], and the strongest blue is represented by [0, 0, 255]. Therefore, the RGB color space is a three-dimensional space, and the CMYK (cyan, magenta, yellow, black) color space is a four-dimensional space. A color can be thought of as a point in the color space, and a vector such as [0, 0, 255] points to the point. A multispectral or hyperspectral color space with n color bands will correspondingly produce an n-dimensional color space, where each color is represented by an n-tuple color space coordinate.

[0062] The spectrum recorded in the digital white light color image 114 and the spectrum recorded in the digital fluorescent color image 112 are preferably complementary to each other, ie, do not overlap.

[0063] More specifically, the digital imaging system 102 may include a digital imaging system 102 for generating a digital fluorescent color image 112 and a digital white light color image 114. The digital imaging system 102 may include a white light color camera 110 and a fluorescent color camera 111.

[0064] The white light color camera 110 is configured to record a digital white light color image 114. In particular, the white light color camera 110 may be configured to generate a stream of digital white light color images 114 in the form of a video stream. The white light color camera 110 is preferably configured to record digital images of the entire visible spectrum of wavelengths referred to above. The white light color camera 110 may be a CCD, CMOS, or multi-spectral or hyper-spectral camera.

[0065] The fluorescence color camera 111 is configured to record a digital fluorescence image 112. In particular, the fluorescence camera 111 may be configured to generate a stream of digital fluorescence color images 112 in the form of a video stream. The fluorescence color camera 111 may be configured to record the digital fluorescence color image 114 only in the fluorescence spectrum or fluorescence spectrum of at least one fluorophore 116, 118. The fluorescence color camera 111 may be configured to record the digital fluorescence image only in one or more narrow bands of light. These narrow bands should overlap with the fluorescence spectrum or spectrum of the one or more fluorophores 116, 118, the fluorescence of which is to be recorded. Preferably, the fluorescence spectra of the fluorophore 116 and the second fluorophore 118 are at least partially separated, preferably completely separated, so that the fluorescence camera 111 can record light in two separate fluorescence bands spaced apart from each other.

[0066] The fluorescence color camera 111 may be a CCD, CMOS or a multi-spectral or hyper-spectral camera. Preferably, the white light color camera 110 and the fluorescence color camera 111 are of the same type, although this is not necessary.

[0067] The respective fields of view 184 of the cameras 110, 111 are preferably aligned or even coincident and coaxial. Therefore, it is preferred that the cameras 110, 111 provide the same field of view 184 with the same viewing angle and focal length. This results in the same representation of the object 106 in the images 112, 114 generated by the different cameras 110, 111. The cameras 110, 111 may use the same objective lens 174.

[0068] If a match of the field of view and viewing angle cannot be generated optically, it may be generated by image processing that applies a matching or registration procedure to the digital images 112 , 114 , as explained further below.

[0069] Preferably, the two cameras 110, 111 are operated synchronously. Specifically, the exposure time may be synchronized. Therefore, the medical observation apparatus 100 may be configured to generate the digital white light color image 114 and the digital fluorescence image 112 simultaneously.

[0070] Preferably, the gains of the two cameras 110, 111 are synchronized, i.e. adjusted simultaneously in both cameras 110, 111. Furthermore, the ratio of the gain applied in camera 110 to the gain applied in camera 111 may be constant, even if the gain is changed. Gamma correction and color adjustment or white balance may be turned off or left unchanged.

[0071] Any of the above measures may facilitate comparison, joint processing, and / or combination of the two images 112 , 114 .

[0072] In order to separate the spectrum recorded in the digital white light color image 114 from the spectrum recorded in the digital fluorescent color image 112, that is, to separate the reflection spectrum from the fluorescent spectrum, an optical color separation component 176 may be provided. The color separation component 176 may include optical elements, such as a beam splitter 192, which may be dichroic. The color separation component 176 may further or alternatively include an optical white light filter 188 and / or an optical fluorescence filter 190.

[0073] The fluorescence filter 190 is preferably configured to transmit light in the fluorescence spectrum or frequency spectrum of the one or more fluorophores 116, 118 and to block light outside of the fluorescence spectrum or frequency spectrum.

[0074] The fluorescence filter 190 can be configured as a bandpass filter including one or more passbands. Each passband should overlap with the fluorescence emission spectrum of the corresponding fluorophore 116, 118, the fluorescence of which is to be recorded. Since the fluorescence filter 190 is located in the light path between the beam splitter 192 and the fluorescence camera 111, only the wavelengths in the passband of the fluorescence filter 190 are transmitted to the fluorescence camera 111.

[0075] The white light filter 188 is preferably configured to block light in the fluorescence spectrum or frequency spectrum of one or more fluorophores 116, 118. The white light filter 188 may also be configured to block light in the fluorescence excitation spectrum.

[0076] The white light filter 188 is preferably configured as a band stop filter, the stop band of which corresponds to or at least includes the pass band of the fluorescence filter 190. The white light filter 188 is located in the light path between the beam splitter 192 and the white light camera 110. Therefore, the white light camera 110 only records wavelengths outside the stop band of the white light filter 188 and, therefore, also wavelengths outside the pass band of the fluorescence filter 190.

[0077] Either the white light filter 188 or the fluorescence filter 190 may be a tunable filter.

[0078] If the beam splitter 192 is a dichroic beam splitter, at least one of the filters 188, 190 can be omitted, since in this case the optical spectral filtering is already integrated in the dichroic beam splitter. The above description of passband and stopband should apply mutatis mutandis to the dichroic beam splitter 192.

[0079] The medical viewing device 100 may further include an illumination assembly 178 configured to illuminate the object 106, preferably through the objective lens 174, through which the imaging system 102 records at least one digital image 112, 114. The illumination assembly 178 may be configured to selectively generate white light, i.e., light uniformly distributed throughout the visible spectrum, and fluorescence excitation light, which contains light only in wavelengths that excite fluorescence of at least one fluorophore 116, 118. The illumination light generated by the illumination assembly 178 may be fed back into the objective lens 174 using an illumination beam splitter 180.

[0080] Depending on the fluorophore and its fluorescence-specific excitation spectrum, an illumination filter 179 may be provided. For example, if 5-ALA is used as the fluorophore, the illumination filter may have a transmittance of up to 90% to 98% for a wavelength of 425 nanometers, 0.5% to 0.7% for wavelengths between 450 nanometers and 460 nanometers, no higher than 0.1% for wavelengths between 460 nanometers and 535 nanometers, and almost zero transmittance for wavelengths above 535 nanometers.

[0081] Instead of or in addition to the illumination filter 179, the illumination assembly 178 may include an adjustable light source, including, for example, a plurality of differently colored LEDs or OLEDs.

[0082] The medical observation device 100 may further include an image processor 170. The image processor 170 may be a hardware module, such as a microprocessor, or a software module. The image processor 170 may also be a combination of a hardware module and a software module, such as by using a software module configured to run on a specific processor, such as a vector processor, a floating point graphics processor, a parallel processor, and / or multiple processors. The image processor 170 may be part of a general purpose computer 186, such as a personal computer (PC).

[0083] The image processor 170 is configured to retrieve the digital white light color image 114 and the digital fluorescence image 112. For example, the image processor 170 may be configured to retrieve the digital white light color image 114 and the digital fluorescence image 112 from the memory 194 and / or directly from the cameras 110, 111. The memory 194 may be part of the image processor 170, or reside elsewhere in the medical viewing device 100.

[0084] Image processor 170 is further configured to calculate digital output color image 160 from digital white light color image 114 and digital fluorescent image 112. Digital output color image 160 is a color image represented in a color space. The color space of the digital output color image may be different from the color space of any of digital white light color image 114 and digital fluorescent image 112. However, preferably, the color space of digital output color image 160 is the same as the color space of digital white light color image 114 and digital fluorescent image 112.

[0085] In particular, image processor 170 is configured to calculate digital output color image 160 from digital white light color image 114 and digital fluorescent image 112 by using color conversion function 140. Color conversion function 140 can be any type of function, such as a one-dimensional or n-dimensional interpolation function. Color conversion function 140 can be stored in memory 194.

[0086] Preferably, the color conversion function 140 is a linear transformation function. Specifically, the color conversion function 140 may be a color conversion matrix 142. One dimension of the color conversion matrix 142 may be the sum of the number of color bands in the digital fluorescent color image 112 and the number of color bands in the digital white light color image 114, i.e., the sum of the dimensions of the color space of the images 112, 114. Another dimension of the color conversion matrix 142 may correspond to the number of color bands in the digital output color image 160, i.e., the dimension of the color space of the digital output color image 160.

[0087] If all images 112, 114, 160 are in the RGB color space, the dimension of the color conversion matrix is ​​preferably 6x3 or 3x6.

[0088] The particular color conversion matrix 142 may be selectable from a set of color conversion matrices 142a-d stored in the medical viewing device 100 or the image processor 170, such as the memory 194. For example, different color conversion matrices 142a-d may be stored in the memory for different fluorophores.

[0089] For example, the first color conversion matrix 142a may represent the case where ICG is used as the fluorophore 116 in the object 106, the second color conversion matrix 142b may represent the case where 5-ALA is used as the fluorophore 118 in the object 106, and / or the third color conversion matrix 142c may represent the case where 5-ALA and ICG are simultaneously used as the fluorophores 116, 118 in the object 106. Similarly, additional color conversion matrices 142d may be used for other fluorophores or combinations of fluorophores or combinations of fluorophores. In its simplest form, the color conversion matrix 142 separately performs the addition of the intensities in each color band of the color space of the fluorescence color image and the white light color image.

[0090] Each different color conversion matrix 142 may represent a different filter setting for the color separation component 176. Since different fluorophores have different emission spectra, the wavelengths to be recorded by the white light color camera 110 and the fluorescence camera 111 are different for each fluorophore. Therefore, for each fluorophore, the color bands in the two cameras 110, 111 are used to a different extent, requiring a different calibration matrix for adjusting the relative intensities in the color bands of the images 112, 114.

[0091] The medical viewing device 100 can be adjusted to different fluorophores by reconfiguring the color separation component 176, for example by replacing its optical elements, such as filters 190 and / or 192, or the dichroic beam splitter 180, and by selecting a corresponding color conversion matrix 142. The selection of the color conversion matrix 142 can be done automatically according to the current settings of the color separation component 176 or manually by the user.

[0092] The digital output color image 160 may be displayed on a display 132 that is integrated with the medical viewing device 100. For example, the display 132 may be integrated into a viewing mirror or eyepiece 104 of the medical viewing device 100.

[0093] The medical viewing device 100 may include a direct light path 134 from the object 106 through the objective lens 174 to the eyepiece 104. In such a case, the display may be a semi-transparent display 132 located in the direct light path 134, or the display may be projected into the direct light path 134. A beam splitter 136 may be provided to split the light between the optical eyepiece 104 and the digital imaging system 102. In one embodiment, up to 80% of the light may be directed to the eyepiece 104.

[0094] Alternatively, the medical viewing device 100 may not have a direct optical path 134, but instead simply display images from an integrated display 132. As a further alternative, the medical viewing device may not have any display at all.

[0095] The medical viewing device 100 may include an output interface 172 to which one or more (external) displays 182 may be connected. To this end, the output interface 172 may include standardized connectors and data transmission protocols, such as Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Interactive Digital Video System (DVI), DisplayPort, Bluetooth, and / or others. The external display may be a monitor, 3D goggles, viewing glasses, etc. Any combination of external displays may be connected to the output interface 172.

[0096] The computer 186 or the image processor 170 is connected to the digital imaging system 102 using one or more data transmission lines 196. The data transmission line may be wired or wireless, or partially wired and partially wireless. The computer 186 and / or the image processor 170 may not be integrated entirely in the medical viewing device 100, but may be physically remote from the digital imaging system 102. To this end, the digital imaging system 102 and the computer 186 and / or the image processor 170 may be connected to a network, such as a local area network (LAN), a wireless local area network (WLAN), or a wide area network (WAN), and at least one display 182 is also connected to the network.

[0097] According to a modification, the medical observation device 100 can be stereoscopic, but only includes two cameras, one in each stereo channel. In one stereo channel, a fluorescence color camera 111 is used and is configured to selectively also record white light reflections, while in another stereo channel, a white light color camera 110 is used. If fluorescence is not used, this arrangement provides a stereoscopic white light color image, and if fluorescence is used, a monoscopic white light color image and a monoscopic fluorescence color image are provided. The above and following descriptions also apply to this configuration.

[0098] Figure 2 A quantitative example of a first imaging spectrum 202 is shown at reference numeral 200 as it may be recorded by a digital white light camera 110 and / or represented in a digital white light color image 114, respectively. The intensity I over wavelength / color λ is shown normalized. The first imaging spectrum 202 preferably extends at least across a visible spectrum 212.

[0099] By way of example only, the color space in which the first imaging spectrum 202 is recorded is an RGB color space having three primary colors or color bands 204, 206, and 208. One primary color 204 is blue, another primary color 206 is green, and the third primary color 208 is red. The sensitivity of the sensor of the white light color camera 110 in the different primary colors 204, 206, 208 is tuned to result in a sensitivity over the visible spectrum 212 that is as constant as possible.

[0100] If a color space other than RGB is used, the number, position and / or width of the color bands may be different. Apart from this, there are no major differences from the RGB color space.

[0101] The first imaging spectrum 202 does not include the fluorescence excitation light and the fluorescence emission spectrum of at least one fluorophore 116, 118. Therefore, the first imaging spectrum 202 may include at least one stop band 210, which coincides with the fluorescence emission of at least one fluorophore, the fluorescence of which is to be recorded by the fluorescence color camera 111. The stop band 210 is generated, for example, by the white light filter 188. The number, width and / or position of the stop bands 210 depends on the number and type of fluorophores to be observed in the object 106.

[0102] At reference numeral 220, a second imaging spectrum 222 is shown as it may be recorded by the fluorescence color camera 111 and / or represented in the digital fluorescence color image 112, respectively. Merely as an example, the color space in which the second imaging spectrum 222 is recorded is also the RGB color space. The sensitivity of the sensor of the fluorescence color camera 111 in the different primary colors 204, 206, 208 is tuned to produce a sensitivity that is as constant as possible over the visible spectrum 212.

[0103] The spectra 202, 222 need not be recorded in the same color space, although this is preferred.

[0104] The second imaging spectrum 222 may include one or more passbands 224. The number, position and / or width of the passbands depends on the number and type of fluorophores used. At least one passband 224 preferably corresponds to at least one stopband 210. At least one passband is generated, for example, by the fluorescence filter 190.

[0105] The first imaging spectrum 202 and the second imaging spectrum 222 are complementary to each other. They preferably complement each other to cover the entire or most of the visible spectrum 212.

[0106] Each passband 224 of the second imaging spectrum 222 preferably overlaps with the fluorescence emission spectrum 226, 228 of the fluorophores 116, 118 whose fluorescence is to be recorded, and overlaps with one or more primary colors 204, 206, 208 of the color spectrum. For example, the fluorescence emission spectrum 226 of one fluorophore 116 may overlap with all three primary colors 204, 206, 208. The sensor of the fluorescence camera 111 recording the color band 204 records only a small part 230 of the fluorescence emission spectrum 226. The sensor of the fluorescence camera 111 recording the color band 206 records a large part 232 of the fluorescence emission spectrum 226, whereas the sensor of the fluorescence camera 111 recording the color band 208 also records only a small part 236 of the fluorescence emission spectrum 226.

[0107] In contrast, fluorescence spectrum 228 , if present under certain circumstances, is only recorded by the sensor of fluorescence camera 111 that records color strip 208 .

[0108] Color conversion function 140 is applied to digital white light color image 114 and digital fluorescent color image 112 to obtain digital output color image 160, whose spectrum is in Figure 2 In essence, spectrum 252 of digital output image 160 results from a combination of first imaging spectrum 202 and second imaging spectrum 222 .

[0109] Higher accuracy can be achieved because individual spectra 202 and 222 are treated as a single, combined multispectral spectrum having a number of color bands corresponding to the sum of the color bands in first imaging spectrum 202 and second imaging spectrum 222. In other words, digital fluorescent color image 112 and digital white light color image 114 are processed together as a single (virtual) multispectral image.

[0110] This is referenced below Figure 3 To keep it simple, Figure 3 shows the RGB color space Figure 2Only individual passbands 224 or stopbands 210 are shown.

[0111] The digital white light color image 114 recorded in the first imaging spectrum 202 is composed of signals R1, G1, and B1, each representing the intensity I in a corresponding color band. Signal G1 is produced by two separate wavelength bands due to the stop band 210. However, the sensor recording G1 cannot distinguish between these two wavelength bands.

[0112] The digital fluorescent color image 112 recorded in the second imaging spectrum 222 is composed of signals R2, G2, B2 in each color band.

[0113] Because the first imaging spectrum 202 and the second imaging spectrum 222 are complementary to each other, each color band is divided into two signals, each signal comes from a different camera 110, 111, and represents or is equivalent to a sub-band of the color band. The color band 204 is divided into R1 and R2, respectively. The color band 206 is divided into G1 and G2, respectively, and the color band 208 is divided into B1 and B2, respectively. Preferably, there is no overlap between the various signals in the color band.

[0114] Therefore, even if the same type of color camera is used, the first imaging spectrum 202 and the second imaging spectrum 222 are recorded in the same color space, resulting in a multispectral image of the object 106, which in this case consists of six color bands R1, R2, G1, G2, B1, B2. This is schematically shown at reference numeral 300. It should be noted that this is independent of whether the fluorescence camera actually records fluorescence emission or reflection.

[0115] In other words, at least one color band 204, 206, 208 of the color space of at least the digital output color image 160 is subdivided into two sub-bands R1, R2, G1, G2, B1, B2, wherein one of the two sub-bands is included in the digital fluorescent color image 112 and the other of the two sub-bands is included in the digital white light color image 114. Preferably, the two sub-bands in the color band do not overlap. The two sub-bands in the color band are preferably complementary. Most preferably, the two sub-bands in the color band together at least nearly complete the corresponding color band. The sub-bands may include or consist of two separate spectral bands. Each sub-band may be considered to be a color band in itself.

[0116] The color conversion function 140 represents the subdivision of the color band into sub-bands R1, R2, G1, G2, B1, B2, which is determined by the stop band 210 and the pass band 224. Since the width and / or position of the sub-band determines how much light the corresponding camera 110, 111 will collect in that sub-band, the color conversion function 140 needs to be adjusted for each different filter setting of the color separation component 176.

[0117] Thus, the digital white light image 112 and the digital fluorescent image 114 may be processed together as an image composed of the union of the color bands in the two images 112 and 114. This provides improved color resolution.

[0118] Using color conversion matrix 142 , color conversion function 140 may be applied in the form of a linear transformation to generate a digital output color image 160 .

[0119] If the digital output color image 160 is represented in RGB color space, a 6×3 or 3×6 color conversion matrix can be applied to the combined digital fluorescent and white light images 112, 114 or their component signals or color space coordinates R2, G2, B2, R1, G1, B1 to obtain RGB signals R*, G*, B*, respectively. The matrix coefficients C11, C12, ..., C63 can be determined experimentally.

[0120] As a result of applying the color conversion function 302, an output spectrum 302 is obtained, which is represented by three RGB signals or color space coordinates.

[0121] The above transformation is applied to each image location or pixel.

[0122] Figure 4 A schematic representation of an imaging method is shown, which may be implemented as a computer-implemented method running on, for example, the image processor 170 .

[0123] In optional step 400, a digital white light color image 114 is recorded using, for example, a white light color camera 110. In optional step 402, a digital fluorescence color image 112 is recorded using, for example, a fluorescence camera 111. Steps 400 and 402 are optional, as the images 112, 114 can be retrieved from memory. As explained above, the cameras 110, 111 are synchronized with respect to exposure time and are locked to each other with respect to gain, i.e., maintain the same gain ratio. Gamma (γ) can be set to a fixed value, and all automatic color adjustments are preferably turned off.

[0124] In optional step 404, the respective images 112, 114 are demosaiced.

[0125] In optional step 406 , one or both of the images 112 or 114 are matched and registered so that the same image features are represented geometrically identically in each image 112 , 114 with respect to size and orientation.

[0126] At step 408 , a color conversion is performed. The images 112 , 114 may be linearly transformed, for example using the color conversion matrix 142 to obtain a digital output color image 160 .

[0127] At step 410, further post-processing may be performed. For example, the digital output color image 160 may be homogenized, its contrast may be enhanced, and / or a color space conversion such as from RGB to sRGB and / or gamma correction may be performed. At step 412, the digital output image 160 is displayed.

[0128] Some embodiments involve combining Figures 1 to 4 Alternatively, the microscope may be a combination of Figures 1 to 4 One or more of the described systems may be part of or connected to the system.

[0129] Figure 5 A schematic diagram of a system 500 configured to perform the methods described herein is shown. System 500 includes a microscope 510 and a computer system 520. Microscope 510 is configured to capture images and is connected to computer system 520. Computer system 520 is configured to perform at least part of the methods described herein. Computer system 520 can be configured to execute a machine learning algorithm. Computer system 520 and microscope 510 can be separate entities, but can also be integrated together in a common housing. Computer system 520 can be part of a central processing system of microscope 510, and / or computer system 520 can be part of a subcomponent of microscope 510, such as a sensor of microscope 510, an actor in reinforcement learning, a camera or a lighting unit, etc.

[0130] The computer system 520 may be a local computer device (e.g., a personal computer, laptop, notebook computer, or mobile phone) having one or more processors and one or more storage devices, or may be a distributed computer system (e.g., a cloud computing system having one or more storage devices and one or more processors distributed in different locations, for example, at a local client and / or one or more remote server farms and / or data centers). The computer system 520 may include any circuit or combination of circuits. In one embodiment, the computer system 520 may include one or more processors that may be of any type. As used herein, a processor may refer to any type of computing circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA) such as a microscope or a microscope component (e.g., a camera), or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 520 may be custom circuits, application specific integrated circuits (ASICs), etc., or the like, such as one or more circuits (such as communication circuits) for wireless devices (such as mobile phones, tablet computers, laptops, two-way radios, and similar electronic systems). The computer system 520 may include one or more storage devices, which may include one or more storage elements suitable for a particular application, such as a main memory in the form of a random access memory (RAM), one or more hard disk drives, and / or one or more drives for handling removable media (such as compact disks (CDs), flash memory cards, digital video disks (DVDs), etc.). The computer system 520 may also include a display device, one or more speakers, and a keyboard and / or controller, which may include a mouse, a trackball, a touch screen, a voice recognition device, or any other device that allows a system user to input information to and receive information from the computer system 520.

[0131] Some method steps or all method steps can be performed by (or using) hardware devices, such as processors, microprocessors, programmable computers or electronic circuits. In some embodiments, some one or more of the most important method steps can be performed by such devices.

[0132] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation may be performed using a non-transitory storage medium, such as a digital storage medium, such as a floppy disk, a digital video disk (DVD), a Blu-ray, a compact disk (CD), a read-only memory (ROM), a programmable read-only memory (PROM), and an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory, on which electronically readable control signals are stored, which cooperate (or are capable of cooperating) with a programmable computer system to perform the corresponding method. Therefore, the digital storage medium may be computer readable.

[0133] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0134] Generally, the embodiments of the present invention can be implemented as a computer program product with a program code, when the computer program product runs on a computer, the program code is operative for performing one of the methods.The program code may, for example, be stored on a machine readable carrier.

[0135] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0136] In other words, therefore, an embodiment of the inventive method is a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0137] Therefore, another embodiment of the present invention is a storage medium (or data carrier, or computer-readable medium), including a computer program stored thereon, which, when executed by a processor, is used to perform one of the methods described herein. The data carrier, digital storage medium or recorded medium is generally tangible and / or non-transient. Another embodiment of the present invention is a device comprising a processor and a storage medium as described herein.

[0138] A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection, for example via the Internet.

[0139] Further embodiments comprise a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0140] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0141] Another embodiment according to the invention comprises a device or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. For example, the receiver may be a computer, a mobile device, a storage device, etc. The device or system may, for example, comprise a file server for transmitting the computer program to the receiver.

[0142] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the method described herein. In some embodiments, the field programmable gate array can collaborate with a microprocessor to perform one of the methods described herein. Typically, the method is preferably performed by any hardware device.

[0143] Reference numerals:

[0144] 100 Medical observation device

[0145] 101L Left Channel Stereo Subassembly

[0146] 101R Right Channel Stereo Subassembly

[0147] 102 Digital Imaging System

[0148] 104 Eyepiece

[0149] 106 Objects

[0150] 107 Biological tissues

[0151] 110 White light color camera

[0152] 111 Fluorescent Color Camera

[0153] 112 Digital Fluorescent Color Image

[0154] 114 Digital white light color images

[0155] 116 Fluorophore

[0156] 118 Second Fluorophore

[0157] 132 Internal display

[0158] 134 Direct light path

[0159] 136 beam splitter

[0160] 140 color conversion function

[0161] 142 color conversion matrix

[0162] 142a-142d Different color conversion matrices

[0163] 160 digital output color images

[0164] 170 Image Processor

[0165] 172 output interface

[0166] 174 objective lens

[0167] 176 Color Separation Components

[0168] 178 Lighting components

[0169] 179 Lighting Filters

[0170] 180 illumination beam splitter

[0171] 182 Display

[0172] 184 Field of View

[0173] 186 Computer

[0174] 188 White Light Filter

[0175] 190 Fluorescence Filter

[0176] 192 Two-way beam splitter

[0177] 194 Memory

[0178] 196 data transmission lines

[0179] 200 Spectra

[0180] 202 First Imaging Spectrum

[0181] 204, 206, 208 primary colors or color ribbons

[0182] 210 Stopband

[0183] 212The visible spectrum or range of light

[0184] 220 Spectrum

[0185] 222 Second Imaging Spectrum

[0186] 224 passband

[0187] Fluorescence emission spectra of 226 fluorophores

[0188] 228 Fluorescence emission spectrum of another fluorophore

[0189] 230 portion of the fluorescence emission spectrum in the color band 204

[0190] 232 portion of the fluorescence emission spectrum in the color band 206

[0191] 236 The portion of the fluorescence emission spectrum in the color band 208

[0192] 250 Spectra

[0193] 252 Spectrum of digital output image

[0194] 300 A multi-spectral spectrum generated by the first imaging spectrum and the second imaging spectrum

[0195] 400 digital white light color images

[0196] 402 Recording Digital Fluorescent Color Images

[0197] 404 Demosaic

[0198] 406 Registration

[0199] 408 Color Conversion

[0200] 410 Post-processing

[0201] 412 Display

[0202] 500 system

[0203] 510 Microscope

[0204] 520 Computer Systems

[0205] λ wavelength

[0206] I Strength

[0207] C11, C12, ..., C63 Coefficients of the color conversion matrix

[0208] Example of RGB color space

[0209] Color space coordinates / (intensity) signals of R1, G1, B1, R2, G2, B2, R*, G*, B* in the corresponding color band or sub-band of the color space

Claims

1. A computer-implemented image processing method for generating a digital output color image (160) of an object (106), the method comprising the following steps: retrieving a digital white light color image (114) of the object (106) recorded in the first imaging spectrum (202); retrieving a digital fluorescent color image (112) of the object (106) recorded in a second imaging spectrum (222), the second imaging spectrum (222) overlapping with a fluorescence emission spectrum (226) of at least one fluorophore (116), the second imaging spectrum (222) being different from the first imaging spectrum (202), the first imaging spectrum (202) and the second imaging spectrum (222) both overlapping with a visible spectrum (212); and The digital output color image (160) is generated by combining the digital fluorescent color image (112) and the digital white light color image (114).

2. The computer-implemented image processing method according to claim 1, wherein the first imaging spectrum (202) and the second imaging spectrum (222) are complementary to each other.

3. The computer-implemented image processing method of claim 2, wherein representing the digital white light color image (114) in a first color space (RGB) using at least three first color bands (R, G, B); representing the digital fluorescent color image in a second color space including at least three second color bands (112); generating the digital output color image (160) in a third color space including at least three third color bands; mapping the first color band and the second color band to the third color band of the digital output color image using a linear transformation; as well as Each color band of the first color band and each color band of the second color band are input into the linear transformation as separate color bands.

4. The computer-implemented image processing method of claim 3, wherein The linear transformation comprises a color conversion matrix having dimensions of a number X times a number Y, The number X is the sum of the number of the first color bands and the number of the second color bands, The number Y is the number of color bands in the third color space.

5. A computer-implemented image processing method according to claim 3 or 4, wherein the first color space and the second color space are the same.

6. A computer-implemented image processing method according to claim 5, wherein the first imaging spectrum (202) comprises a first sub-band (R1, G1, B1) in a color band (204, 206, 208) of a color space (RGB), the second imaging spectrum (222) comprises a second sub-band (R2, G2, B2) in this color band, and the first sub-band and the second sub-band are complementary to each other.

7. The computer-implemented image processing method of claim 6, wherein a color band of a third color space includes the first sub-band and the second sub-band.

8. The computer-implemented image processing method of any one of claims 1 to 7, wherein the second imaging spectrum (222) comprises near infrared spectrum (NIR) wavelengths.

9. The computer-implemented image processing method according to any one of claims 1 to 8, wherein: The digital fluorescent color image (112) and the digital white light color image (114) are jointly processed as a digital multispectral image including at least five color bands.

10. A method (100) for operating a medical observation device (100), in particular a microscope or an endoscope, the method comprising the following steps: Execute the computer-implemented image processing method according to any one of claims 1 to 9; recording the digital white light color image (114) using a white light color camera (110); and The digital fluorescent color image (112) is recorded using a fluorescent color camera (111).

11. The method of claim 10, wherein the white light color camera (110) and the fluorescent color camera (111) are synchronized with respect to at least one of exposure time and gain.

12. An image processor (170) for a microscope or an endoscope, the image processor (170) being configured to: retrieving a digital white light color image (114) of the object (106) recorded in the first imaging spectrum (202); retrieving a digital fluorescent color image (112) of the object (106) recorded in a second imaging spectrum (222), the second imaging spectrum (222) overlapping with a fluorescence emission spectrum (226) of at least one fluorophore (116), the second imaging spectrum (222) being different from the first imaging spectrum (202), the first imaging spectrum (202) and the second imaging spectrum (222) both overlapping with the visible spectrum (222); and By combining the digital fluorescent color image (112) and the digital white light color image (114), a digital output color image (160) is generated.

13. A medical observation device (100), the medical observation device (100) such as a microscope or an endoscope, comprising An image processor (170) according to claim 12; a fluorescent color camera (111), the fluorescent color camera (111) being configured to record the digital fluorescent color image (112); as well as A white light color camera (110) is configured to record the digital white light color image (114).

14. In a medical observation device (100), a fluorescent color camera (111) is used to record a portion (222) of a spectrum (250) of a digital output color image (160) generated by the medical observation device (100), and the medical observation device (100) further includes a white light color camera (110), and the white light color camera (110) is configured to record another portion (202) of the spectrum (250) of the digital output color image (160).

15. A computer program product or a computer readable medium comprising instructions which, when a computer executes said program, cause the computer to perform the method of any one of claims 1 to 9.

Images