Medical imaging device and method for medical imaging

Abstract

The invention relates to a medical imaging device (410), comprising: an illumination unit (412) having at least one light source (414) that is configured to provide illumination light (416) to illuminate an object (418) to be imaged; and an image capture unit (420) that comprises at least one image capture sensor system (422) and at least one observation filter (424). The observation filter (424) is configured to block light in a first spectral range (426), which comprises visible light, and to transmit light in a second spectral range (428), which differs from the first spectral range (426). The image capture unit (430) is configured to record, through the observation filter (424), fluorescence images (430) of the object (418) to be imaged, during the recording of which images, illumination light (432), the spectrum of which lies at least partly within the first spectral range (426), is used as excitation light and, during the recording of which images, furthermore light (434) which is emitted from the object (418) to be imaged and the spectrum of which lies at least partly within the second spectral range (428) is detected by the image capture sensor system (422). In addition, the image capture unit (430) is configured to record anatomy images (436) of the object (418) to be imaged, during the recording of which images, illumination light (438), the spectrum of which lies at least partly within the second spectral range (428), is used and, during the recording of which images, furthermore light (440) which is remitted from the object (418) to be imaged and the spectrum of which lies at least partly within the second spectral range (428) is detected by the image capture sensor system (422). The imaging device further comprises a display generation unit (442) which is configured to generate at least one display (444) from the fluorescence images (430) and the anatomy images (436). The invention also relates to a method for medical imaging, program code and a computer-readable medium (448).

Description

[0001] Medical imaging device and method for medical imaging

[0002] The invention relates to a medical imaging device, in particular an endoscope device, exoscope device and / or microscope device, a method for medical imaging, associated program code and an associated computer program product.

[0003] Imaging devices for performing fluorescence imaging are known from the prior art; they can acquire both fluorescence images and white-light images. Suitable excitation light is used to specifically excite fluorescent dyes or, where appropriate, natively occurring fluorescent substances, and to detect the emitted light and use it for imaging. To simultaneously display anatomical structures in the image area, a white-light image is often acquired in parallel or sequentially. Based on the white-light image, the user can assess whether the anatomical structure is being imaged. Fluorescence images and white-light images can also be superimposed, allowing the user to simultaneously perceive and analyze anatomical information and fluorescence information.

[0004] Imaging devices such as endoscopic or exoscopic devices that generate multispectral or hyperspectral images are also known from the prior art. Multispectral or hyperspectral images have, in addition to two spatial dimensions, such as a conventional camera image, a spectral dimension. The spectral dimension encompasses several spectral bands (wavelength bands). Multispectral and hyperspectral images differ essentially in the number and width of their spectral bands. Such systems can, in principle, also be suitable for fluorescence imaging.

[0005] Several imaging devices for generating such multispectral or hyperspectral images are known, particularly in the context of medical applications. For example, DE 20 2014 010 558 U1 describes a device for capturing a hyperspectral image of an examination area of ​​a body. The device includes an input lens for generating an image in an image plane and a slit-shaped aperture in the image plane for masking out a slit-shaped region of the image. The light passing through the aperture is spread out by a dispersive element and recorded by a camera sensor. As a result, the camera sensor can record a plurality of spectra, each with an associated spatial coordinate, along the longitudinal direction of the slit-shaped aperture.The described device is further configured to record additional spectra along the longitudinal direction of the slit-shaped aperture in a direction different from the longitudinal direction of the slit-shaped aperture. The method underlying this disclosure for generating multispectral or hyperspectral images is also known as the so-called pushbroom method.

[0006] In addition to the pushbroom method, there are other methods for generating multispectral or hyperspectral images. In the so-called whiskbroom method, the examination area or object is scanned point by point, and a spectrum is obtained for each point. In contrast, the staring method acquires multiple images with the same spatial coordinates. Different spectral filters and / or illumination sources are used from image to image to resolve spectral information. Furthermore, there are methods in which a two-dimensional multicolor image is decomposed into several individual spectral images using suitable optical elements such as optical slicers, lenses, and prisms. These individual images are simultaneously acquired on different detectors or detector areas. This is sometimes referred to as the snapshot approach.

[0007] As described in DE 10 2020 105 458 A1, multispectral and hyperspectral imaging devices are particularly suitable as endoscopic imaging devices. In this context, multispectral and / or hyperspectral imaging is a fundamental field of application, for example, for diagnostics and for assessing the success or quality of a procedure.

[0008] Multimodal imaging devices allow for selective acquisition of white light images, multispectral images, fluorescence images, and / or hyperspectral images. Examples of such imaging devices include multimodal endoscopes and multimodal exoscopes.

[0009] Regardless of the exact design, to capture fluorescence images, tissue is illuminated in a specific wavelength range in order to specifically excite fluorescent dye molecules introduced into certain entities, such as tissue regions. The resulting emitted light with a red-shifted wavelength can be observed through a suitably selected filter, which can be used to filter out excitation light. If the wavelength ranges of the excitation light and the emitted light are in the longer wavelength range relative to white light used for parallel or sequential white-light imaging, particularly in the near-infrared range, i.e., outside of visible white light, white-light imaging is possible despite these observation filters. Recently, however, dyes that can be excited by visible light have been increasingly used.The conventional method of image generation is not feasible here because the filters required for fluorescence imaging impair the white-light image by blocking visible light in certain wavelength ranges or usually only, or at least primarily, transmitting light below a certain wavelength. Therefore, if fluorescent dyes are used that are intended to be excited in the visible range, the aforementioned parallel or sequential white-light acquisition is not feasible. However, if the white-light image is omitted, it becomes difficult for the user to accurately assess anatomical structures.

[0010] Based on the state of the art, the invention is based on the object of enabling fluorescence imaging in a wide range of applications.

[0011] This object is achieved according to the invention by an imaging device, a method for medical imaging, program code and a computer program product as described herein and defined in the claims.

[0012] According to the invention, a medical imaging device, in particular an endoscope device, exoscope device, and / or microscope device, comprises an illumination unit with at least one light source configured to provide illumination light for illuminating an object to be imaged. Furthermore, the medical imaging device comprises an image acquisition unit comprising at least one image acquisition sensor and at least one observation filter. The observation filter is configured to block light in a first spectral range, which includes visible light, and to transmit light in a second spectral range different from the first spectral range.Furthermore, the image acquisition unit is configured to acquire fluorescence images of the object to be imaged through the observation filter. During the acquisition of these images, illumination light, the spectrum of which lies at least partially within the first spectral range, is used as excitation light. During the acquisition of these images, light emitted by the object to be imaged, the spectrum of which lies at least partially within the second spectral range, is detected by the image acquisition sensor. Furthermore, the image acquisition unit is configured to acquire anatomical images of the object to be imaged. During the acquisition of these images, illumination light, the spectrum of which lies at least partially within the second spectral range, is used. During the acquisition of these images, light remitted by the object to be imaged, the spectrum of which lies at least partially within the second spectral range, is detected by the image acquisition sensor.Furthermore, the medical imaging device comprises a representation generation unit which is configured to generate at least one representation from the fluorescence images and the anatomical images.

[0013] A method according to the invention can be carried out in particular with the medical imaging device according to the invention. The method comprises providing illumination light for illuminating an object to be imaged. Furthermore, the method comprises recording fluorescence images of the object to be imaged through an observation filter configured to block light in a first spectral range, which includes visible light, and to transmit light in a second spectral range different from the first spectral range. During the recording of the fluorescence images, illumination light, the spectrum of which lies at least partially within the first spectral range, is used as excitation light, and light emitted by the object to be imaged, the spectrum of which lies at least partially within the second spectral range, is detected.The method also comprises taking anatomical images of the object to be imaged, wherein, when taking the anatomical images, illumination light whose spectrum lies at least partially within the second spectral range is used and light remitted by the object to be imaged, whose spectrum lies at least partially within the second spectral range, is detected.

[0014] The features of the invention allow imaging to be performed in a wide range of applications. In particular, fluorescent dyes that absorb in the visible range can be used, and at the same time, a meaningful anatomical image can be provided to the user. This allows different fluorescent dyes to be used equally without having to accept limitations in the creation of an anatomical image. Thanks to the features of the invention, even with fluorescence excitation in the visible range, a fluorescence image and an anatomical image can be acquired without having to swivel filters in and out of the beam path between the individual images. Furthermore, anatomical images do not have to be completely dispensed with, even if fluorescence is to be excited in the visible range.In principle, it would also be conceivable to irradiate white light and fluorescence excitation light in parallel and separate the fluorescence and white light using the Bayer pattern of a detector. However, this would require considerable processing effort due to the associated and expected sensor crosstalk, and the expected image quality may be poor.

[0015] The imaging device can be a microscopic, macroscopic, and / or exoscopic imaging device. The imaging device can be configured as and / or comprise a microscope, macroscope, and / or exoscope. In some embodiments, the imaging device can be an endoscopic imaging device. The imaging device can be an endoscope device. It can comprise and / or be configured as an endoscope and / or an endoscope system and / or form at least a part and preferably at least a major part and / or a main component of an endoscope and / or an endoscope system. “At least a major part” can mean at least 55%, preferably at least 65%, more preferably at least 75%, more preferably at least 85%, and most preferably at least 95%, in particular with reference to a volume and / or a mass of an object.

[0016] In some embodiments, the imaging device is configured to be insertable into a cavity for inspection and / or observation, for example, into an artificial and / or natural cavity, such as the interior of a body, a body organ, tissue, or the like. The imaging device can also be configured to be insertable into a housing, casing, shaft, pipe, or other, particularly artificial, structure for inspection and / or observation.

[0017] In particular, if the imaging device is an exoscopic imaging device, it can be configured to record tissue parameters, images of wounds, images of body parts, etc. For example, the imaging device can be configured to image a surgical field. The image acquisition sensor system can be configured to detect light in both the visible range and the near-infrared range. In some embodiments, a smallest detectable wavelength can be at most 500 nm, at most 450 nm, or even at most 400 nm. In some embodiments, a largest detectable wavelength can be at least 800 nm, at least 900 nm, or even at least 1000 nm. The image acquisition sensor system can, for example, comprise at least one white-light image sensor and at least one near-infrared image sensor. In some embodiments, the imaging device comprises a white-light camera and / or sensors for white-light image acquisition.The imaging device can be configured for white-light imaging. The anatomical images can be captured using the white-light camera and / or the sensor system for white-light image acquisition.

[0018] The image acquisition unit can have a filter unit with optical observation filters. The filter unit can define multiple fluorescence modes defined by different observation filters. For example, different edge filters can be used that absorb / block the respective spectrum of the associated light element used for excitation and at least substantially transmit only fluorescent light. The observation filter, which blocks light in the first spectral range, is then part of the filter unit. In some embodiments, the observation filters can also be switchable between a multispectral mode and a fluorescence mode.

[0019] The imaging device, and in particular an optical system and / or the image acquisition sensor system, can be configured for multispectral and / or hyperspectral imaging, specifically for capturing and / or generating multispectral and / or hyperspectral image data. Multispectral imaging or multispectral image data can refer in particular to imaging in which at least two, in particular at least three, and in some cases at least five spectral bands can be and / or are captured independently of one another. Hyperspectral imaging or hyperspectral image data can refer in particular to imaging in which at least 20, at least 50, or even at least 100 spectral bands can be and / or are captured independently of one another.The imaging device may operate according to the pushbroom method and / or the whiskbroom method and / or the staring method and / or a snapshot principle.

[0020] For some applications, it can be advantageous to be able to use a high spectral resolution. In this case, hyperspectral imaging is a good option. This can be combined with white light imaging. This enables real-time observation via a white light image, even if the acquisition of spectrally resolved image data only takes place essentially in real time, i.e., several seconds are needed to create a spectrally resolved image. For some applications, it can be advantageous to generate spectral image data in real time. This includes, for example, generating a spectrally resolved image in less than a second or even several times per second. In this case, it can be useful to use multispectral imaging. A possibly lower spectral resolution is then offset by a higher frame rate.Depending on the application, it may be sufficient to consider only a few different spectral ranges and / or wavelengths, for example, two, three, four, or generally fewer than ten. In this case, additional white-light imaging can optionally be omitted. Spectrally resolved image data acquired in real time or delivering multiple images per second can also be used for surveillance purposes. It is not necessary to create a single image for a user to display; the image data can also be processed in the background.

[0021] The medical imaging device can have at least a proximal section, a distal section and / or an intermediate section. The distal section is designed in particular to be introduced into and / or located in a cavity to be examined in an operating state, for example during the diagnostic and / or therapeutic action. The proximal section is designed in particular to be arranged outside the cavity to be examined in an operating state, for example during the diagnostic and / or therapeutic action. “Distal” should be understood in particular to mean facing towards a patient and / or away from a user during use. “Proximal” should be understood in particular to mean facing away from a patient and / or away from a user during use. In particular, proximal is the opposite of distal.The medical imaging device has, in particular, at least one, preferably flexible, shaft. The shaft can be an elongate object. Furthermore, the shaft can at least partially and preferably at least largely form the distal section. An “elongate object” is to be understood, in particular, as an object whose main extent is at least a factor of five, preferably at least a factor of ten, and particularly preferably at least a factor of twenty greater than a greatest extent of the object perpendicular to its main extent, i.e., in particular, a diameter of the object. A “main extent” of an object is to be understood, in particular, as its longest extent along its main extension direction.A “main extension direction” of a component is to be understood in particular as a direction which runs parallel to a longest edge of a smallest imaginary cuboid which just completely encloses the component.

[0022] The image acquisition unit can be arranged at least partially and preferably at least largely in the region of the proximal section and / or form this. In other embodiments, the image acquisition unit can be arranged at least partially and preferably at least largely in the distal section and / or form this. Furthermore, the image acquisition unit can be arranged at least partially distributed between the proximal section and the distal section. The image acquisition sensor system has, in particular, at least one image sensor. Furthermore, the image acquisition sensor system can also have at least two and preferably more image sensors, which can be arranged one behind the other.Furthermore, the two and preferably multiple image capture sensors can have spectral detection sensitivities that differ from one another, so that, for example, a first sensor is particularly sensitive or comparatively more sensitive than the other sensors in a red spectral range, a second sensor in a blue spectral range, and a third sensor in a green spectral range. The image sensor can be designed, for example, as a CCD sensor and / or a CMOS sensor.

[0023] An optics system of the image capture unit may include suitable optical elements such as lenses, mirrors, gratings, prisms, optical fibers, etc. The optics system may be configured to guide object light coming from an imaged object to the image capture sensor system, for example, to focus and / or project it.

[0024] The image acquisition unit is particularly configured to generate at least two-dimensional spatial image data. The image acquisition unit can be spatially resolving in such a way that it delivers a resolution of at least 100 pixels, preferably of at least 200 pixels, preferably of at least 300 pixels, and advantageously of at least 400 pixels in at least two different spatial directions. The image data is preferably at least three-dimensional, with at least two dimensions being spatial dimensions and / or with at least one dimension being a spectral dimension. A plurality of spatially resolved images of the image area can be obtained from the image data, each of which is assigned to different spectral bands. The spatial and spectral information of the image data can be such that an associated spectrum can be obtained for a plurality of spatial pixels.

[0025] In some embodiments, the image acquisition unit is configured to generate continuously updated image data. For example, the image acquisition unit may be configured to generate the image data substantially in real time, which may include, for example, generating updated image data at least every 30 seconds, in some cases at least every 20 seconds, and in some cases even at least every 10 seconds or at least every 5 seconds. Preferably, the image acquisition unit is configured to generate at least the anatomical images and the fluorescence images, as well as the representation based thereon, in real time, for example, at a frame rate of at least 5 fps, at least 10 fps, at least 20 fps, or even at least 30 fps.

[0026] The lighting unit can be multimodal and comprise a plurality of independently selectively activatable lighting elements which are configured to emit light according to different emission spectra in order to provide the illumination light.

[0027] The term "visible light" can refer here in particular to light with a wavelength between 400 nm and 750 nm. Generally, this means light that is perceptible to the human eye. The first spectral range in particular comprises only a sub-range of visible light. The second spectral range can also comprise visible light. The observation filter can be an edge filter, in particular a high-pass filter. The edge of the observation filter or, more generally, the first spectral range can be selected to match the fluorescent dye used. If, for example, fluorescein or Cy5 is used as the dye, excitation occurs in the middle of the visible range, in particular at a distance from its edges, for example at approximately 430 nm for fluorescein or at approximately 660 nm for Cy5. In order not to distort fluorescence images due to excitation light, the excitation light must be blocked accordingly.The first spectral range must then be selected such that the excitation light is blocked at all times. In the case of fluorescein, for example, an edge filter with an edge at approximately 450 nm could be selected. In the case of Cy5, for example, an edge filter with an edge at approximately 680 nm could be selected. The first spectral range and the second spectral range are preferably at least substantially adjacent to one another. A spacing between the spectral ranges can, for example, be defined merely by the width of the filter edge. A third spectral range can be present which lies between the first spectral range and the second spectral range and in which the observation filter and in particular the transmittance of the observation filter changes significantly depending on the wavelength. As mentioned, this can be the filter edge.The first spectral range and / or the second spectral range may comprise a plurality of sub-ranges which may be spaced apart from one another.

[0028] In some embodiments, the second spectral range extends beyond the visible range into the near-infrared range. In particular, the light detected during the acquisition of the anatomical images can also be at least in the near-infrared range, for example, in a wavelength range beyond 800 nm, beyond 850 nm, or even beyond 900 nm.

[0029] The fluorescence images and the anatomical images can be acquired in parallel or sequentially. Preferably, both the fluorescence images and the anatomical images are moving images.

[0030] Images can be acquired particularly easily and at short intervals if the image acquisition unit is configured to acquire the anatomical images of the object to be imaged through the observation filter. The observation filter can therefore be permanently located in the beam path for both the anatomical images and the fluorescence images during the acquisition and display of a superimposed anatomical and fluorescence image. It can be changed, for example, if a different excitation wavelength and / or a different imaging mode is selected. The observation filter is located, in particular, in front of an optical input of an image acquisition sensor system of the imaging device. As mentioned, this can comprise sensors that respond in different spectral ranges; the observation filter is nevertheless expediently arranged in front of all of these sensors, i.e., in front of the image acquisition sensor system.This allows different filters to be used easily without having to integrate the filters directly into the sensors.

[0031] In other embodiments, the observation filter can be integrated into the image acquisition sensor system. Especially for multi-chip image sensors, different image sensors can be equipped with different observation filters. The acquisition of the anatomical images and the acquisition of the fluorescence images can then be performed using disjoint groups of image sensors. Images can be acquired in parallel. In this case, the observation spectral range used for fluorescence image acquisition is not available for anatomical image acquisition.

[0032] A shortest wavelength of the first spectral range can be greater than 430 nm and in particular greater than 500 nm. A longest wavelength of the first spectral range can be less than 800 nm and in particular less than 700 nm. In some embodiments, the illumination light used as excitation light for fluorescence imaging is narrowband. For example, the illumination light can originate from at least one colored LED, laser diode, or another at least substantially monochromatically emitting light element.

[0033] In some embodiments, the medical imaging device comprises a lighting device comprising the lighting unit. The lighting device can comprise an optical interface for optically connecting an imaging device. The lighting unit can be configured to supply illumination light to the optical interface. The lighting unit can be multimodal and comprise a plurality of independently selectably activatable lighting elements configured to emit light according to different emission spectra in order to supply the illumination light. The lighting unit can be operable in at least one multispectral mode in which a first group of the lighting elements is at least temporarily activated and in which the lighting unit supplies illumination light for multispectral imaging.Furthermore, the illumination unit can be operable in at least one fluorescence mode, in which a second group of the luminous elements is at least temporarily activated and in which the illumination unit provides illumination light for fluorescence imaging. The luminous elements can comprise at least one luminous element that is contained in both the first group and the second group.

[0034] In addition, a method for generating illumination light for an imaging device using an illumination device can be provided. The illumination device comprises an optical interface for optically connecting an imaging device and a illumination unit configured to supply illumination light to the optical interface, wherein the illumination unit comprises a plurality of independently selectably activatable luminous elements configured to emit light according to different emission spectra in order to supply the illumination light. The method comprises the step of at least temporarily activating a first group of the luminous elements in order to supply illumination light for multispectral imaging and the step of at least temporarily activating a second group of the luminous elements in order to supply illumination light for fluorescence imaging.At least one of the lighting elements is activated at least temporarily both when the first group of lighting elements is activated at least temporarily and when the second group of lighting elements is activated at least temporarily.

[0035] The optical interface can be either connectable or detachable. Furthermore, the optical interface can be combined with a mechanical interface, so that an optical connection is automatically established, for example, when the imaging device is mechanically coupled.

[0036] The lighting elements can comprise single-color LEDs (light-emitting diodes) and / or laser diodes. Furthermore, at least one of the lighting elements can be a white light LED or another white light source. In some embodiments, the lighting unit comprises at least one blue lighting element, at least one red lighting element, at least one far-red lighting element, and at least one near-IR (near-infrared) lighting element, in particular LEDs or laser diodes. Additionally, the lighting unit can comprise at least one white light LED or another white light source.

[0037] The first group can comprise at least two light elements that emit spectrally differently. A high degree of efficiency in multispectral imaging can be achieved if the multispectral mode comprises different states, in each of which a specific light element or a specific type of light element is activated at least temporarily. This allows targeted illumination in a specific spectral range, whereby different spectral images can be captured. Different light elements that are activated in different states can serve as different support points for the multispectral imaging. At least one of these support points can be selected such that it is adapted to characteristic points of absorption spectra of physiologically relevant components, for example, to an isosbestic point of the hemoglobin oxygenation curve.Multispectral imaging may additionally include the use of appropriate observation filters.

[0038] Furthermore, the second group can comprise at least two luminous elements that emit spectrally differently. The fluorescence mode can comprise different submodes and / or states, in each of which a specific luminous element or a specific luminous element type is activated at least temporarily. This allows for targeted excitation in a specific spectral range, so that fluorescence imaging can be performed, for example, for a specifically selected dye. In other words, the at least one luminous element that is contained in both the first group and the second group can be used for both the multispectral mode and the fluorescence mode.

[0039] In some embodiments, the first group comprises only some but not all of the luminous elements. Alternatively or additionally, in some embodiments, the second group comprises only some but not all of the luminous elements. In the multispectral mode, in particular, only luminous elements of the first group are activated at least temporarily, whereas luminous elements that do not belong to the first group are deactivated. In the fluorescence mode, in particular, only luminous elements of the second group are activated at least temporarily, whereas luminous elements that do not belong to the second group are deactivated. In general, it is understood that the luminous elements can comprise different luminous element types and that, in particular, exactly one luminous element of each of the different luminous element types can be present.It is understood that mixed operating modes can also occur according to the invention, in which the said modes are used sequentially. For example, multispectral imaging and fluorescence imaging can be carried out sequentially. Synergy with regard to the use of a luminous element for different modes and associated efficiency gains can be achieved in particular if at least one luminous element which is contained in both the first group and the second group emits light in the red spectral range, in particular in a spectral range between 600 nm and 680 nm, for example between 610 nm and 650 nm or between 620 and 660 nm or between 630 and 670 nm. The spectral range can be narrowband and include the wavelength 660 nm. “Narrowband” can include a spectral width of at most 80 nm, in particular of at most 40 nm or even of at most 20 nm.This at least one luminous element can be configured to excite dyes absorbing in the red spectral range and to contribute to the illumination in the red spectral range for multispectral imaging.

[0040] In some embodiments, the illumination unit can be operable in at least one white-light mode, in which the illumination unit provides illumination light for white-light imaging. The illumination light for white-light imaging can be broadband white light. Alternatively, the illumination light for white-light imaging can comprise several narrow wavelength bands that are separated from one another, for example, a blue, a red, and a far-red band. "Dark red" is to be understood in the sense of "longer wavelength than red" and refers to the spectral position, not the light intensity. The illumination light for white-light imaging can be mixed from light from different lighting elements.

[0041] In the white light mode, a third group of light elements can be activated at least temporarily to supply the illumination light for white light imaging. The light elements can comprise at least one light element that is contained in both the first group and / or the second group and the third group. In some cases, the third group can comprise only some but not all of the light elements. In the white light mode, in particular, only light elements of the third group are activated at least temporarily, whereas light elements that do not belong to the third group are deactivated. In other words, the illumination unit can comprise light elements that serve one, two, or all three of the aforementioned illumination modes. This allows multiple light elements to be used multiple times.

[0042] At least one luminous element contained in both the first group and / or the second group and the third group can emit light in the red spectral range, in particular in a spectral range between 600 nm and 680 nm, for example between 610 nm and 650 nm or between 620 and 660 nm or between 630 and 670 nm. The advantages of using luminous elements together are particularly evident when at least one red luminous element can be used for all three modes.

[0043] At least one luminous element contained in both the first group and / or the second group and the third group can emit light in the blue spectral range, in particular in a spectral range between 440 and 480 nm. At least one blue luminous element can expediently be used in both the fluorescence mode and the white light mode.

[0044] Generally speaking, the luminous elements can, as mentioned, comprise at least one, in particular blue, luminous element that emits light in a spectral range between 440 and 480 nm. Furthermore, the luminous elements can, as mentioned, comprise at least one, in particular red, luminous element that emits light in a spectral range between 600 and 680 nm, for example between 610 nm and 650 nm or between 620 and 660 nm or between 630 and 670 nm. Alternatively or additionally, the luminous elements can comprise at least one, in particular dark-red, luminous element that emits light in a spectral range between 750 and 790 nm. Alternatively or additionally, the luminous elements can comprise at least one, in particular near-IR-emitting, luminous element that emits light in a spectral range between 920 and 960 nm. In addition, the luminous elements can comprise a white-light luminous element.A compact and versatile lighting unit can be provided in particular if at least one light element of each of the aforementioned light element types is present. For example, in fluorescence mode, the blue and red light elements can be used, and if suitable dyes are used, the far-red light element can also be used. In multispectral mode, the far-red and the near-IR emitting light element can be used. In white light mode, the white light light element can be used. In white light mode, this can be supplemented by the blue light element and, if necessary, the red light element. This allows wavelength ranges to be supplemented by colored light elements in which the white light light element provides a reduced intensity, for example due to its design, but in particular due to filters and optical elements of the lighting unit.In addition, the colored light-emitting elements can be used to adjust a color temperature during white-light imaging. In some embodiments, the second group comprises a single light-emitting element and / or a single type of light-emitting element. For example, a white-light light-emitting element, a red light-emitting element, and an IR-emitting light-emitting element can be provided, with particular reference being made to the above values ​​with regard to possible spectral ranges. The first group can then, for example, comprise the red and the IR-emitting light-emitting element. The second group can comprise the IR-emitting light-emitting element, in particular as the only light-emitting element or as the only type of light-emitting element.

[0045] A favorable arrangement of lighting elements is particularly possible if the lighting unit comprises at least one crossed beam splitter, by means of which light can be deflected from opposite input sides to an output side, wherein at least one of the lighting elements is arranged on each of the opposite input sides of the crossed beam splitter. In some embodiments, two or more crossed beam splitters can be provided, which are arranged optically one behind the other. The at least one crossed beam splitter can comprise two beam splitter elements, the transmittance of which is adapted to the respectively assigned lighting element. The beam splitter elements each comprise, in particular, a notch filter, so that they each reflect in a narrow spectral band but otherwise transmit.The spectral position and / or width of the corresponding notch can be adapted to the spectral range of the respective associated luminous element, so that its light is redirected, but light from other luminous elements is at least largely transmitted.

[0046] In some embodiments, the luminous elements can comprise at least four narrow-band emitting single-color luminous elements, each with different spectral ranges, and at least one broadband emitting white light luminous element. In this regard, reference is also made to the above statements regarding the colored luminous elements.

[0047] A wide range of functions combined with a compact design and the exploitation of synergy effects when using lighting elements can be achieved, in particular, if the illumination unit is operable in at least one hyperspectral mode in which several lighting elements are activated, whose emission spectra together cover at least a spectral range from 450 nm to 850 nm, and in which the illumination unit provides illumination light for hyperspectral imaging. This can, in particular, involve all of the lighting elements.

[0048] It is understood that, particularly when using laser diodes, suitable polarization filters can be used for the optical filters mentioned herein. Furthermore, particularly when using laser diodes, at least one crossed beam splitter can be used, the beam splitter elements of which are provided with polarization filters. Selective transmission can then be achieved by combining different polarizations.

[0049] The representation generated by the representation generation unit from the fluorescence images and the anatomical images can comprise a combined representation, in particular an overlay representation based on a combination of fluorescence images and anatomical images. A user can then particularly easily associate fluorescence information with anatomical information. In particular, the fluorescence images and the anatomical images can be focused on the same object, in particular the same anatomy. Image sections can differ from one another.

[0050] The imaging device may comprise a display and / or be designed for connection to a display. A corresponding interface may be provided for this purpose. The display generation unit may be configured to generate the displays for the display. The display may, for example, comprise a screen on which the display is made available to a user.

[0051] In some embodiments, the image acquisition unit is configured to capture the fluorescence images and the anatomical images in real time. Furthermore, the representation generation unit can be configured to generate the at least one representation in real time. In this way, an anatomical and a fluorescence-based moving image can be made available to a user, even if excitation occurs in the visible range.

[0052] As mentioned, in some embodiments, the image acquisition unit can be configured to record the fluorescence images and the anatomical images sequentially. It can then be advantageous if the representation generation unit is configured to assign sequentially recorded fluorescence images and anatomical images to one another, in particular chronologically, and to generate the at least one representation based on assigned fluorescence images and anatomical images. The assignment can be frame-synchronous. “Frame-synchronous” is to be understood here as meaning that assigned fluorescence images and anatomical images are temporally correlated. This does not necessarily mean simultaneity in the narrow sense. Furthermore, the term “frame-synchronous” also encompasses situations in which a first number of fluorescence images and a different second number of anatomical images are recorded alternately.For example, the invention can provide for an update rate for anatomical images to be higher than an update rate for fluorescence images. The recording is then nevertheless frame-synchronous within the meaning of this disclosure if the assignment is performed in such a way that a temporal relationship exists between the displayed images. In other words, the combined display can be based on at least one fluorescence image and at least one anatomical image that were recorded within a predetermined period of time, for example, within a period of at most 1000 ms, at most 500 ms, at most 100 ms, or even at most 50 ms.

[0053] In addition to the above statements regarding the image acquisition sensor system, it can generally be provided that the image acquisition sensor system comprises at least one first image acquisition sensor configured to acquire image information for the fluorescence images, and that the image acquisition sensor system further comprises at least one second image acquisition sensor configured to acquire image information for the anatomical images. The image acquisition sensors can be combined by means of at least one optical element such that detected light always or at least selectively falls on several or all image acquisition sensors simultaneously. The first image acquisition sensor can comprise at least one near-infrared image sensor. The second image acquisition sensor can comprise a white-light image acquisition chip. Alternatively or additionally, the second image acquisition sensor can comprise different color channels, for example a red, a green, and a blue color channel.This can be achieved, for example, by using a color filter with a suitable pattern. In some embodiments, for example, a Bayer sensor is used, but other arrangements are also possible according to the invention. Furthermore, it can be provided that a separate two-dimensional image sensor is used for each color channel. It is understood that in some embodiments, a fluorescence image can additionally or alternatively also be generated using the first image acquisition sensor and / or that an anatomical image can additionally or alternatively also be generated using the second image acquisition sensor. The anatomical images can, for example, comprise multiple anatomical images and / or be based on multiple individual images assigned to different color channels, such as images from a red color channel, a green color channel, a blue color channel and / or a near-infrared color channel.Alternatively or additionally, a single anatomy image may comprise multiple color channels, for example a red color channel, a green color channel, a blue color channel and / or a near-infrared color channel.

[0054] In other embodiments, the image acquisition sensor system comprises a single image acquisition sensor, in particular a single-chip image acquisition sensor and / or a single-chip camera head that images in both the visible range and the near-infrared range.

[0055] In some embodiments, the image acquisition sensor system can be configured to simultaneously capture fluorescence images and anatomical images. "Simultaneously" refers in particular to a temporal simultaneity, which differs, for example, from sequential capture of fluorescence images and anatomical images. In this case, for example, the capture of an anatomical image and the simultaneous capture of a fluorescence image can be triggered within a period of no more than 500 ms, no more than 100 ms, or even no more than 20 ms. However, "simultaneously" can encompass the capture of the corresponding images taking different lengths of time, for example, if different exposure times are used.

[0056] Available image information can be used in a targeted manner and to generate a meaningful anatomical image, in particular if the representation generation unit is configured to generate at least one anatomical image from a plurality of, in particular, single-color, individual images and / or from a plurality of color channels of anatomical images. The anatomical image can thus be synthesized from a plurality of color channels, whereby the omission of a specific color channel and / or a specific spectral range due to the observation filter can be at least partially compensated. Depending on the image acquisition sensor technology used, individual image sensors can also provide individual images, for example, single-color individual images, such as a blue, green, red, and / or near-infrared individual image, or even different and possibly partially multi-colored individual images, such as an RGB individual image and a near-infrared individual image.In some embodiments, the anatomical image corresponds to a white-light image with multiple color channels, in which at least one color channel from the visible range is replaced by at least one color channel that lies outside the visible range, particularly in the near-infrared range. This allows a nearly natural anatomical image to be generated, even if spectral information is lost due to the observation filter.

[0057] The invention also relates to program code which, when executed in a processor, is designed to effect implementation of a method according to the invention.

[0058] Furthermore, the invention relates to a program code comprising a computer-readable medium on which the program code according to the invention is stored.

[0059] The devices and systems according to the invention, as well as the methods according to the invention, are not intended to be limited to the application and embodiment described above. In particular, to fulfill a functionality described herein, they may comprise a number of individual elements, components, units, and method steps that differs from the number stated herein. Furthermore, in the value ranges specified in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily.

[0060] It is particularly noted that all features and properties described with reference to a device, as well as procedures, are transferable to methods and applicable within the meaning of the invention and are considered to be included in the disclosure. The same applies in reverse. This means that structural features mentioned with reference to methods, i.e., features related to the device, can also be considered, claimed, and included in the disclosure within the scope of the device claims.

[0061] The present invention is described below by way of example with reference to the attached figures. The drawings, the description and the claims contain numerous features in combination. A person skilled in the art will expediently also consider the features individually and use them sensibly in combination within the scope of the claims. If there is more than one example of a particular object, only one of them will be provided with a reference symbol in the figures and the description. The description of this example can be applied accordingly to the other examples of the object. If objects are named in particular using numerical words such as first, second, third object etc., these serve to name and / or assign objects. Accordingly, for example, a first object and a third object may be included, but not a second object.However, using number words, it might also be possible to derive a number and / or order of objects.

[0062] They show:

[0063] Fig. 1 is a schematic representation of an imaging device with an illumination device;

[0064] Fig. 2 is a schematic representation of the lighting device;

[0065] Fig. 3 schematic transmission curves of beam splitter elements of the lighting device;

[0066] Fig. 4 is a schematic representation of the imaging device;

[0067] Fig. 5 is a schematic representation of another embodiment of the

[0068] imaging device;

[0069] Fig. 6 is a schematic representation of yet another embodiment of the imaging device;

[0070] Fig. 7 is a schematic perspective view of another embodiment of the imaging device;

[0071] Fig. 8 is a schematic flow diagram of a method for generating illumination light for an imaging device by means of an illumination device;

[0072] Fig. 9 is a schematic flow diagram of a method for operating an imaging device; Fig. 10 is a schematic flow diagram of a method for operating an imaging device;

[0073] Fig. 11 is a schematic representation of a medical imaging device;

[0074] Fig. 12 schematic spectra of a fluorescent dye and a transmission curve of an associated observation filter;

[0075] Fig. 13 schematic spectra of the fluorescent dye and a transmission curve of an alternative associated observation filter;

[0076] Fig. 14 schematic representations of a series of fluorescence images and anatomical images;

[0077] Fig. 15 is a schematic representation of color channels of an anatomical image;

[0078] Fig. 16 is a schematic diagram illustrating a first example of calculating fluorescence images and anatomical images;

[0079] Fig. 17 is a schematic diagram illustrating a second example of calculating fluorescence images and anatomical images;

[0080] Fig. 18 is a schematic diagram illustrating a third example of calculating fluorescence images and anatomical images;

[0081] Fig. 19 is a schematic diagram illustrating a fourth example of calculating fluorescence images and anatomical images;

[0082] Fig. 20 is a schematic flow diagram of a method for medical imaging; and

[0083] Fig. 21 is a schematic representation of a computer program product.

[0084] Fig. 1 shows a schematic representation of an imaging device 10. In the exemplary case shown, the imaging device 10 is an endoscopic imaging device, specifically an endoscope device. Alternatively, the imaging device 10 could be an exoscopic, a microscopic, or a macroscopic imaging device. The imaging device 10 is shown as an example of a medical imaging device. The imaging device 10 is intended, for example, for examining a cavity.

[0085] The imaging device 10 comprises a medical imaging device 14. In the illustrated case, this is an endoscope.

[0086] The imaging device 10 further comprises an illumination device 12 with an optical interface 16 and an illumination unit 18. The imaging device 14 can be optically connected to the optical interface 16. The optical interface 16 can be part of an optical-mechanical interface that can be selectively connected and detachable. The illumination device 14 can be selectively decoupled from the illumination device 12. The illumination unit 18 is configured to supply illumination light to the optical interface 16. During imaging using the imaging device 14, the illumination unit 18 can accordingly provide the required illumination light, which is guided to the illumination device 14 and from there coupled out onto an object to be imaged, such as a site.

[0087] In the illustrated case, the imaging device 10 further comprises a display unit on which images based on image data acquired by the imaging device 14 can be displayed. These may be video images, still images, overlays of different images, partial images, image sequences, etc.

[0088] The imaging device 10 is multimodal. By way of example, the imaging device can be operated in three basic modes: a multispectral mode, a fluorescence mode, and a white light mode. Furthermore, it can be provided that the imaging device 10 can be operated in a hyperspectral mode in addition to or alternatively to the multispectral mode.

[0089] The illumination device 12 is multimodal. The illumination device 12 can be operated in different illumination modes, in which it provides light for different imaging modes. In the present case, the illumination device 12 can be operated in three basic modes: a multispectral mode, a fluorescence mode, and a white light mode. Likewise, the imaging device 14 can be operated in different operating modes, specifically also in at least one multispectral mode, one fluorescence mode, and one white light mode. In the corresponding operating mode of the imaging device 10, the modes of the illumination device 12 are coordinated with one another.

[0090] Fig. 2 shows a schematic representation of the lighting device 12. The lighting unit 18 comprises a plurality of independently activatable lighting elements 20, 22, 24, 26, 28. These are designed to emit light according to different emission spectra in order to provide illumination light, ie the respective emission spectrum differs from lighting element to lighting element.

[0091] For example, the light elements 20, 22, 24, 26, 28 are embodied as LEDs. Specifically, a first light element 20 is embodied as a red LED, a second light element 22 as a dark-red LED, a third light element 24 as a blue LED, and a fourth light element 26 as a near-IR LED. The colored light elements 20, 22, 24, 26 each emit in a narrowband, for example, with emission peaks at wavelengths of approximately 660 nm (first light element 20), 770 nm (second light element 22), 460 nm (third light element 24), and 940 nm (fourth light element 26).

[0092] Furthermore, a fifth luminous element 28 is provided, which in this case is a white light luminous element, such as a white light LED. The fifth luminous element 28 emits, for example, in a spectral range of approximately 400 to 700 nm. In other embodiments, laser diodes can also be used, in particular as colored luminous elements.

[0093] Depending on the lighting mode, some of the lighting elements 20, 22, 24, 26, 28 are activated at least temporarily, whereas other lighting elements 20, 22, 24, 26, 28 may not be used in the lighting mode in question.

[0094] In the present case, a first group comprises the first light element 20 and the fourth light element 26. The first group can additionally comprise the light element 22 and / or the light element 24. The first group is used for multispectral imaging, wherein the included light elements 20, 26 and optionally 22 and 24 each serve as a support point. In multispectral mode, for example, the first light element 20 is first illuminated and an image is recorded. The fourth light element 26 is then illuminated and an image is recorded. The images are each based on remission, i.e. the light scattered back from the object to be imaged is observed. Spectral information about the object to be imaged can be obtained from the two different support points. For example, this can be used to assess certain types of tissue, a perfusion state, a tissue texture or the like.

[0095] Furthermore, a second group comprises the first light-emitting element 20, the second light-emitting element 22, and the third light-emitting element 24. The second group is used for illumination in fluorescence imaging. For example, objects colored with suitably selected dyes can be specifically viewed here. Different dyes can also be introduced into different types of tissue or the like, which are then viewed simultaneously. By specifically exciting a specific dye, it is excited to fluoresce. The fluorescent light is then imaged. The first light-emitting element 20 is suitable, for example, for exciting the dye cyanine 5.5 (Cy 5.5). The second light-emitting element 22 is suitable for exciting the dye indocyanine green (ICG). The third light-emitting element 24 is suitable for exciting the dye fluorescein.

[0096] Furthermore, a third group comprises the fifth luminous element 28. In the present embodiment, the third group also comprises the first luminous element 20 and the third luminous element 24. The third group serves to provide illumination light for white light imaging. For this purpose, white light from the fifth luminous element 28 can be mixed with light from certain colored luminous elements, thereby compensating for spectral losses and / or allowing a color temperature to be specifically adjusted.

[0097] It can be seen that some of the lighting elements 20, 22, 24, 26, 28 are assigned to several groups, for example the first lighting element 20 to all three groups and the third lighting element 24 and possibly also the second lighting element 22 to the second and third groups.

[0098] Alternatively or additionally, it can also be provided that some or all of the light elements 20, 22, 24, 26, 28 are used in a hyperspectral mode. This generates a broad excitation spectrum. In combination with a suitable hyperspectral detector, spectral information relating to the object to be imaged can then be acquired across the entire visible and near-IR spectrum. For this purpose, the imaging device 14 can comprise a pushbroom arrangement as a hyperspectral detector. In other embodiments, a whiskbroom arrangement, a staring arrangement and / or a snapshot arrangement is used. The imaging device 14 can be a hyperspectral imaging device. With regard to different methods of hyperspectral imaging and the components required for this purpose, reference is made to the specialist article “Review of spectral imaging technology in biomedical engineering: achievements and challenges” by Quingli Li et al.Published in Journal of Biomedical Optics 18(10), 100901 , October 2013, and reference is made to the article “Medical hyperspectral imaging: a review” by Guolan Lu and Baowei Fei, published in Journal of Biomedical Optics 19(1), 010901 , January 2014.

[0099] The illumination unit 18 comprises two crossed beam splitters 30, 32. These each comprise an output side 42, 44, an input side 37, 41 opposite the output side 42, 44, and two opposite input sides 34, 36, 38, 40. All input sides 34, 36, 37, 38, 40, 41 guide incident light to the corresponding output side 42, 44. The output side 42 of a first crossed beam splitter 30 faces an input side 41 of the second crossed beam splitter 32. The output side 44 of the second crossed beam splitter 32 faces the optical interface 16. The two crossed beam splitters 30, 32 are preferably arranged coaxially to one another and / or to the optical interface.

[0100] The lighting unit 18 can comprise suitable optical elements such as lenses and / or mirrors (not shown). Several lenses 78, 80, 82, 84, 86, 88 are shown as examples in Fig. 2. A lens 78 is assigned, for example, to the optical interface 16 and couples light coming from the output side 44 of the second crossed beam splitter 32 into the optical interface 16. Furthermore, a lens 80, 82, 84, 86, 88 can be assigned to each of the lighting elements 20, 22, 24, 26, 28. A particularly high degree of compactness can be achieved in particular if the lighting elements 20, 22, 24, 26, 28 are each arranged on the input sides 34, 36, 37, 38, 40 of the at least one crossed beam splitter 30, 32 without an intermediate mirror. The lighting elements 20, 22, 24, 26, 28 can then be moved very close to at least one crossed beam splitter 30, 32.

[0101] The crossed beam splitters 30, 32 each comprise two beam splitter elements 90, 92, 94, 96. These can generally be partially transparent, so that light from all input sides 34, 36, 37, 38, 40, 41 is redirected to the respective output side 42, 44. In the present embodiment, the beam splitter elements 90, 92, 94, 96 are selectively transparent. This is illustrated with further reference to Fig. 3. The beam splitter elements 90, 92, 94, 96 can be filters that reflect only in a defined area but otherwise have high transmission. Fig. 3 shows transmission curves 98, 100, 102, 104 of the beam splitter elements 90, 92, 94, 96 of the two crossed beam splitters 30, 32. Each of the colored light elements 20, 22, 24, 26 or each of the opposite input sides 34, 36, 38, 40 is assigned one of the beam splitter elements 90, 92, 94, 96.The beam splitter elements 90, 92, 94, 96 are selected such that they each reflect in the wavelength range in which the associated light element 20, 22, 24, 26 emits, but also largely transmit. For this purpose, notch filters can be used in the mid-wavelength range, which can, for example, have the transmission spectra 100 and 102. At spectral edges, high-pass or low-pass filters can also be used instead of notch filters, see transmission spectra 98 and 104.

[0102] Due to the specific transmission spectra 98, 100, 102, 104 of the crossed beam splitters 30, 32, light from the fifth luminous element 28 is spectrally clipped. It may therefore be expedient, as already mentioned, to supplement the light blocked by the beam splitters 30, 32 in a targeted manner using the luminous elements 20 and 24, optionally also 22 and / or 26. This allows supplementation specifically in those spectral ranges in which the beam splitters 30, 32 absorb and / or reflect light from the fifth luminous element 28, but in any case does not transmit it to the optical interface 16. The additionally used luminous elements 20, 24 and optionally 22 are preferably operated at reduced power or with adjusted power. The aim here can be to at least largely restore the original spectrum of the fifth luminous element 28.

[0103] In some embodiments, the fifth light-emitting element 28 can alternatively be a green light-emitting element, or more generally, a colored light-emitting element that emits primarily in the spectral range transmitted by the at least one beam splitter 30, 32. For example, in such embodiments, the fifth light-emitting element 26 can be an LED with an emission peak at approximately 530 nm. A green laser diode is also suitable for this purpose. In this case, it can be provided that color mixing takes place in white light mode and, in particular, no individual white light source such as a white light LED is used, but rather white light from separate light elements is specifically mixed. It is understood that, in the case of suitable dyes, such a green light-emitting element can also be used in fluorescence mode. Alternatively or additionally, it could be used in multispectral mode.

[0104] The illumination unit 18 defines a common optical path 54 into which the emitted light from the lighting elements 20, 22, 24, 26, 28 can be coupled. The common optical path 54 extends from the output side 44 of the second crossed beam splitter 32 to the optical interface. In this case, the common optical path 54 is arranged coaxially with the fifth lighting element 26.

[0105] In the embodiment shown, the lighting elements 20, 26 of the first group are arranged such that light emitted by the lighting elements 20, 26, starting from the respective lighting element 20, 26, each travels a light path of at least substantially equal length to the optical interface 16. The lighting elements 20, 26 of the first group each have a light-emitting surface 56, 58. The light-emitting surfaces 56, 62 are arranged equidistantly with respect to the common optical path 54. This is achieved in the present case by arranging the two lighting elements 20, 26 at the same distance from their associated beam splitter 32 (here, by way of example, the second beam splitter 32), specifically from its opposite input sides 38, 40. The light is coupled into the common optical path 54 by the crossed beam splitter 32.

[0106] The beam splitters 30, 32 are arranged in particular such that light-emitting surfaces 56, 58, 60, 62, 64 of the lighting elements 20, 22, 24, 26, 28 are each arranged equidistantly with respect to their associated crossed beam splitter 30, 32.

[0107] By using crossed beam splitters 30, 32 and lighting elements 20, 22, 24, 26, 28 that can be used jointly for different modes, the illumination unit 18 or the illumination device 12 has a high degree of compactness. Furthermore, the equidistant arrangement ensures that no spectral shifts occur when the imaging device 14 or its light guide is rotated relative to the optical interface 16.

[0108] It is understood that a different number of light-emitting elements 20, 22, 24, 26, 28 and / or a different number of crossed beam splitters 30, 32 may be used. The use of crossed beam splitters 30, 32 has proven particularly useful. However, in other embodiments, other types of beam splitters and / or other optical elements may be used to couple light from the light-emitting elements 20, 22, 24, 26, 28 into the optical interface 16.

[0109] Fig. 4 shows a schematic representation of the imaging device 10. The imaging device 14 is optically coupled to the optical interface 16, for example via a light guide 106 such as at least one optical fiber.

[0110] The imaging device 10 has a controller 66 configured to automatically coordinate an operating state of the imaging device 14 and a lighting mode of the lighting unit 18. In this case, a user can specify the operating mode of the imaging device 14 through a user action. The controller 66 then sets the appropriate lighting mode of the lighting unit 18. Alternatively or additionally, the user can set a specific lighting mode of the lighting unit 18 through a user action. The controller 66 can then set an appropriate operating mode of the imaging device 14. The lighting device 12 and / or the imaging device 10 has, for example, a user interface via which the user can enter corresponding commands.

[0111] The imaging device 14 comprises a camera unit 68 and a distal shaft 76. The distal shaft 76 is optically coupled to the camera unit 68. The camera unit 68 can have a connection for the distal shaft 76, wherein the distal shaft 76 can be selectively coupled and decoupled. The distal shaft 76 can also be permanently optically and / or mechanically coupled to the camera unit 68. The camera unit 68 is arranged proximally with respect to the shaft 76. The camera unit 68 comprises imaging sensors 108, in the present case, for example, a white light sensor 110 and a near-IR sensor 112. Generally speaking, the imaging sensors 108 can have one or more at least spatially resolving light sensors / image sensors, for example, at least one CMOS sensor and / or at least one CCD sensor.The shaft 76 comprises optical elements (not shown) by means of which light can be guided to the camera unit 68 in order to optically capture the object to be imaged. Furthermore, the shaft 76 comprises at least one light path 114, for example defined by a light guide such as an optical fiber, which leads to a distal portion 116 of the shaft 76 and by means of which the illumination light originating from the optical interface 16 of the illumination device 12 can be coupled out to the object to be imaged.

[0112] The camera unit 68 has different operating states, specifically, for example, at least one multispectral operating state and one fluorescence operating state, and in the present embodiment, additionally a white light operating state and possibly a hyperspectral operating state. The controller 66 automatically adapts the illumination mode of the illumination unit 18 to the current operating state of the camera unit 68. In this case, the controller 66 can make settings to the image recording behavior of the camera unit 68. For example, the controller 66 can set the exposure time, sensitivity / amplification / gain and / or other operating parameters of the camera unit 68, or specifically its image acquisition sensor system 108 and, if applicable, its optics, and thereby define different operating states of the imaging device 14. In the present case, the controller 66 triggers the illumination unit 18 synchronously with the camera.

[0113] The imaging device 14 comprises a filter unit 46 with optical filters 48, 50, 52. Three optical filters are shown as an example, but it is understood that a different number can be used. The filter unit 46 can be switched between a multispectral mode and a fluorescence mode. Furthermore, the filter unit 46 can additionally be switched to a white light mode and / or a hyperspectral mode. The optical filters 48, 50, 52 can be selectively inserted into an observation beam path 70 of the camera unit 68, thereby defining different observation modes. In this case, these define the operating states of the camera unit 68.

[0114] A basic imaging mode can be assigned multiple optical filters 48, 50, 52. For fluorescence imaging in particular, a different suitable optical filter can be used depending on the light element 20, 22, 24, 26, 28 used for excitation. For example, in the present case, the first light element 20 (red) is combined with an optical filter that transmits wavelengths greater than 730 nm but blocks shorter wavelengths. This ensures, in particular, that only fluorescent light and not the excitation light itself is detected. For example, this optical filter can absorb at least in the range from 600 nm to 730 nm. Furthermore, in the present case, for example, the second light element 22 (dark red) is combined with a filter that absorbs in the range from 700 to 850 nm or that only transmits significantly above 850 nm.

[0115] The user can select a specific filter 48, 50, 52 and thereby directly select a corresponding observation mode or operating state of the camera unit 68. For this purpose, the camera unit 68 has a filter sensor 72 that can automatically detect an optical filter currently inserted into the observation beam path 70. The user can thus manually insert a selected filter 48, 50, 52 into the observation beam path 70. In the example shown, the optical filters 48, 50, 52 are mounted on a filter carrier 118. This can be moved into different positions, allowing one of the optical filters 48, 50, 52 to be selected at a time. The filter sensor 72 then detects the currently selected optical filter 48, 50, 52.The controller can then determine the current operating state of the camera unit 68 and thus of the imaging device 14 based on a sensor signal from the filter sensor 72 and automatically adjust the illumination mode of the illumination unit 18 accordingly. The user thus sets the entire imaging device 10 to the desired mode through a simple user action such as manually selecting an optical filter 48, 50, 52. In principle, a user can combine different filters with different illumination modes and thereby create different types of contrast.

[0116] In the illustrated case, the imaging device 14, and in particular the shaft 76, comprises a broadband transmitting optic 77 that can be used consistently in the different illumination modes. In this case, the broadband optic 77 is designed for a spectral range of at least 400 nm to 1000 nm. It can be used consistently for different illumination and / or observation spectral ranges.

[0117] In some embodiments, the imaging device 14 can be configured as a stereoendoscope comprising a stereoscopic eyepiece with two sides. Different optical filters can be connected independently of one another to these sides, allowing different contrast images to be superimposed on one another.

[0118] In the following, in the context of further embodiments and modifications, the same reference numerals as above are used for identical or similar components. Regarding their description, reference is generally made to the above explanations, whereas the following primarily explains differences between the embodiments. Furthermore, some reference numerals have been omitted in the following figures for the sake of clarity.

[0119] Fig. 5 shows a schematic representation of another embodiment of the imaging device 10. The imaging device 10 comprises an illumination device 12 with an optical interface 16 and an illumination unit 18, as well as an imaging device 14 connected to the optical interface 16. The imaging device 14 comprises a camera unit 68 with an automated filter unit 210. The automated filter unit 210 comprises a plurality of optical filters 48, 50, 52, which can be automatically inserted into an observation beam path 70 of the camera unit 68 according to an observation mode specified by a user.

[0120] The automated filter unit 210 includes a filter drive 212 configured to automatically move the optical filters 48, 50, 52 into or out of the observation beam path 70. The optical filters 48, 50, 52 can be mounted on a filter carrier 118 connected to the filter drive 212. The filter drive 212 can be configured to move the filter carrier 118, for example, to shift and / or rotate and / or pivot it.

[0121] The imaging device 14 has a user interface 214, by means of which the user can set a desired observation mode. For example, a desired position of the filter carrier 118 can be specified using the user interface 214.

[0122] The imaging device 14 further includes a controller 66. The controller 66 is coupled to the filter drive 212 and the user interface 214. The controller 66 is configured, in particular, to process a user specification of an observation mode and to control both the filter unit 210 and the illumination unit 18 in accordance with this user specification. The controller 66 can thus set an operating state of the imaging device 14 and a corresponding illumination mode of the illumination unit 18 in accordance with an observation mode selected by the user.

[0123] Fig. 6 shows a schematic representation of yet another embodiment of the imaging device 10. The imaging device 10 comprises an illumination device 12 with an optical interface 16 and an illumination unit 18, as well as an imaging device 14 connected to the optical interface 16. The imaging device 14 comprises a proximal base unit 310. The proximal base unit 310 is connected to the optical interface 16 of the illumination device 12. Illumination light generated by the illumination device 12 can thus be supplied to the proximal base unit 310. The imaging device 14 further comprises a controller 66, which in some embodiments can be integrated into the base unit 310.

[0124] Different interchangeable shafts 312, 314 can be optically and electronically coupled to the proximal base unit 310. The base unit 310 has an interface 316 for coupling different interchangeable shafts 312, 314. This interface 316 supplies the illumination light coming from the illumination device 12 to a coupled interchangeable shaft 312, 314. Furthermore, the interface 316 is configured to electrically supply a coupled interchangeable shaft 312, 314 and / or to electronically connect it to the controller 66 of the imaging device 14.

[0125] The interchangeable shafts 312, 314 each have an integrated camera 318, 320 and integrated optical filters 322, 324. The integrated cameras 318, 320 are designed as tipcams. In the present case, the integrated camera 318 of a first interchangeable shaft 312 is configured for multispectral imaging. Furthermore, the integrated camera 310 of a second interchangeable shaft 314 is configured for fluorescence imaging. The optionally present optical filters 322, 324 can be adapted to this.

[0126] In other embodiments, interchangeable shafts can also be used that only include optical filters but no integrated camera. These can then be coupled to a proximal camera unit. In some cases, the proximal camera unit can then be designed without an additional filter unit. The selection of a specific optical filter or a specific observation mode can be achieved by selecting a suitably equipped interchangeable shaft.

[0127] The controller 66 is configured to detect a connected interchangeable shaft 312, 314. This can be done software-based, mechanically, and / or by sensor detection. Depending on the detected interchangeable shaft 312, 314, the controller 66 can then determine in which operating state or observation mode the imaging device 14 should be operated. The control unit 66 is also configured to set an illumination mode of the illumination unit 18. The control unit 66 is thus configured to set an illumination mode of the illumination unit 18 depending on the observation mode defined by a currently connected interchangeable shaft 312, 314.

[0128] In this case, the interchangeable shafts 312, 314 and the imaging device 10 are part of a medical imaging system 316. The medical imaging system 316 allows a user to select a suitable interchangeable shaft 312, 314, couple it to the base unit 310, and thus define a mode for the entire imaging device 10. Simply changing the interchangeable shaft 312, 314 thus automatically adapts the illumination device 18 to the desired image acquisition mode.

[0129] Fig. 7 shows a schematic perspective view of another embodiment of an imaging device 10'. The reference numerals of this embodiment are provided with apostrophes for differentiation. In this embodiment, the imaging device 10' is designed as an exoscopic imaging device. It comprises an illumination device 12' and an imaging device 14'. Their basic functionality corresponds to that described above, but in this embodiment, the imaging device 14' is designed as an exoscope.

[0130] Aspects of the above description can also be summarized or described as follows. Fig. 8 shows a schematic flow diagram of a method for generating illumination light for an imaging device 14 by means of an illumination device 12. The sequence of the method also follows from the above explanations. The illumination device 12 comprises an optical interface 16 for optically connecting an imaging device 14 and a illumination unit 18 which is configured to supply illumination light to the optical interface 16, wherein the illumination unit 18 comprises a plurality of independently selectably activatable lighting elements 20, 22, 24, 26, 28 which are configured to emit light according to different emission spectra in order to supply the illumination light.

[0131] The method comprises a step S11 of at least temporarily activating a first group of the light-emitting elements 20, 22, 24, 26, 28 to provide illumination light for multispectral imaging. Furthermore, the method comprises a step S12 of at least temporarily activating a second group of the light-emitting elements 20, 22, 24, 26, 28 to provide illumination light for fluorescence imaging. One of the light-emitting elements 20, 22, 24, 26, 28 is at least temporarily activated both during the at least temporary activation of the first group of the light-emitting elements 20, 22, 24, 26, 28 and during the at least temporary activation of the second group of the light-emitting elements 20, 22, 24, 26, 28.

[0132] Fig. 9 shows a schematic flow diagram of a method for operating an imaging device 10. The sequence of the method also follows from the above explanations. In a step S21, an imaging device 10 with an imaging device 14 is provided. In a step S22, illumination light is supplied to the imaging device 14. The supply of the illumination light to the imaging device 14 occurs according to a method as described with reference to Fig. 8.

[0133] Fig. 10 shows a schematic flow diagram of a method for operating an imaging device 10. The sequence of the method also follows from the above explanations. The method comprises a step S31 of providing an illumination device 12 for providing illumination light for an imaging device 14. The imaging device 14 comprises an optical interface 16 for optically connecting an imaging device 14 and a illumination unit 18 which is configured to supply illumination light to the optical interface 16. The illumination unit 18 is multimodal and can be operated in several different illumination modes. Furthermore, the method comprises a step S32 of providing an imaging device 14 which can be connected to the optical interface 16 of the illumination device 12.Furthermore, the method comprises a step S33 of automated coordination of an operating state of the imaging device 14 and an illumination mode of the illumination unit 18.

[0134] The following describes an aspect relating to the generation of representations from fluorescence images and anatomical images. Fig. 11 shows a schematic representation of a medical imaging device 410 according to this aspect. The medical imaging device 410 can, in principle, be constructed and / or designed like the imaging device 10 described above or also like the above imaging device 10'. In particular, reference is made to the above description with regard to the functioning of the components and details of the design of the imaging device 410. To explain the present aspect, it is expedient to describe the technical facts with reference to Fig. 11, which is to be understood purely schematically, and the other figures.

[0135] In the specific example, the imaging device 410 is an endoscope device, but can also be an exoscope device and / or a microscope device. The imaging device 410 comprises an illumination unit 412 with at least one light source 414. The illumination unit 412 can, for example, be designed as described above with reference to the illumination device 12. For the following description, it is assumed that the illumination unit 412 is designed in this way. However, this is to be understood purely as an example. Basically, the illumination unit 412 is configured to provide illumination light 432, 438, by means of which an object 418 to be imaged can be illuminated. This will be discussed in more detail below.

[0136] The imaging device 410 further comprises an image capture unit 420 with suitable image capture sensor technology 422. The image capture sensor technology 422 is configured here to capture images in both the visible and near-infrared ranges. For example, the image capture sensor technology 422 is sensitive at least in a range between 450 nm and 950 nm, and in some embodiments, in a range between 400 nm and 1000 nm.

[0137] The object 418 to be imaged is, for example, an anatomical structure, such as in a patient's cavity. The anatomical structure includes a region 454 marked with a fluorescent dye. Using the imaging device 410, a user can view the anatomical structure as well as the marked region 454 and distinguish it from the surrounding tissue based on the fluorescence of the marked region 454. As an example application, reference is made to the exposure of the marked region 454, although those skilled in the art will also be familiar with other applications of fluorescent tissue marking.

[0138] In the present case, the marked region 454 is marked with a fluorescent dye that absorbs light in the visible range and also fluoresces at least partially or exclusively in the visible range. An exemplary absorption spectrum and an exemplary emission spectrum are plotted in the upper region of Fig. 12 as solid and dash-dotted lines, respectively, in a diagram in which intensity is plotted against wavelength. To excite the dye, light is irradiated at a wavelength that lies in the absorption range of the dye. This is shown in Fig. 12 by a dashed line, with the relevant wavelength marked as a point on the x-axis. This is the illumination light 432, which is shown as an arrow in Fig. 11.

[0139] The dye is, for example, fluorescein. This dye can be conveniently excited at a wavelength of 430 nm or 460 nm. In particular, this can be done using the third light-emitting element 24 described above. Another example of the dye is Cy5. This dye can be conveniently excited at a wavelength of 660 nm. In particular, this can be done using the first light-emitting element 20 described above.

[0140] In order to be able to perform fluorescence imaging, the image acquisition unit 420 comprises an observation filter 424. This is mounted in front of the image acquisition sensor system 422 and / or integrated therein and ensures that the illumination light 432 is not remitted to the image acquisition sensor system 422 or at least not to individual sensors thereof, but exclusively or at least primarily light 434 emitted by the object 418 due to the fluorescence excitation of the dye.

[0141] The lower part of Fig. 12 shows a suitable transmission spectrum of the observation filter 424, which allows such fluorescence imaging. The observation filter 424 is configured to block light in a first spectral range 426 and to transmit light in a second spectral range 428, which differs from the first spectral range 426. The observation filter 420 can be selected depending on the dye used, or rather, the absorption spectrum and emission spectrum. As described above, the imaging device 410 can comprise a plurality of optical filters that can be selectively used as observation filters. As a result, fluorescence imaging for different fluorescent dyes can be performed using the imaging device 410.

[0142] The observation filter 424 with the transmission shown in Fig. 12 is a high-pass edge filter. In the region of the edge, the transmission behavior of the observation filter 424 changes abruptly, although the corresponding edge may still have a certain width. Under certain circumstances, the edge region may not fall into either of the two spectral ranges 426, 428.

[0143] Depending on the position, width, and shape of the absorption peak of the respective dye, a dual-edge observation filter can be used instead of a single-edge observation filter 424. This is illustrated by way of example in Fig. 13. The first spectral range 426' lies between two subranges of the second spectral range 428'. Thus, light is transmitted both above and below the first spectral range 426'. In this case, the observation filter only masks a subrange of the observable spectral range that lies in the visible range.

[0144] The image capture unit 420 is further configured to operate in a white light mode or an anatomy mode, in which the object 418 is illuminated by illumination light 438 that lies at least partially within the second spectral range 428 and in which light 440 remitted by the object 418 is detected. As schematically illustrated in Fig. 11, a portion of the remitted light 440 is blocked by the observation filter 424, and the portion that lies outside the first spectral range 426 or within the second spectral range 428 is passed through to the image capture sensor 422. If the first spectral range 426 is selected to lie in the visible range due to the dye used, a complete white light image cannot be captured, even if white light is irradiated. This can be achieved, for example, using a white light illuminating element, as described above.

[0145] In the following, reference is also made to Fig. 14. The image acquisition unit 420 is configured to acquire, in the manner described, fluorescence images 430 of the object to be imaged through the observation filter 424. Furthermore, the image acquisition unit 420 is configured to acquire, in the manner described, anatomical images 436 of the object 418 to be imaged. In the present case, the anatomical images 436 are images in which broadband illumination light is irradiated and remitted light is detected. Fig. 14 illustrates the case in which multiple anatomical images 436 and multiple fluorescence images 430 are acquired or obtained from corresponding image data over time.Specifically, these are either acquired sequentially, by alternately acquiring one or more anatomical images 436 and then one or more fluorescence images 430 (or vice versa) according to a specific logic, or by simultaneously acquiring anatomical images 436 and fluorescence images 430. This depends on the design and operating mode of the image acquisition sensor system 422.

[0146] The anatomical images 436 are not true white-light images due to their capture through the observation filter 424. Rather, they are obtained by processing individual images and / or color channels according to one or more of the principles described below. The imaging device 410 further comprises a representation generation unit 442 configured to generate a representation 444 from the fluorescence images 430 and the anatomical images 436. In the present case, this representation comprises a combination representation and / or an overlay representation based on a combination or overlay of one or more anatomical images 436 and one or more fluorescence images 430.

[0147] The representation generation unit 442 is configured to associate recorded fluorescence images 430 and recorded anatomical images 436 with one another. This can be done both for sequential image acquisition and for simultaneous image acquisition. In the illustrated case, this is a temporal association. The association is based on the information regarding the time at which the images were acquired, so that the representation 444 contains temporally related anatomical information and fluorescence information. For example, a moving image is made available to a user in real time.

[0148] As schematically illustrated in Fig. 11, the imaging device 410 includes a display 452, via which the generated representation 444 can be displayed to a user. This embodiment is to be understood purely as an example. In particular, in other embodiments, a display can be connected to the imaging device 410 merely via an interface but is not necessarily part of the same.

[0149] As mentioned, even with broadband illumination, image information is not available across the entire visible range when capturing the anatomical images 436 due to the observation filter 424. If regular white-light imaging were performed, the white-light image in question would be color-stained or might not correctly depict certain anatomical structures.

[0150] It is therefore intended that, in addition to image information in the visible range, image information from the near-infrared range is also used to generate the anatomical image 436. This allows the information that may not be available due to the observation filter 424 to be provided elsewhere.

[0151] Purely by way of example, this is shown in Fig. 15 for four color channels of an image, as it would result when illuminated with white light and near-infrared light. For the purposes of explanation, it should be assumed that certain anatomical structures or other image information can only be recognized in the blue (B), green (G), or red (R) color channel and / or individual image. If these color channels and / or individual images B, G, R were fully combined, a correct white light image of the object 418 would be created. However, since the imaging occurs through the observation filter 424, the image information from the blue color channel and / or blue individual image, for example, is missing. If only the green and red color channels and / or the green and red individual images were combined, an incomplete image would result. The present aspect, on the other hand, provides for the additional inclusion of a near-infrared color channel (NIR) and / or a near-infrared individual image.This may make it possible to identify details of the anatomical structure that would otherwise be visible in the unavailable spectral range (here, for example, in the blue range). If this additional image information is then combined with the available color channels and / or individual images, an anatomical image 436 can be obtained that accurately depicts the anatomy, even though it is not a true white-light image.

[0152] Anatomical images 436 obtained in this way are then combined with fluorescence images 34 in the manner described to generate the representation 444, which contains information regarding both an anatomy of the object 418 and a fluorescence of the marked area 454.

[0153] Below, various possibilities are described for how anatomical images 436 can be obtained for different configurations of the image acquisition sensor system 422 and / or different approaches to utilizing available information. It is assumed in each case that a multimodal illumination unit is used. This has, for example, and in accordance with the embodiment described above, a white-light image illumination element and a blue illumination element that emits, for example, at 430 nm or 460 nm, a red illumination element that emits, for example, at 660 nm, a far-red illumination element that emits, for example, at 765 nm or 770 nm, and a near-infrared illumination element that emits, for example, at 940 nm. As already mentioned, however, this primarily serves to clarify the explanation, and differently configured illumination units can also be used.

[0154] Fig. 16 illustrates a first example in which the image capture sensor system 422 comprises a two-chip image sensor. Imaging light is distributed between two sensors, as already described above by way of example. A first image sensor (VIS) operates in the visible range, for example at wavelengths up to approximately 700 nm, and a second image sensor (NIR) operates in the near-infrared range, for example at wavelengths from approximately 700 nm. In this example, fluorescein is used as the dye. This is excited using the blue light-emitting element. Due to the observation filter 424, blue light cannot be detected. In addition to the illumination for exciting the dye, illumination is provided, for example, using the far-red light-emitting element and / or the near-infrared light-emitting element.

[0155] Fluorescence images 430 are obtained using the VIS image sensor. Anatomical images 436 are obtained using the NIR image sensor. The anatomical image 436 is output as a grayscale image, so that acquisition in the near-infrared range, which is imperceptible to the user, does not pose an obstacle. The representation generation unit 442 thus generates a representation 444 that superimposes a grayscale anatomical image 436 and a fluorescence image 434. Both the fluorescence images 430 and the anatomical images 436 can be acquired in real time, for example, at at least 24 fps, at least 30 fps, or even at least 60 fps. Since different image sensors are used to acquire the fluorescence image 430 and the anatomical image 436, they can also be acquired simultaneously. Sequential acquisition is not required.

[0156] Fig. 17 illustrates a second example in which the image acquisition sensor system 422 also includes a two-chip image sensor. As before, a fluorescence image 430 is acquired using the VIS image sensor. However, unlike in the first example, anatomy images 436 and fluorescence images 430 are acquired sequentially. This makes it possible to use not only the NIR image sensor but also the VIS image sensor for the anatomy image 436.

[0157] The VIS image sensor provides three color channels 436B, 436G, and 436R, which can generally be used for the anatomical image 436. Due to the observation filter 424, the blue color channel 436B is hardly usable, or not usable at all, since little to no light remitted by the object 418 passes through the observation filter 424 in the corresponding spectral range. The blue color channel 436B is therefore disregarded here. However, both the green color channel 436G and the red color channel 436R of the VIS image sensor image data, as well as a near-infrared single image 436NIR from the NIR image sensor, are used to obtain the anatomical image 436. The anatomical image 436 is thus reconstructed by replacing the blue channel with the near-infrared channel or the near-infrared single image. A color anatomical image can thus be generated. However, due to the sequential acquisition of anatomical images 436 and fluorescence images 430, the frame rate is halved in this example.

[0158] In some applications, a lower frame rate may be acceptable for fluorescence information than for anatomical information. Assuming an available frame rate of 60 fps, for example, several anatomical images, such as two, three, or five, can be acquired consecutively before a fluorescence image is acquired. The representation generation unit then assigns the most recently acquired fluorescence image to each of the multiple anatomical images. For example, with this approach, 50 anatomical images and 10 fluorescence images can be acquired per second. The actual frame rate of the fluorescence information in this example is then only 10 fps, which may not be noticeable to the user due to the higher frame rate of the anatomical information.

[0159] Fig. 18 illustrates a third example. In this example, the image acquisition sensor system 422 comprises a single-chip image sensor (one-chip; OC). This uses a red, a green, and a blue channel and is also capable of detecting near-infrared light on all three channels. In the example, the dye Cy5 is excited. This occurs, for example, using a red light-emitting element.

[0160] Anatomical images 436 and fluorescence images 430 are acquired sequentially. For the fluorescence image 430, the image data from the image sensor can be used unchanged.

[0161] In this case, the observation filter 424 masks a first spectral range 426, which extends from approximately 600 nm to approximately 700 nm. The second spectral range 428 lies both above and below the first spectral range 426 (cf. the exemplary illustration in Fig. 13). Light in the first spectral range 428 is not detected due to the observation filter 424. If an anatomical image 436 is acquired, the red channel still provides a signal, since near-infrared light is captured by all three channels of the image sensor. In this example, the anatomical image 436 is calculated by combining the three channels in a suitable manner. An anatomical image 436 with the color channels R'G'B 1is obtained, for example, by using the red channel of the image sensor as the red channel of the anatomy image 436, i.e. R'=R, by using a difference between the green and red channels of the image sensor as the green channel G', i.e. G'=GR, and by using a difference between the blue and red channels of the image sensor as the blue channel of the anatomy image 436, i.e. B'=BR.

[0162] It is understood that any other combinations, in particular linear combinations, can be used. These can also be weighted if necessary depending on the response of the respective image sensor and / or the observation filter 424 used. As a further example, in the case of fluorescein excitation, in which blue light would be blocked, the color channels of an anatomical image 436 could be calculated as follows: R'=RB; G'=GB; B'=B.

[0163] Fig. 19 illustrates a fourth example. In this example, the image acquisition sensor system 422 comprises a four-chip image sensor with separate image sensors for a blue channel (B), a green channel (G), a red channel (R), and a near-infrared channel (NIR). Light coming from the object 418 is divided among these four sensors. In this example, an illumination unit can also be used which, instead of a white light illumination element, comprises a combination of three or more illumination elements by means of which light is mixed such that this can be used instead of white light. For example, a combination of a red, green, and blue illumination element can be used.Although these emit only in specific and possibly very narrow-band wavelength ranges, they can provide illumination for capturing a red, green, and blue image, thus creating an image that a user perceives as a white-light image. In this case, for example, the aforementioned single-color light elements are present, along with the aforementioned combination of colored light elements used instead of the white-light light element.

[0164] In principle, this configuration can be operated analogously to the first and second examples described above. In this case, for example, the observation filter 424 is arranged in front of the four-chip image sensor. A further variant that can be implemented using the system according to the fourth example will be explained below.

[0165] Fluorescein, for example, is again used as the dye. To excite the dye, light is irradiated by a blue light element. Remitted light is in the green range and is detected by the green image sensor. A filter integrated into the green image sensor and / or arranged in front of the image sensor, which makes it selective for green light, serves as an observation filter 424. The image from the green image sensor can thus be used as a fluorescence image 430, since the blue excitation light does not fall on the green image sensor.

[0166] To additionally obtain an anatomical image, illumination is provided, for example, by a near-infrared illuminator. Light is thus remitted from the object in both the blue and near-infrared ranges. Furthermore, no fluorescence of the dye used occurs in these ranges. The fluorescein dye emits predominantly in the green range, and its emission spectrum also extends into the red with low intensity. The green and red image sensors are therefore not used for the anatomical image 436. The anatomical image 436 is thus created by combining a blue individual image 436B recorded by the blue image sensor and a near-infrared individual image 436NIR recorded by the near-infrared image sensor.

[0167] It is understood that an analogous procedure can be followed if another dye is used which, for example, can be excited with green light and emits red light.

[0168] Fig. 20 shows a schematic flow diagram of a method for medical imaging. The flow of the method also follows from the above explanations. In a step S41, illumination light 438 is provided to illuminate an object 418 to be imaged. In a step S42, fluorescence images 430 of the object 480 to be imaged are recorded through an observation filter 424, which is configured to block light in a first spectral range 426, which includes visible light, and to transmit light in a second spectral range 428 different from the first spectral range 426. When recording the fluorescence images 430, illumination light 432, the spectrum of which lies at least partially within the first spectral range 426, is used as excitation light, and light 434 emitted by the object 418 to be imaged, the spectrum of which lies at least partially within the second spectral range 428, is detected.Furthermore, the method comprises a step S43 in which anatomical images 436 of the object 418 to be imaged are recorded, wherein, when recording the anatomical images 436, illumination light 438 whose spectrum lies at least partially within the second spectral range 428 is used and light 440 remitted by the object 418 to be imaged, whose spectrum lies at least partially within the second spectral range 428, is detected.

[0169] Fig. 21 shows a schematic representation of a computer program product 448 with a computer-readable medium 450. Program code is stored on the computer-readable medium, which program code is configured to cause one and / or all of the described methods to be carried out when executed in a processor.

[0170] List of reference symbols

[0171] 10 Imaging device

[0172] 12 Lighting device

[0173] 14 Imaging device

[0174] 16 optical interface

[0175] 18 lighting unit

[0176] 20 light elements

[0177] 22 lighting elements

[0178] 24 light elements

[0179] 26 light elements

[0180] 28 light elements

[0181] 30 beam splitters

[0182] 32 beam splitters

[0183] 34 Entrance page

[0184] 36 Entrance page

[0185] 37 Entrance page

[0186] 38 Entrance page

[0187] 40 Entrance page

[0188] 41 Entrance page

[0189] 42 Exit page

[0190] 44 Exit page

[0191] 46 Filter unit

[0192] 48 filters

[0193] 50 filters

[0194] 52 filters

[0195] 54 optical path

[0196] 56 light-emitting surface

[0197] 58 light-emitting surface

[0198] 60 light-emitting surface

[0199] 62 light-emitting surface

[0200] 64 light-emitting surface

[0201] 66 Control

[0202] 68 Camera unit

[0203] 70 Observation beam path

[0204] 72 filter sensor

[0205] 74 display unit

[0206] 76 Shaft - M -

[0207] 77 Optics

[0208] 78 lens

[0209] 80 lens

[0210] 82 lens

[0211] 84 lens

[0212] 86 lens

[0213] 88 lens

[0214] 90 beam splitter element

[0215] 92 beam splitter element

[0216] 94 beam splitter element

[0217] 96 beam splitter element

[0218] 98 Transmission spectrum

[0219] 100 transmission spectrum

[0220] 102 Transmission spectrum

[0221] 104 Transmission spectrum

[0222] 106 light guides

[0223] 108 Imaging sensors

[0224] 110 White light sensor

[0225] 112 Near-IR sensor

[0226] 114 Light Path

[0227] 116 distal section

[0228] 210 filter unit

[0229] 212 filter drive

[0230] 214 User interface

[0231] 310 base unit

[0232] 312 Exchangeable shaft

[0233] 314 Exchangeable shaft

[0234] 316 Imaging System

[0235] 318 Camera

[0236] 320 Camera

[0237] 322 filters

[0238] 324 filters

[0239] 410 Imaging device

[0240] 412 lighting unit

[0241] 414 Light source

[0242] 416 Illumination light

[0243] 418 Object

[0244] 420 Image acquisition unit 422 Image acquisition sensors

[0245] 424 observation filters

[0246] 426 first spectral range

[0247] 428 second spectral range 430 fluorescence image

[0248] 432 Illumination light

[0249] 434 Light

[0250] 436 anatomy image

[0251] 438 Illumination light 440 Light

[0252] 442 Representation generation unit

[0253] 444 Representation

[0254] 448 Computer program product

[0255] 450 computer-readable medium 452 advertisement

Claims

Claims 1. A medical imaging device (410), in particular an endoscope device, an exoscope device and / or a microscope device, comprising: - an illumination unit (412) having at least one light source (414) configured to provide illumination light (416) for illuminating an object (418) to be imaged; and - an image capture unit (420) comprising at least one image capture sensor (422) and at least one observation filter (424), wherein the observation filter (424) is configured to block light in a first spectral range (426) comprising visible light and to transmit light in a second spectral range (428) different from the first spectral range (426); wherein the image capture unit (420) is configured to capture fluorescence images (430) of the object (418) to be imaged through the observation filter (424), during the capture of which illumination light (432), the spectrum of which lies at least partially within the first spectral range (426), is used as excitation light, and during the capture of which light (434) emitted by the object (418) to be imaged, the spectrum of which lies at least partially within the second spectral range (428), is detected by the image capture sensor system (422),and wherein the image acquisition unit (420) is configured to acquire anatomical images (436) of the object (418) to be imaged, during the acquisition of which illumination light (438) is used, the spectrum of which lies at least partially within the second spectral range (428), and during the acquisition of which light (440) remitted by the object (418) to be imaged, the spectrum of which lies at least partially within the second spectral range (428), is detected by the image acquisition sensor system (422); and, - a representation generation unit (442) which is configured to generate at least one representation (444) from the fluorescence images (430) and the anatomical images (436).

2. The medical imaging device (410) according to claim 1, wherein the image acquisition unit (24) is configured to capture the anatomical images (436) of the object (84) to be imaged through the observation filter (424).

3. Medical imaging device (410) according to one of the preceding claims, wherein a smallest wavelength of the first spectral range (426) is greater than 430 nm and in particular greater than 500 nm.

4. Medical imaging device (410) according to one of the preceding claims, wherein the at least one representation (444) comprises a combination representation, in particular an overlay representation, which is based on a combination of fluorescence images (34) and anatomical images (436).

5. The medical imaging device (410) according to any one of the preceding claims, wherein the image acquisition unit (420) is configured to acquire the fluorescence images (430) and the anatomical images (436) in real time, and wherein the representation generation unit (442) is configured to generate the at least one representation (444) in real time.

6. Medical imaging device (410) according to one of the preceding claims, wherein the image acquisition unit (24) is configured to record the fluorescence images (430) and the anatomical images (436) sequentially, wherein the representation generation unit (442) is configured to associate sequentially recorded fluorescence images (430) and anatomical images (436) with one another, in particular temporally, and to generate the at least one representation (444) based on associated fluorescence images (430) and anatomical images (436).

7. The medical imaging device (410) according to any one of the preceding claims, wherein the image acquisition sensor system (422) comprises at least one first image acquisition sensor (VIS, NIR, OC, R, G, B) configured to acquire image information for the fluorescence images (430), and wherein the image acquisition sensor system (422) comprises at least one second image acquisition sensor (VIS, NIR, OC, R, G, B) configured to acquire image information for the anatomical images (436). The medical imaging device (410) according to any one of the preceding claims, wherein the image acquisition sensor system (422) is configured to simultaneously acquire fluorescence images (430) and anatomical images (436). The medical imaging device (410) according to any one of the preceding claims, wherein the representation generation unit (442) is configured to generate at least one anatomical image (436) from a plurality of, in particular single-color, individual images (436B', 436NIR 1) and / or from multiple color channels of anatomical images (436B, 436G, 436R, 436NIR). Medical imaging device (410) according to one of the preceding claims, wherein the anatomical image (436) comprises at least two color channels, of which at least one lies in the visible range and at least one lies outside the visible range, in particular in the near-infrared range. Medical imaging device (410) according to one of the preceding claims, wherein the anatomical image (436) corresponds to a white light image with multiple color channels, in which at least one color channel from the visible range is replaced by at least one color channel that lies outside the visible range, in particular in the near-infrared range.The medical imaging device (410) according to one of the preceding claims, wherein the illumination unit (412) is multimodal and comprises a plurality of independently selectably activatable lighting elements (20, 22, 24, 26, 28) configured to emit light according to different emission spectra to provide the illumination light (438). The medical imaging device (410) according to one of the preceding claims, wherein the image acquisition unit (420) is configured for multispectral and / or hyperspectral imaging. Method for medical imaging, in particular with a medical imaging device (410) according to one of the preceding claims, comprising: Providing illumination light (438) for illuminating an object (418) to be imaged; Recording fluorescence images (430) of the object (480) to be imaged through an observation filter (424) configured to block light in a first spectral range (426) comprising visible light and to transmit light in a second spectral range (428) different from the first spectral range (426), wherein, when recording the fluorescence images (430), illumination light (432) whose spectrum lies at least partially within the first spectral range (426) is used as excitation light, and light (434) emitted by the object (418) to be imaged, whose spectrum lies at least partially within the second spectral range (428), is detected;and capturing anatomical images (436) of the object (418) to be imaged, wherein, when capturing the anatomical images (436), illumination light (438) whose spectrum lies at least partially within the second spectral range (428) is used, and light (440) remitted by the object (418) to be imaged, whose spectrum lies at least partially within the second spectral range (428), is detected. Program code configured, when executed in a processor, to effect implementation of a method according to claim 14. A computer program product (448) comprising a computer-readable medium (450) on which program code according to claim 15 is stored.

Images