Medical imaging device, method for operating same, and method of medical imaging

Abstract

The invention relates to a medical imaging device (10), comprising: a spatially and spectrally resolving image capture unit (12), which comprises at least one optics (14) and at least one image detection sensor system (16) coupled to the optics, which are designed to perform an image capture of an image region (18), in the course of which spatially and spectrally resolved image data are generated, which comprise both spatial and spectral information; an evaluation unit (20), which is designed to perform an analysis of the spatially and spectrally resolved image data that is based on spatial and spectral information and based on at least one analysis parameter calculated from the spatially and spectrally resolved image data; a fault detection unit (22), which is designed to detect the presence of a fault in the course of image capture, independently of the analysis of the image data and the calculated analysis parameter, and to determine a fault state of the image capture; and an output unit (24), which is designed to generate, according to the fault state, a user output which is based on the analysis parameter. The invention also relates to a method for operating a medical imaging device (10) and to a method of medical imaging.

Description

[0001] Medical imaging device, method for operating such a device and method for medical imaging

[0002] The invention relates to a medical imaging device, a method for operating a medical imaging device and a method for medical imaging.

[0003] Medical imaging devices such as endoscopic or exoscopic devices that generate multispectral or hyperspectral images are known from the prior art. Multispectral or hyperspectral images have, in addition to two spatial dimensions, such as a conventional camera image, a spectral dimension. The spectral dimension encompasses several spectral bands (wavelength bands). Multispectral and hyperspectral images differ primarily in the number and width of their spectral bands.

[0004] Several imaging devices for generating such multispectral or hyperspectral images are known, particularly in the context of medical applications. For example, DE 20 2014 010 558 U1 describes a device for capturing a hyperspectral image of an examination area of ​​a body. The device includes an input lens for generating an image in an image plane and a slit-shaped aperture in the image plane for masking out a slit-shaped region of the image. The light passing through the aperture is spread out by a dispersive element and recorded by a camera sensor. As a result, the camera sensor can record a plurality of spectra, each with an associated spatial coordinate, along the longitudinal direction of the slit-shaped aperture.The described device is further configured to record additional spectra along the longitudinal direction of the slit-shaped aperture in a direction different from the longitudinal direction of the slit-shaped aperture. The method underlying this disclosure for generating multispectral or hyperspectral images is also known as the so-called pushbroom method.

[0005] In addition to the pushbroom method, there are other methods for generating multispectral or hyperspectral images. In the so-called whiskbroom method, the examination area or object is scanned point by point, and a spectrum is obtained for each point. In contrast, the staring method acquires multiple images with the same spatial coordinates. Different spectral filters and / or illumination sources are used from image to image to resolve spectral information. Furthermore, there are methods in which a two-dimensional multicolor image is decomposed into several individual spectral images using suitable optical elements such as optical slicers, lenses, and prisms. These individual images are simultaneously acquired on different detectors or detector areas. This is sometimes referred to as the snapshot approach.

[0006] As described in DE 10 2020 105458 A1, multispectral and hyperspectral imaging devices are particularly suitable as endoscopic imaging devices. In this context, multispectral and / or hyperspectral imaging is a fundamental field of application, for example, for diagnostics and for assessing the success or quality of a procedure.

[0007] The robustness and low susceptibility to errors of the imaging device are crucial to enabling users to achieve high quality and reliability in the interpretation of multispectral and hyperspectral images based on the imaging device. This robustness and susceptibility to errors can depend on the influence of disturbances during the image acquisition process. Common disturbances in this context can include, for example, smoke following thermal manipulation of tissue, dirty or fogged lenses of the imaging device, or inorganic materials in the image area (instruments, clamps, threads, etc.). These disturbances can lead to disturbances in image acquisition, which can complicate the reliable interpretation of multispectral and hyperspectral images.

[0008] Based on the prior art, the present invention is based on the object of providing an imaging device and an imaging method by means of which disturbances and / or disturbance states can be detected during image acquisition.

[0009] An imaging device according to the invention can comprise a spatially and spectrally resolved image acquisition unit, which comprises at least one optical system and at least one image acquisition sensor coupled to the optical system, which are configured to perform an image acquisition of an image region, generating spatially and spectrally resolved image data that includes both spatial and spectral information. Furthermore, the imaging device can comprise an evaluation unit configured to create an analysis of the spatially and spectrally resolved image data, which analysis is based on spatial and spectral information and is based on at least one analysis parameter calculated from the spatially and spectrally resolved image data.Furthermore, the imaging device may comprise a fault detection unit configured to detect the presence of a fault during image acquisition and to determine a fault state of the image acquisition. Finally, the imaging device may comprise an output unit configured to generate a user output based on the analysis parameter in accordance with the fault state.

[0010] According to one aspect, a medical system is provided that includes the imaging device and a medical instrument.

[0011] Furthermore, the present invention can relate to a method for operating a medical imaging device, wherein the medical imaging device comprises a spatially and spectrally resolved image acquisition unit. The image acquisition unit comprises at least one optical system and at least one image acquisition sensor coupled to the optical system, which are configured to perform an image acquisition of an image region, in which spatially and spectrally resolved image data are generated. The image data comprise both spatial and spectral information. This can be an imaging device according to the invention. The method can comprise the step of acquiring spatially and spectrally resolved image data of an image region using the medical imaging device.Furthermore, the method may include the step of creating an analysis of the spatially and spectrally resolved image data, which analysis is based on spatial and spectral information and is based on at least one analysis parameter calculated from the spatially and spectrally resolved image data. Furthermore, the method may include performing a disturbance detection, in which the presence of a disturbance during image acquisition is detected and in which a disturbance state of the image acquisition is determined. The method may further include the step of generating a user output based on the disturbance state, which is based on the analysis parameter.

[0012] Furthermore, the present invention can comprise a method for medical imaging. The method can be carried out using an imaging device according to the invention and / or using a medical system according to the invention. Such a method can comprise performing an image acquisition of an image region, generating spatially and spectrally resolved image data that includes both spatial and spectral information. Furthermore, one step of such a method can be creating an analysis of the spatially and spectrally resolved image data, which analysis is based on spatial and spectral information and is based on at least one analysis parameter calculated from the spatially and spectrally resolved image data. Furthermore, the method can comprise the step of performing a disturbance detection, detecting the presence of a disturbance during the image acquisition and determining a disturbance state of the image acquisition.Furthermore, the method may comprise the step of generating a user output in accordance with the fault condition based on the analysis parameter.

[0013] The features according to the invention enable the reliable implementation and / or assessment of diagnostic and / or therapeutic actions. In particular, a high degree of quality in multispectral and / or hyperspectral imaging can be achieved, since any disturbances in the display of both spatially and spectrally resolved information in the form of spatially and spectrally resolved image data can be detected. By displaying disturbances in image acquisition and disturbance states in image acquisition to the user as such, misinterpretation by the user can be avoided.Referring to an application example in which the present invention is part of a medical system, misinterpretation of physiological tissue parameters such as, among others, oxygen saturation, blood, water or fat content, which underlie the implementation and / or assessment of therapeutic and / or diagnostic measures, can be prevented.

[0014] The fault detection unit can be configured to detect the presence of a fault during image acquisition and to determine a fault state of the image acquisition, independently of the analysis of the image data and the calculated analysis parameter. Alternatively or additionally, in the method according to the invention for operating a medical imaging device and / or in the method according to the invention for medical imaging, the step of performing a fault detection can comprise detecting the presence of a fault during image acquisition and determining a fault state of the image acquisition during the fault detection, independently of the analysis of the image data and the calculated analysis parameter. The imaging device can be a microscopic, macroscopic, and / or exoscopic imaging device. The imaging device can be designed as a microscope, macroscope, and / or exoscope and / or comprise such a microscope.In some embodiments, the imaging device may be an endoscopic imaging device. The imaging device may be an endoscope device. It may comprise an endoscope and / or an endoscope system and / or be configured as such and / or form at least a part and preferably at least a major part and / or a main component of an endoscope and / or an endoscope system. "At least a major part" may mean at least 55%, preferably at least 65%, more preferably at least 75%, particularly preferably at least 85%, and most preferably at least 95%, in particular with reference to a volume and / or a mass of an object.

[0015] In some embodiments, the imaging device is configured to be insertable into a cavity for inspection and / or observation, for example, into an artificial and / or natural cavity, such as the interior of a body, a body organ, tissue, or the like. The imaging device can also be configured to be insertable into a housing, casing, shaft, pipe, or other, particularly artificial, structure for inspection and / or observation.

[0016] In particular, if the imaging device is an exoscopic imaging device, it may be configured to acquire tissue parameters, images of wounds, images of body parts, etc. For example, the imaging device may be configured to image a surgical field.

[0017] The imaging device, and in particular the optics and / or the image acquisition sensor system, can be configured for multispectral and / or hyperspectral imaging, specifically for capturing and / or generating multispectral and / or hyperspectral image data. Multispectral imaging or multispectral image data can refer in particular to imaging in which at least two, in particular at least three, and in some cases at least five spectral bands can be and / or are captured independently of one another. Hyperspectral imaging or hyperspectral image data can refer in particular to imaging in which at least 20, at least 50, or even at least 100 spectral bands can be and / or are captured independently of one another.The imaging device may operate according to the pushbroom method and / or the whiskbroom method and / or the staring method and / or a snapshot principle.

[0018] In some embodiments, the imaging device comprises a white-light camera and / or sensors for white-light image acquisition. The imaging device can be configured for white-light imaging in addition to spectrally resolved imaging. Separate optics and / or shared optics can be used for this purpose. The white-light imaging and the spectrally resolved imaging can be performed simultaneously or alternately, or at times simultaneously and at times sequentially.

[0019] For some applications, it can be advantageous to be able to use a high spectral resolution. In this case, hyperspectral imaging is a good option. This can be combined with white light imaging. This enables real-time observation via a white light image, even if the acquisition of spectrally resolved image data only takes place essentially in real time, i.e., several seconds are needed to create a spectrally resolved image. For some applications, it can be advantageous to generate spectral image data in real time. This includes, for example, generating a spectrally resolved image in less than a second or even several times per second. In this case, it can be useful to use multispectral imaging. A possibly lower spectral resolution is then offset by a higher frame rate.Depending on the application, it may be sufficient to consider only a few different spectral ranges and / or wavelengths, for example, two, three, four, or generally fewer than ten. In this case, additional white-light imaging can optionally be omitted. Spectrally resolved image data acquired in real time or delivering multiple images per second can also be used for surveillance purposes. It is not necessary to create a single image for a user to display; the image data can also be processed in the background.

[0020] The medical imaging device can have at least a proximal section, a distal section and / or an intermediate section. The distal section is designed in particular to be introduced into and / or located in a cavity to be examined in an operating state, for example during the diagnostic and / or therapeutic action. The proximal section is designed in particular to be arranged outside the cavity to be examined in an operating state, for example during the diagnostic and / or therapeutic action. “Distal” should be understood in particular to mean facing towards a patient and / or facing away from a user during use. “Proximal” should be understood in particular to mean facing away from a patient and / or facing away from a user during use. In particular, proximal is the opposite of distal.The medical imaging device has, in particular, at least one, preferably flexible, shaft. The shaft can be an elongate object. Furthermore, the shaft can at least partially and preferably at least largely form the distal section. An “elongate object” is to be understood, in particular, as an object whose main extent is at least a factor of five, preferably at least a factor of ten, and particularly preferably at least a factor of twenty greater than a greatest extent of the object perpendicular to its main extent, i.e., in particular, a diameter of the object. A “main extent” of an object is to be understood, in particular, as its longest extent along its main extension direction.A “main extension direction” of a component is to be understood in particular as a direction which runs parallel to a longest edge of a smallest imaginary cuboid which just completely encloses the component.

[0021] The image acquisition unit can be arranged at least partially and preferably at least largely in the region of the proximal section and / or form this. In other embodiments, the image acquisition unit can be arranged at least partially and preferably at least largely in the distal section and / or form this. Furthermore, the image acquisition unit can be arranged at least partially distributed between the proximal section and the distal section. The image acquisition sensor system has, in particular, at least one image sensor. Furthermore, the image acquisition sensor system can also have at least two and preferably more image sensors, which can be arranged one behind the other.Furthermore, the two and preferably more image capture sensors can have spectral capture sensitivities that differ from one another, so that, for example, a first sensor is particularly sensitive in a red spectral range, a second sensor in a blue spectral range, and a third sensor in a green spectral range, or comparatively more sensitive than the other sensors. The image sensor can be designed, for example, as a CCD sensor and / or a CMOS sensor. The optics of the image capture unit can comprise suitable optical elements such as lenses, mirrors, gratings, prisms, optical fibers, etc. The optics can be configured to guide object light coming from the image area to the image capture sensor system, for example, to focus and / or project it. The object light can, in particular, originate from illumination of the image area.

[0022] The image acquisition unit is particularly configured to generate at least two-dimensional spatial image data. The image acquisition unit can be spatially resolving in such a way that it delivers a resolution of at least 100 pixels, preferably of at least 200 pixels, preferably of at least 300 pixels, and advantageously of at least 400 pixels in at least two different spatial directions. The image data is preferably at least three-dimensional, with at least two dimensions being spatial dimensions and / or with at least one dimension being a spectral dimension. A plurality of spatially resolved images of the image area can be obtained from the image data, each of which is assigned to different spectral bands. The spatial and spectral information of the image data can be such that an associated spectrum can be obtained for a plurality of spatial pixels.

[0023] In some embodiments, the image acquisition unit is configured to continuously generate updated image data. For example, the image acquisition unit may be configured to generate the image data substantially in real time, which may include, for example, generating updated image data at least every 30 seconds, in some cases at least every 20 seconds, and in some cases even at least every 10 seconds or at least every 5 seconds.

[0024] The image area may include at least a part and / or section of an imaged object. The image area may relate to tissue and / or organs and / or a part of a patient's body. The image area may relate to a site.

[0025] The imaging device can comprise an illumination device that includes at least one illuminant configured to illuminate and / or illuminate the image area in at least one operating state. The illuminant can comprise a white light source, a particularly tunable monochrome light source, a laser, a white light laser, at least one light-emitting diode and / or a light-emitting diode array, at least one laser diode and / or a laser diode array, or the like. The illumination device can be formed integrally with the image capture unit. In particular, the illumination device can utilize individual or all components of the optics of the image capture unit and / or have separate illumination optics. An illumination light beam can be guided and / or guideable, at least in sections, coaxial with a measurement light beam.

[0026] The analysis can be based on processing both spatial and spectral information to determine the analysis parameter. The analysis parameter can be obtained, in particular, from spectral information assigned to the image area using a mathematical rule. The analysis parameter can, in particular, assume a continuous value range from 0-100% or a discrete value range 0, 1, 2, ... The mathematical rule can include an algebraic calculation rule and / or a machine learning method, e.g., a KL model. The analysis parameter is based, in particular, on a spectral value for a specific wavelength and / or a specific wavelength range and / or a specific spectral band.

[0027] In some embodiments, the analysis may include a parameter image. The analysis parameter may, in particular, be spatially resolved. The parameter image may, for example, display the value of the analysis parameter as a function of an image coordinate. Alternatively or additionally, the analysis may provide a spectrum. In this case, the analysis parameter may, for example, be a spectrally resolved intensity value.

[0028] It can be provided that the disturbance is not recognizable and / or detected from spatial and spectral information, and / or that the disturbance detection unit does not rely, or at least not exclusively, on spatial and spectral information. Disturbance detection can be based on spatial and spectral information. Disturbance detection can detect disturbances using a mathematical rule. The mathematical rule can comprise an algebraic calculation rule and / or a machine learning method, for example a KL model. The mathematical rule can also comprise, for example, at least one filter rule that can specifically modify image data using algorithms. Furthermore, at least one optical filter can be integrated into the imaging device.Furthermore, a mathematical rule can be used to calculate a comparison parameter from spatial and spectral image data, which can be compared with other parameters that are based on interference-free imaging.

[0029] Interferences may include smoke, contamination and / or wear of the imaging device, in particular the optics of the imaging device, inorganic materials, in particular instruments, nets, tilters, threads, trocars, overexposed and / or underexposed image areas, contrast agents, motion artifacts, in particular motion artifacts resulting from a relative movement of the imaging device to the image area, incorrect white balance in white light imaging as provided in some embodiments, and / or an endoscope tip located in the trocar during image data generation by the imaging device. Another form of interference can occur during images taken in / under water, for example in arthroscopy or urology. Particles suspended in the water, such as urine or blood, can obscure the view of the tissue, which can also lead to distorted parameters.This case is in some ways similar to the case of smoke described here and below, but in a different medium.

[0030] The imaging device, and in particular the fault detection unit, can each comprise at least one processor and / or an associated memory with program code that implements the described functions and steps, and / or an associated main memory and / or associated connections and / or data interfaces, and / or an electronic circuit for implementing the functional units mentioned herein and / or for performing the method steps mentioned herein. One or more processors, memories, main memories, connections, data interfaces, and / or circuits can also be associated with one or more functional units and / or implement one or more method steps.

[0031] The output unit can be configured to output a visual output and / or any other output perceivable by a user. For this purpose, the output unit can comprise suitable components, such as one or more lamps, illuminants, loudspeakers, screens, or the like. The output unit can comprise a computer and / or processor and / or memory and / or RAM and / or connections and / or a data interface for receiving, processing, and outputting unprocessed, preprocessed, and / or processed output data. The imaging device can comprise a display unit configured to display an image, in particular a moving image, for a user. The display unit can be part of the output unit and / or form it. The displayed image can be based on the image data. The display unit can comprise a screen and / or control electronics.The display unit may comprise a computer and / or processor and / or memory and / or random access memory and / or ports and / or a data interface for receiving, processing, and outputting unprocessed, preprocessed, and / or processed image data and / or display data. The output generation unit may be connected to the display unit via an interface. The generated output may be processed and / or output by the display unit.

[0032] The output unit can be configured to modify a user output based on the fault state of the imaging device. For example, after detecting a fault state, the user can be alerted to a fault state. In particular, spatial and spectral information that was acquired under the influence of a fault state can be deliberately omitted and / or identified in an image displayed on the display unit. Examples of this include color coding and / or shaping, for example symbols and / or geometric figures. For example, an image line that was acquired under the influence of a fault state of the imaging device can be marked and output as a black, red, or otherwise colored image line. A faulty image line can occur, for example, in the case of imaging using the pushbroom method.Furthermore, it can be provided that, depending on a fault condition, image data acquired under the influence of a fault condition are acquired again. In further embodiments, parameters of the imaging device, in particular the optics, the image acquisition sensor system, the illumination device, and / or a white balance, can be adjusted and / or optimized according to a fault condition.

[0033] The disturbance detection unit can be configured to detect the presence of a disturbance based on an assessment of the spatially and spectrally resolved image data. This makes it easy to infer the presence of a disturbance from available data. In particular, a disturbance can be detected independently of white light imaging. The disturbance detection unit can be configured to process spatial and spectral information according to at least one mathematical rule. For example, by applying a mathematical rule, a comparison parameter can be calculated from the image data and compared with parameters in an expected range for image acquisition without disturbance. This parameter can be calculated, in particular, from spectral information.

[0034] The disturbance detection unit can also be configured to detect an image exposure state based on the spatially and spectrally resolved image data and to detect the presence of a disturbance if the exposure state represents underexposure and / or overexposure, at least in part. This can prevent a user from drawing incorrect conclusions based on falsified spectra. Overexposure occurs, for example, when the image capture sensor, in particular a CCD sensor and / or a CMOS sensor, is exposed to an exposure that oversaturates the image capture sensor. This can mean that an image capture sensor-specific threshold for detecting exposure has been exceeded. Underexposure can mean exposure of the image capture sensor below a threshold that is necessary for detecting exposure, possibly for distinguishing between neighboring image data points and / or image data lines.

[0035] A disturbance can in particular relate to the presence of inorganic material in the image area. The disturbance detection unit can be configured to detect a disturbance due to inorganic material in the image area. In particular, this can prevent misinterpretations of medical devices. Inorganic material can include, among other things, instruments, trocars, nets, clippers and / or threads. Detection of the inorganic material can be based on processing spectral information, in particular spectral information at at least one, for example two or three or four, specific wavelength and / or in at least one, for example two or three or four, specific wavelength range, and comparing the spectral information with a known spectrum and / or with known selected values ​​that are / are associated with inorganic material.This can be particularly useful for distinguishing between organic and inorganic material. In principle, a distinction between organic and inorganic material can be made in many ways, particularly those already described.

[0036] Furthermore, the medical imaging device can comprise a video acquisition unit comprising a camera. The camera can be configured to generate video image data of the image area. The video acquisition unit can be arranged, in particular, in addition to the spectrally resolved imaging. The video acquisition unit can use all units of the imaging device, in particular units such as the optics, the illumination device, and / or output unit, together with the imaging device. In other embodiments, the video acquisition unit can be designed partially or completely separately. The disturbance detection unit can be configured to detect the presence of a disturbance based on an image analysis of the video image data. This allows the presence of disturbances to be detected depending on the situation and at least substantially in real time.The spatial and spectral information can be used in addition to the image analysis of the video data for fault detection. Fault detection that detects a fault solely based on image analysis of the video data is also possible according to the invention. In particular, it can be advantageous for the video data to be available essentially in real time, and preferably in real time, for example, with a time delay of a few milliseconds. The image analysis can include a mathematical rule, in particular an algebraic calculation rule and / or a machine learning method, e.g., a KL model.

[0037] Incorrect or inaccurate recordings can be effectively prevented, in particular, if the disturbance detection unit of the medical imaging device is configured to determine a range of movement of the camera relative to the image area based on the image analysis of the video image data and to detect the presence of a disturbance if the determined range of movement exceeds a threshold value. Because the multispectral and hyperspectral imaging of a single image may occur over a period of time in which, due to the movement, a spatial coordinate of an image data point at the beginning of the imaging no longer corresponds to the spatial coordinate of the same image data point during the imaging period, a spatial coordinate of the image can no longer be assigned to the image area. By analyzing the video image data, a movement of the camera relative to the image area during the imaging can be determined.It may be advantageous if the video image data can be acquired at a higher repetition rate than the multispectral and hyperspectral image data. In particular, a threshold value can be defined by a user that represents an acceptable spatial displacement of an image data point during imaging. According to one embodiment, the range of movement of the camera relative to the image area, determined based on the image analysis of the video image data, can be compared with this threshold value to detect a disturbance in image acquisition.

[0038] Furthermore, the fault detection unit of the medical imaging device can be configured to detect the presence of contamination and / or fogging on at least part of the optics of the spatially and spectrally resolving image acquisition unit based on the image analysis of the video image data, and to detect the presence of a fault based on the detection of contamination and / or fogging. This can prevent falsifications due to contamination and / or fogging on at least part of the optics.

[0039] Furthermore, precise spectral calibration of the imaging device can be particularly important to achieve optimal image quality. Spectral calibration can be performed, for example, by white balance. According to one embodiment, the video capture unit can also be calibrated using white balance.

[0040] The fault detection unit can be configured to assess, based on a comparison of the spatially and spectrally resolved image data and the video image data, whether a white balance of the spatially and spectrally resolved image acquisition unit and a white balance of the video acquisition unit match at least within a predefined tolerance, and to detect the presence of a fault if the predefined tolerance is exceeded. Because it is typically less complex to perform a white balance of the video acquisition unit than a white balance of the spatially and spectrally resolved image acquisition unit, it can be advantageous to first perform a white balance of the video acquisition unit and use the video image data as a reference value when comparing with the spatially and spectrally resolved image data.If a fault is detected, the user can be prompted to perform a white balance check of the spatially and spectrally resolving image capture unit again. According to a further embodiment, a white balance check of the spatially and spectrally resolving image capture unit can be performed automatically in accordance with a fault condition.

[0041] Furthermore, the interference detection unit of the medical imaging device can be configured to detect the unexpected presence of a medical instrument in the image area based on the image analysis of the video image data and to detect the presence of an interference based on the detection of an unexpected presence of a medical instrument. This can alert a user that falsifications of the images could occur due to medical instruments. In particular, the interference detection unit can be configured to compare a detected medical instrument with a medical instrument expected in the image area and to detect the presence of an interference if the detected medical instrument deviates from the expected medical instrument.Furthermore, depending on the detection of the presence of a medical instrument, the portion of the image area in which the medical instrument is present can be excluded from the parameter representation. This may include excluding said portion from underlying calculations.

[0042] Furthermore, the disturbance detection unit of the medical imaging device can comprise a distal end that comprises at least parts of the optics of the image acquisition unit, wherein the disturbance detection unit is configured to detect that the image area at least partially comprises the interior of a trocar, and wherein the output unit is configured to generate a user output that contains the information that the distal end is at least partially located within the trocar. During imaging, the imaging device can be located in such spatial proximity to the trocar that a disturbance of the spatial and spectral information can arise. In particular, incident light can be reflected by the trocar to cause a disturbance of the spatial and spectral information.By detecting unintentional positioning partially within the trocar, corresponding image corruption can be prevented. According to one embodiment, the medical imaging device can comprise a fluorescence imaging unit configured to capture the image area using fluorescence imaging and generate fluorescence imaging data, wherein the disturbance detection unit is configured to detect the presence of a disturbance based on an assessment of the fluorescence imaging data. Individual wavelengths of the spectral information can lie within the emission spectrum of the fluorescence imaging unit and therefore potentially be influenced by the fluorescence imaging unit. For example, the measured intensity of these wavelengths can be altered by the fluorescence imaging unit, which can result in a disturbance in the image acquisition.The analysis unit could then calculate incorrect analysis parameters due to the interference. In particular, a tissue located in the image area could be incorrectly assigned to a different tissue type.

[0043] The interference detection unit of the medical imaging device can further be configured to detect the presence of contrast agent in at least part of the image area based on the fluorescence imaging data, wherein the output unit can be configured to generate the user output based on the analysis parameter in such a way that parts of the image area in which the interference detection unit detects the presence of contrast agent are omitted. This can, for example, ensure that parts of the image area that are not subject to fluorescence-related interference can be subjected to analysis. This is advantageous because it enables multimodal imaging, potentially allowing the advantages of each mode to be specifically selected for a tissue to be examined.For example, contrast agent can be specifically delivered to one tissue type and thus made accessible to fluorescence imaging, while the remaining tissue types within the same image area remain accessible to hyperspectral and / or multispectral imaging.

[0044] Furthermore, the medical imaging device can comprise a sensor unit with at least one sensor configured to measure at least one measured variable describing a state of the imaging device and / or an environment of the imaging device, and configured to generate a sensor signal representing the measured variable. In this context, the sensor unit can operate independently of the imaging device. The sensor unit can operate simultaneously with the imaging device and generate a sensor signal and / or before or after the acquisition of multispectral and / or hyperspectral information. The disturbance detection unit can be configured to detect the presence of a disturbance based on the sensor signal.

[0045] As already described, a relative movement may occur between the imaging device and the image area. In particular, to detect this relative movement, the sensor may be an acceleration sensor, with the sensor signal representing a movement of the imaging device. In this case, the disturbance detection unit may be configured to determine a range of motion based on the sensor signal and to detect the presence of a disturbance if the determined range of motion exceeds a threshold value.

[0046] A distance between the imaging device and an object in the image area that is too small and / or too large can also cause image capture disruption. Particularly in this context, the sensor can be a distance sensor. This can be configured to determine the distance between the imaging device and an object in the image area.

[0047] A therapeutic action, for example thermal manipulation of the tissue, in particular vessel sealing or coagulation using an electrosurgical device such as a high-frequency coagulation instrument, can cause smoke in the image area. Smoke can alter spectral information of the image area before acquisition with the imaging device and thus represent a disturbance. This can lead to analysis parameters being incorrectly calculated by the evaluation unit of the imaging device. The disturbance detection unit of the medical imaging device can be configured to detect the presence of smoke in the image area based on image analysis and to detect the presence of a disturbance based on the detection of smoke in the image area. The disturbance detection unit can detect smoke by analyzing video image data from a video acquisition unit.Smoke detection can be achieved alternatively or additionally by analyzing hyperspectral and / or multispectral information.

[0048] The fault detection unit of the medical imaging device can be configured to prevent the acquisition of spatially and spectrally resolved image data by means of the spatially and spectrally resolved image acquisition unit if a fault is detected. For example, the fault detection unit can be configured to at least temporarily deactivate the image acquisition unit and / or at least temporarily prevent image acquisition. Alternatively or additionally, a user output can be provided to alert the user to a fault condition. The user can then, for example, rectify the fault condition and continue with the acquisition of spatially and spectrally resolved image data and / or continue with the acquisition of spatially and spectrally resolved image data despite the fault condition.

[0049] A user can be specifically alerted to defective areas of an image, particularly if the output unit of the medical imaging device is configured to generate a spatially resolved parameter representation of the image area according to the analysis parameter and to mark these parts of the image area as defective by the defect detection unit. For example, the part of the image area subject to a defect can be identified with black or differently colored image data points. Furthermore, a border, strikethrough, or other highlighting of the affected image area can be displayed.

[0050] The output unit of the medical imaging device can further be configured to provide the user with information on how to rectify the malfunction if a malfunction is detected. For example, the malfunction detection unit can be configured to detect the type of malfunction. This can be based on one and / or a combination of the described methods. After detecting a malfunction, the user can, for example, be prompted to remove a medical instrument from the image area, perform a white balance, and / or clean a dirty lens. In principle, any type of malfunction can be detected, and the user can be prompted and / or instructed to rectify the malfunction.

[0051] The present invention is described below by way of example with reference to the attached figures. The drawings, the description and the claims contain numerous features in combination. A person skilled in the art will expediently also consider the features individually and use them sensibly in combination within the scope of the claims. If there is more than one example of a particular object, only one of them will be provided with a reference symbol in the figures and the description. The description of this example can be applied accordingly to the other examples of the object. If objects are named in particular using numerical words such as first, second, third object etc., these serve to name and / or assign objects. Accordingly, for example, a first object and a third object may be included, but not a second object.However, using number words, it might also be possible to derive a number and / or order of objects.

[0052] They show:

[0053] Fig. 1 is a schematic representation of a medical system with a medical imaging device;

[0054] Fig. 2 is a schematic structural diagram of the imaging device;

[0055] Fig. 3 is a schematic representation of an image area;

[0056] Fig. 4 shows a further schematic representation of the image area to illustrate a possible disturbance;

[0057] Fig. 5 is a further schematic representation of the image area to illustrate another possible disturbance;

[0058] Fig. 6 is a further schematic representation of the image area to illustrate another possible disturbance;

[0059] Fig. 7 is a further schematic representation of the image area to illustrate another possible disturbance;

[0060] Fig. 8 is a further schematic representation of the image area to illustrate another possible disturbance;

[0061] Fig. 9 shows a further schematic representation of the image area to illustrate another possible disturbance; Fig. 10 shows a schematic representation of absorption curves of tissue with different degrees of coagulation;

[0062] Fig. 1 1 is a schematic perspective view of an alternative imaging device;

[0063] Fig. 12 is a schematic flow diagram of a method for operating a medical imaging device; and

[0064] Fig. 13 is a schematic flow diagram of a medical imaging method.

[0065] Fig. 1 shows a schematic representation of a medical system 60 including a medical imaging device 10. In some embodiments, the medical system 60 further includes a medical instrument 34. A schematic structural diagram of the imaging device 10 and the medical device 62 is shown in Fig. 2. Reference is made in part to both figures below.

[0066] In the exemplary case, the imaging device 10 is an endoscopic imaging device, specifically an endoscope device. Alternatively, the imaging device 10 could be an exoscopic, microscopic, or macroscopic imaging device. The medical imaging device is intended for examining a cavity.

[0067] In the illustrated case, the medical instrument 34 is a bipolar electrosurgical instrument. The medical instrument 34 is configured to specifically introduce energy into tissue in order to coagulate it, for example, for vascular occlusion. This embodiment is intended purely as an example. Other types of energy introduction may be provided, as well as other types of medical devices in general, such as surgical, diagnostic, imaging, interventional, anesthetic, or other medical instruments and / or devices.

[0068] The imaging device 10 comprises, for example, a medical imaging device

[0069] 64. In the illustrated case, this is an endoscope. Furthermore, the medical imaging device 10 can include an illumination device 66. In the illustrated case, the illumination device 66 is connected to the imaging device 64 via a light guide. Illumination light can thus be guided to the imaging device 64 and directed from there onto an object to be imaged, in particular a site.

[0070] The imaging device 10 has, for example, a control unit 68. The control unit 68 is connected to the imaging device 64 via a cable and / or an optical line and / or a light guide.

[0071] The imaging device 10 and in particular the imaging device 64 has, by way of example, one or more windows 70 through which illumination light can be coupled out and / or object light can be coupled in.

[0072] The imaging device 64 has a distal section 72 which includes a distal end 36. Generally speaking, this is a distal end 36 of the imaging apparatus 10. The distal section 72 is designed to be inserted into a cavity in an operating state. The distal section 72 faces a patient in the operating state. The distal section 72 faces away from a user in the operating state. Furthermore, the medical imaging device 64 has a proximal section 74. The proximal section 74 is arranged outside a cavity in the operating state. The proximal section 74 faces away from the patient in the operating state. The proximal section 74 faces the user in the operating state.

[0073] The imaging device 64 has a handle 76. The handle 76 is configured, for example, for handling by the user. Alternatively or additionally, the handle 76 can be configured for attachment and / or connection to a medical robot. The imaging device 64 can also be formed integrally with a robot in some embodiments. A position and / or orientation of the imaging device 64 relative to the patient can be changed, for example, by handling by the user and / or by suitable movement of the robot.

[0074] The medical system 60 has a display device 78. The display device 78 can be part of the imaging device 10. The display device 78 can be a separate display such as a screen or the like. In other embodiments, the display device 78 can also be integrated into the imaging device 64 and / or the control device 68. The imaging device 10 has a device interface 80 via which the medical instrument 34 can be connected to the imaging device 10. In the illustrated case, the device interface 80 is part of the control device 68. The device interface 80 is wired in the illustrated case. The device interface 80 is detachable. Furthermore, the device interface 80 can be configured for connection to various medical devices. The device interface 80 can include a socket and / or electrical connections.In some embodiments, the device interface 80 may also be partially or completely wireless, ie the medical instrument 34 can then be connected wirelessly, for example via a radio connection, to the imaging device 10, and the imaging device 10 and the medical instrument 34 may have correspondingly suitable antennas.

[0075] In other embodiments, the medical instrument 34 may be entirely independent of the imaging device 10. This may then also be free of a device interface.

[0076] The imaging device 10 has a user interface 82 through which the user can provide inputs. The user interface 82 may include multiple control elements that may be attached to different components of the imaging device 10 and / or the medical system 60.

[0077] The imaging device 10 has a spatially and spectrally resolving image acquisition unit 12, which has at least one optical system 14. The image acquisition unit 12 further has an image acquisition sensor system 16 coupled to the optical system 14. The optical system 14 and the image acquisition sensor system 16 are configured to generate image data of an image area 18. The image area 18 is shown in Fig. 1 as an example on a display of the display device 78. The image data comprises both optical and spectral information. In this case, the image data correspond to two-dimensional spatial data that define spatial pixels, as well as spectral data that is assigned to the individual pixels. A spectrum is thus obtainable from the image data for each pixel. Furthermore, a two-dimensional image is obtainable from the image data for each spectral band. The image data correspond to a multispectral or hyperspectral data cube.The optics 14 comprises optical elements (not shown) that collect object light and guide it to the image capture sensor 16. The image capture sensor 16 comprises a CMOS or CCD sensor (not shown). The optics 14 and the image capture sensor 16 are arranged together in a pushbroom arrangement. In other embodiments, a whiskbroom arrangement, a staring arrangement and / or a snapshot arrangement is used. In the present case, the image capture unit 12 is configured for hyperspectral image capture, and the imaging device 10 is accordingly a hyperspectral imaging device. With regard to different methods of hyperspectral imaging and the components required for this, reference is made to the specialist article "Review of spectral imaging technology in biomedical engineering: achievements and challenges" by Quingli Li et al.Published in Journal of Biomedical Optics 18(10), 100901, October 2013, and reference is made to the article "Medical hyperspectral imaging: a review" by Guolan Lu and Baowei Fei, published in Journal of Biomedical Optics 19(1), 010901, January 2014. In other embodiments, the imaging device 10 can also be multispectral. Multiple spectral ranges can be viewed, for example, through filters that can be selectively inserted into an object light beam path and / or through sequential illumination with different wavelengths.

[0078] The image capture unit 12 can be at least partially contained in the imaging device 64. Parts of the optics 14 and / or the image capture sensor system 16 can be contained in the control unit 68. For example, object light can be guided to the image capture sensor system 16 via a fiber optic cable, and the latter can be arranged in the control unit 68. In other embodiments, the entire image capture sensor system 16 is contained in the imaging device 64, and only data is transmitted to the control unit 68.

[0079] The imaging device 10 further comprises an evaluation unit 20. This is configured to perform an analysis of the image data. The analysis is based on both spatial and spectral information. The analysis includes at least one analysis parameter. This will be discussed in more detail below.

[0080] In the exemplary case, the imaging device 10 comprises an output unit 24. The output unit 24 can comprise the display device 78 and / or other output devices. In the illustrated case, the output unit 24 comprises, in addition to the display device 78, a loudspeaker 86 and an additional display 88, which are formed, for example, on the control unit 68. In other embodiments, the output unit 24 has an interface for connecting one or more output devices. The output generation unit 84 can also comprise the output unit 24 or its components.

[0081] The imaging device 10 comprises a control unit 90. This comprises a processor, a main memory, a storage, and correspondingly configured circuitry. Program code is stored in the memory, which, when executed by the processor, causes the methods described herein to be carried out or implements functionalities of the described units.

[0082] Furthermore, in the exemplary embodiment, the imaging device 10 includes a database 92. The evaluation unit 20 can access the database to create an evaluation of the image data. For example, the evaluation unit 20 can compare an evaluation parameter calculated by the evaluation unit 20 with parameters stored in the database 92. This will be discussed in more detail below.

[0083] The imaging device 10 further comprises a disturbance detection unit 22. The disturbance detection unit 22 is configured to detect a disturbance during image acquisition and to determine a disturbance state of the image acquisition. The disturbance detection unit 22 can detect a disturbance state based on hyperspectral and / or multispectral information. Alternatively or additionally, the disturbance detection unit 22 can determine a disturbance state based on video image data generated by a video acquisition unit 28 and / or based on fluorescence imaging data generated by a fluorescence imaging unit 38. In some embodiments, the disturbance detection unit 22 can additionally or alternatively be configured to determine a disturbance state based on a sensor signal generated by a sensor unit 42. The disturbance detection will be discussed in more detail below.

[0084] The imaging device 10 further comprises a video capture unit 28, which comprises a camera 30 and is configured to generate video image data of the image area 18. The video capture unit 28 can also be configured to generate images in other display types, for example, false color display, in addition to or instead of white light images. Video image data that do not contain spectral information is captured by the video capture unit 28. Image capture with spectral information often takes more time than video data capture. Video image data can be generated and / or output in real time. Image data that includes spectral information can be captured and / or output essentially in real time or at a slower refresh rate / as individual images. It can be advantageous, for example,For example, for positioning a medical instrument 34 or for positioning the image acquisition unit 12, the imaging device 10 may be operated without generating image data with spectral information. In such a case, the video acquisition unit 28, in particular, may generate video image data in real time. In some embodiments, the imaging device 10 may be multimodal. Image data may be generated in different modes. The different modes may possibly be used simultaneously, alternately, and / or sequentially.

[0085] In this case, the image acquisition unit 12 is configured to generate white-light images and hyperspectral images of the image area 18. White-light images do not contain any spectral information. Both modes, hyperspectral imaging and white-light imaging, can be performed simultaneously and / or alternately and / or sequentially. Images from both modes can be combined after imaging. Alternatively or additionally, the hyperspectral image can also be combined with video image data that differs from white-light images.

[0086] The imaging device 10 also includes a fluorescence imaging unit 38. The fluorescence imaging unit 38 is configured to capture the image area 18 using fluorescence imaging and to generate fluorescence imaging data. For example, a specific type of tissue can be made recognizable by fluorescence imaging using a contrast agent 40. The generation of fluorescence imaging data using the fluorescence imaging unit 38 represents a mode that can possibly be used simultaneously, alternately, and / or sequentially with other modes. The disturbance detection unit 22 is configured to detect a disturbance based on an assessment of the fluorescence imaging data. In particular, the disturbance detection unit 22 can detect the presence of contrast agent 40 in at least part of the image area 18.

[0087] According to the exemplary embodiment, the imaging device 10 also comprises a sensor unit 42, which comprises at least one sensor 44. The sensor 44 is configured to measure a measured variable that describes a state of the imaging device 10 and / or an environment of the imaging device 10. The fault detection unit 22 is configured to detect the presence of a fault based on the sensor signal.

[0088] The sensor 44 can be an acceleration sensor, for example. The acceleration sensor measures a movement of the imaging device relative to the image area 18. The disturbance detection unit 22 is configured to determine a range of motion based on the sensor signal and to detect the presence of a disturbance if the determined range of motion exceeds a threshold value.

[0089] In another exemplary embodiment, sensor 44 may be a distance sensor. This sensor is configured to determine a distance between imaging device 10 and an object (not shown in detail) located in image area 18. Depending on the distance, optimized imaging can be achieved either automatically or after a user output via output unit 24.

[0090] In the present application example, spatially and spectrally resolved image data is acquired using the pushbroom method. Figures 3 to 10 explain image acquisition using the imaging device 10 and the interference detection unit 22. Interference detection is not limited to the pushbroom method. For example, if image data is acquired point by point using the whiskbroom method, interference detection can also be applied. Furthermore, not all possible interferences are listed. The interferences shown in the following figures are selected merely as examples to explain the functionality of the interference detection unit 22.

[0091] Fig. 3 shows a schematic representation of an image region 18. The image region 18 is observed, for example, during a microinvasive procedure. For this purpose, the distal section 72 of the imaging device 64 (cf. Fig. 1 ) is introduced into a cavity. Various native structures 94, 96 and a vessel 98, for example a blood vessel, are located in the image region. This vessel is to be sealed by the medical device 62. First, spatially and spectrally resolved image data of the image region 18 are generated. For this purpose, the image region 18 is illuminated and object light coming from the image region 18 is then detected. In the present case, the detected spectral information relates to light absorption. Accordingly, an absorption spectrum can be obtained for each pixel. Image data of the image region 18 are presently also detected by the video detection unit 28.Image data of image area 18, generated by video acquisition unit 28, is available at a higher repetition rate than the spatially and spectrally resolved image data. The user can observe image area 18 in real time using the video image data during the generation of spatially and spectrally resolved image data via output unit 24. The fault detection unit 22 has access to spatially and spectrally resolved image data as well as video image data.

[0092] Additionally, the distance of the imaging device 12 to at least one object in the image area 18 and / or the relative movement between the imaging device 12 and at least one object in the image area 18 is detected. Said object can be, for example, the medical instrument 34 and / or the native structure 94, 96 and / or the vessel 98.

[0093] If, as in the present example, the vessel 98 is to be sealed using the medical device 62, high-quality hyperspectral imaging of the vessel 98 is of utmost importance for assessing the quality and success of the procedure. If a disturbance caused by a large relative movement between the imaging device 12 and the vessel 98 is detected by the sensor unit 42, the user is notified of the disturbance. They can then be prompted to perform another hyperspectral imaging of the image area 18.

[0094] The evaluation unit 20 is configured to analyze the spatially and spectrally resolved image data. The evaluation includes, for example, image segmentation and a comparison of spectral information with information stored in the database 92 and relating to properties of various structures, tissue entities, etc. Regions identified during image segmentation can be analyzed for their spectral properties. The database 92 stores, for example, which tissue entities are typically characterized by which absorption spectra and / or exhibit certain absorption values ​​for certain spectral bands, in particular relative to other spectral bands. The evaluation unit 20 creates at least one analysis parameter. The analysis parameter can, for example, specify which tissue entity the corresponding image segment relates to.

[0095] During the analysis, the medical instrument 34 can also be detected. The presence of an inorganic material 26, such as the medical instrument 34 in this example, can cause a disruption in image acquisition. The medical instrument 34 can be detected and identified using hyperspectral information. For this purpose, known spectral properties of the inorganic material in question can be taken into account.

[0096] A low degree of error susceptibility can be achieved if video image data generated with the video acquisition unit 28 in addition to hyperspectral image acquisition is examined for the presence of inorganic material 26 using a KI algorithm. This allows a high degree of accuracy in the detection of inorganic material 26 to be achieved.

[0097] Furthermore, on the basis of the analysis, an overlay 99 is generated, which can be displayed to the user via the output unit 24 (cf. Fig. 1). As illustrated in Fig. 4, the various detected tissue entities are highlighted differently, for example by overlaying a colored overlay, a texture, a label, or a combination. The user can therefore easily understand which sub-regions, instruments, organs, vessels, nerves, fascia, etc. are located in the image area 18. The creation of the analysis and the generation of the overlay can be carried out repeatedly. As mentioned, video image data can form the basis for this. The overlaid highlights of different tissue structures, based on spectral information, can be updated essentially in real time.

[0098] The medical instrument 34 is detected by the interference detection unit 22 based on video image data and spatial and spectral information. In the present example, the medical instrument 34 can be assigned to data in the database 92 based on the spectral information and is detected based on a corresponding comparison. The accuracy of the assignment is increased by the video image data. If unassignable inorganic material is present in the image area 18, it is detected by the interference detection unit 22 and displayed, for example, without an overlay 99 or marked in color, in particular black, and in any case without spectral parameter representation. A faulty partial area 48 containing inorganic material is detected, and the faulty partial area 48 is identified via the output unit. This occurs, for example, by encircling the faulty partial area 48 with a color highlight.

[0099] As explained, the fault detection of inorganic material 26 according to this application example is carried out using two types of image data. Firstly, inorganic material 26 is detected using spectral information. Fault detection therefore takes place sequentially after image acquisition. Secondly, inorganic material 26 is detected using a KI algorithm from video image data. Fault detection takes place before and / or during, in particular simultaneously with, the acquisition of the spatially and spectrally resolved image data.

[0100] Fig. 5 shows another possible fault. The same image area 18 is visible as in the previous figures, i.e., structures 94, 96 and the vessel 98 are visible. In addition, a fault caused by contamination and / or fogging 32 of the optics 14 (cf. Fig. 2) in the image area 18 is visible. The contamination and / or fogging 32 shown is detected from video image data using a Kl algorithm. According to this example, the user is prompted to clean the optics 14 in order to achieve more fault-free imaging. According to other embodiments, the optics 14 can be cleaned automatically after a fault caused by contamination and / or fogging 32 is detected.

[0101] Fig. 6 again shows the same image area 18 with the known structures 94, 96 and the vessel 98 to illustrate another possible disturbance. Only a section of the image area 18 is clearly visible because, according to this example, the distal end 36 is still partially located within a trocar 37 used to introduce the imaging device 10. Consequently, part of the interior of a trocar 37 can be seen in Fig. 6 in the image area 18. The trocar 37 is recognized in the image area 18 using video image data. The user can be prompted to eliminate the disturbance, for example by pushing the distal end 36 further through the trocar 37 for imaging. In Fig. 7, the image area 18 with the structures 94, 96 and the vessel 98 as well as the medical instrument 34 is shown a short time, for example a few seconds, after the coagulation of the vessel 98 has taken place. By coagulation of the vessel

[0102] Smoke 46 has developed between the vessel 98 and the optics 14 and is visible in the image area 18. The smoke 46 means that spectral information of the vessel 98 obscured by the smoke 46 is not recorded and / or is recorded with interference. The smoke 46 can be detected using spectral information. Alternatively or additionally, the smoke 46 can be detected using video image data. In the latter case, a suitable AI algorithm is used for smoke detection, such as a binary classification with ResNet 50. A combination of both detection methods increases the reliability and accuracy of the detection. Image data points assigned to smoke 46 are indicated to the user. In addition, the noisy sub-area 48 of the image area 18 is optically highlighted on the output unit 24, for example, by a colored circle.The user will also be prompted to initiate the acquisition of the spatial and spectral information of image area 18 again, if necessary, observing a suitable waiting time. Alternatively or additionally, the user may be prompted to activate a smoke extraction system (not shown).

[0103] Fig. 8 and Fig. 9 show the image acquisition of the image area 18 with the additional use of the fluorescence imaging unit 38. The vessel 98 is colored using a fluorescent contrast agent 40. The fluorescence imaging unit 38 detects the vessel 98 colored with the contrast agent 40, highlighted by the hatching in Fig. 9, and displays it to the user in combination with a white light image. In addition, spatial and spectral information of the image area 18 is acquired, evaluated, and displayed using overlays.

[0104] 99 is displayed to the user.

[0105] The noisy subregion 48, in which contrast agent 40 is present and which is detected by the fluorescence imaging unit 38 and / or interference detection unit 22, is excluded from the evaluation of the spatial and spectral information. In this region, tissue detection could yield erroneous results or fail due to the contrast agent.

[0106] In addition to fluorescent contrast agent 40, inorganic material 26 can be seen in image area 18. This is detected by the disturbance detection unit 22 based on spectral information and video image data, as already described. This application example illustrates the multimodal nature of the invention. Spatial and spectral information is acquired simultaneously with white light images, video image data, and fluorescence image data. Therefore, the disturbance detection unit 22 can simultaneously detect various types of disturbances—in the case of Fig. 9, fluorescent contrast agent 40 and inorganic material 26—and the imaging device 10 can display tissue entities by means of overlays 99 in sub-areas of image area 18 that are not subject to any detected disturbance.

[0107] It is understood that fault detection can be performed at any time, especially at the same time as image acquisition.

[0108] Fig. 10 shows schematic absorption curves in the visible and near-infrared range of tissue coagulated with different parameters. Such curves can be obtained from spatially and spectrally resolved image data during an analysis, for example, by averaging over the image area 18 or a suitably selected subarea. The analysis can include absorption curves instead of a parameter image as described above, or in addition to it. In this case, the analysis parameter is, for example, the spectrally resolved intensity.

[0109] The inventors have found that the absorption curves for less and more strongly coagulated tissue behave differently in different spectral ranges. In a short-wave range, indicated by line a1 in Fig. 10, absorption decreases with increasing coagulation. In a long-wave range, indicated by line a2 in Fig. 10, absorption increases with increasing coagulation. The spectra in Fig. 10 are normalized, in this case by division by the respective absorption value at line n. As described below, this can be used to analyze and assess whether a coagulation treatment was completed with sufficient quality.

[0110] The analysis is based on a before-and-after comparison, using image data from before the diagnostic and / or therapeutic procedure as a reference. The before-and-after comparison involves calculating the difference, which indicates the extent to which absorption has changed in the spectral ranges under consideration. To compensate for effects attributable to different amounts of tissue in the image area or other deviations, the spectra to be compared are normalized, as illustrated in Fig. 10. In the present case, for example, the differences in lines a1 and a2 are determined as evaluation parameters.

[0111] Database 32 stores the ratios, differences, and / or other correlations between these evaluation parameters that can provide information about coagulation quality. The assessment unit 24 uses this information as assessment information and, based on it, evaluates the evaluation parameters obtained from the analysis. In the present case, this assessment determines a binary attribute that can assume the values ​​"coagulation complete" and "coagulation incomplete."

[0112] If an interference, such as one of the interferences described above, is present during the acquisition of the spatially and spectrally resolved image data used to generate such absorption curves, the absorption curves may not be usable. One or more interferences can be detected by the interference detection unit 22 as described above. It is understood that the interferences are not detected, or at least not exclusively, based on the analysis parameter used, but rather independently of it.

[0113] Subsequently, a user output based on the analysis parameter can be generated based on a detected interference condition. For example, an absorption spectrum is displayed and / or marked as usable only if no interference condition is present.

[0114] Fig. 11 shows an alternative imaging device 10'. For clarity, the reference numerals of this embodiment are provided with inverted commas. The alternative imaging device 10' is part of a medical system 60' that is configured to perform exoscopic imaging. For example, this medical system 60' can be used to observe a surgical site during a procedure. The medical system 60' basically has the same functionality as the previous embodiment. The medical system 60' comprises an image acquisition unit 12', an evaluation unit 20', a disturbance detection unit 22', and an output unit 24', via which an image area 18' as well as disturbances and overlays, etc., are displayed. Reference is made to the above explanations for their functionality.

[0115] Fig. 12 shows a schematic flow diagram of a method for operating an imaging device. The sequence of the methods is also clear from the above explanations.

[0116] By way of example, the imaging device 10, 10' described above is operated.

[0117] The method comprises a step S1 1 of acquiring spatially and spectrally resolved image data of an image area 18 by means of the medical imaging device 10, 10'.

[0118] Furthermore, the method comprises a step S12 of creating an analysis of the spatially and spectrally resolved image data, which analysis is based on spatial and spectral information and is based on at least one analysis parameter calculated from the spatially and spectrally resolved image data.

[0119] Furthermore, the method comprises a step S13 of performing a fault detection, in which, independently of the analysis of the image data and the calculated analysis parameter, the presence of a fault in the image acquisition is detected and in which a fault state of the image acquisition is determined.

[0120] Furthermore, the method comprises a step S14 of generating a user output according to the fault condition based on the analysis parameter.

[0121] Fig. 13 shows a schematic flow diagram of a method for medical imaging. The sequence of the methods is also evident from the above explanations.

[0122] By way of example, the method is carried out using the imaging device 10, 10' described above.

[0123] The method comprises a step S21 of performing an image acquisition of an image area 18, generating spatially and spectrally resolved image data that includes both spatial and spectral information. Furthermore, the method comprises a step S22 of creating an analysis of the spatially and spectrally resolved image data, which analysis is based on spatial and spectral information and is based on at least one analysis parameter calculated from the spatially and spectrally resolved image data.

[0124] Furthermore, the method comprises a step S23 of performing a fault detection, in which, independently of the analysis of the image data and the calculated analysis parameter, the presence of a fault in the image acquisition is detected and in which a fault state of the image acquisition is determined.

[0125] Furthermore, the method comprises a step S24 of generating a user output in accordance with the fault condition based on the analysis parameter.

[0126] As is also clear from the above explanations, the sequence of steps S11 to S14 or S21 to S24 is not necessarily consecutive. Figures 12 and 13 are to be understood purely as examples in this regard.

[0127] List of reference symbols

[0128] 10 Imaging device

[0129] 12 Image acquisition unit

[0130] 14 Optics

[0131] 16 image capture sensors

[0132] 18 Image area

[0133] 20 evaluation unit

[0134] 22 Fault detection unit

[0135] 24 output unit

[0136] 26 inorganic material

[0137] 28 Video capture unit

[0138] 30 Camera

[0139] 32 Soiling and / or condensation

[0140] 34 medical instrument

[0141] 36 distal end

[0142] 37 Trocar

[0143] 38 Fluorescence imaging unit

[0144] 40 contrast agents

[0145] 42 Sensor unit

[0146] 44 Sensor 46 Smoke

[0147] 48 faulty part

[0148] 60 medical system

[0149] 64 imaging device

[0150] 66 Lighting device

[0151] 68 Control unit

[0152] 70 windows

[0153] 72 distal section

[0154] 74 proximal section

[0155] 76 Handle

[0156] 78 Display device

[0157] 80 Device interface

[0158] 82 User interface

[0159] 84 Output generation unit

[0160] 86 speakers

[0161] 88 Display

[0162] 90 Control unit

[0163] 92 Database

[0164] 94 Structure

[0165] 96 Structure

[0166] 98 vessels

[0167] 99 Fade-in

Claims

Claims 1. Medical imaging device (10), in particular endoscopic and / or exoscopic and / or microscopic imaging device, comprising: a spatially and spectrally resolved image acquisition unit (12) comprising at least one optic (14) and at least one image acquisition sensor (16) coupled to the optic, which are configured to perform image acquisition of an image area (18) in which spatially and spectrally resolved image data are generated, comprising both spatial and spectral information; an evaluation unit (20) which is configured to perform an analysis of the spatially and spectrally resolved image data, which is based on spatial and spectral information and is based on at least one analysis parameter calculated from the spatially and spectrally resolved image data;a fault detection unit (22) configured to detect the presence of a fault in image acquisition and to determine a fault state of image acquisition, independently of the analysis of the image data and the calculated analysis parameter; and an output unit (24) configured to generate a user output based on the analysis parameter according to the fault state.

2. Medical imaging device (10) according to claim 1, wherein the interference detection unit (22) is configured to detect the presence of an interference by means of an assessment of the spatially and spectrally resolved image data.

3. Medical imaging device (10) according to claim 2, wherein the interference detection unit (22) is configured to detect an image exposure state and to detect the presence of an interference based on the spatially and spectrally resolved image data if the exposure state represents at least partial underexposure and / or overexposure.

4. Medical imaging device (10) according to claim 2 or 3, wherein the interference detection unit (22) is configured to detect the presence of an interference by means of a detection of inorganic material (26) in the image area (18).

5. Medical imaging device (10) according to one of the preceding claims, further comprising a video acquisition unit (28) comprising a camera (30) and configured to generate video image data of the image area (18), wherein the interference detection unit (22) is configured to detect the presence of an interference by means of an image analysis of the video image data.

6. Medical imaging device (10) according to claim 5, wherein the interference detection unit (22) is configured to determine a range of motion of the camera relative to the image area (18) based on the image analysis of the video image data and to detect the presence of an interference if the determined range of motion exceeds a threshold value.

7. Medical imaging device (10) according to claim 5 or 6, wherein the interference detection unit (22) is configured to detect the presence of soiling and / or fogging (32) on at least a part of the optics of the spatially and spectrally resolving image acquisition unit (12) by means of image analysis of the video image data and to detect the presence of an interference according to the detection of soiling and / or fogging (32).

8. Medical imaging device (10) according to one of the preceding claims, wherein the interference detection unit (22) is configured to assess, by comparing the spatially and spectrally resolved image data and the video image data, whether a white balance of the spatially and spectrally resolved image acquisition unit and a white balance of the video acquisition unit are at least within a predetermined tolerance and to detect the presence of an interference if the predetermined tolerance is exceeded.

9. Medical imaging device (10) according to any one of claims 5 to 8, wherein the interference detection unit (22) is configured to use the Image analysis of the video image data to detect the unexpected presence of a medical instrument (34) in the image area (18) and to detect the presence of a disturbance according to the detection of the unexpected presence of a medical instrument (34).

10. Medical imaging device (10) according to claim 9, wherein the imaging device (10) comprises a distal end (36) which includes at least parts of the optics (14) of the image acquisition unit (12), wherein the interference detection unit (22) is configured to detect that the image area (18) includes at least part of the interior of a trocar (37), and wherein the output unit (24) is configured to generate a user output containing the information that the distal end (36) is located at least part of the trocar (37).

11. Medical imaging device (10) according to one of the preceding claims, further comprising a fluorescence imaging unit (38) which is configured to capture the image area (18) by means of fluorescence imaging and to generate fluorescence imaging data, wherein the interference detection unit (22) is configured to detect the presence of an interference by means of an evaluation of the fluorescence imaging data.

12. Medical imaging device (10) according to claim 11, wherein the interference detection unit (22) is configured to detect the presence of contrast agent (40) in at least a part of the image area (18) based on the fluorescence imaging data, and wherein the output unit is configured to generate the user output based on the analysis parameter in such a way that parts of the image area (18) in which the interference detection unit (22) detects the presence of contrast agent (40) are omitted.

13. Medical imaging device (10) according to any one of the preceding claims, further comprising a sensor unit (42) with at least one sensor (44) configured to measure at least one measurand describing a state of the imaging device (10) and / or an environment of the imaging device (10), and configured to generate a sensor signal representing the measurand, wherein the disturbance detection unit (22) is configured to detect the presence of a disturbance based on the sensor signal.

14. Medical imaging device (10) according to claim 13, wherein the sensor (44) is an accelerometer, wherein the sensor signal represents a movement of the imaging device (10), and wherein the interference detection unit (22) is configured to determine a range of motion based on the sensor signal and to detect the presence of an interference when the determined range of motion exceeds a threshold value.

15. Medical imaging device (10) according to claim 13 or 14, wherein the sensor (44) is a distance sensor configured to measure a to determine the distance between the imaging device (10) and an object in the image area (18).

16. Medical imaging device (10) according to one of the preceding claims, wherein the interference detection unit (22) is configured to detect the presence of smoke (46) in the image area (18) by means of an image analysis and to detect the presence of an interference according to the detection of smoke (46) in the image area (18).

17. Medical imaging device (10) according to one of the preceding claims, wherein the interference detection unit (22) is configured to prevent the acquisition of spatially and spectrally resolved image data by means of the spatially and spectrally resolved image acquisition unit (12) in the event of the detection of an interference.

18. Medical imaging device (10) according to any one of the preceding claims, wherein the output unit (24) is configured to generate a spatially resolved parameter representation of the image area (18) according to the analysis parameter and to identify by the fault detection unit (22) as faulty parts (48) of the image area (18).

19. Medical imaging device (10) according to one of the preceding claims, wherein the output unit (24) is configured to provide the user with information on how to rectify a malfunction in the event of a malfunction being detected.

20. Method for operating a medical imaging device (10), in particular according to one of the preceding claims, wherein the medical imaging device (10) comprises a spatially and spectrally resolving image acquisition unit (12) comprising at least one optic (14) and at least one image acquisition sensor (16) coupled to the optic, which are configured to perform image acquisition of an image area (18) in which spatially and spectrally resolved image data are generated, comprising both spatial and spectral information, comprising the steps: - Acquisition of spatially and spectrally resolved image data of an image area (18) using the medical imaging device (10); - Creating an analysis of the spatially and spectrally resolved image data, based on spatial and spectral information and relying on at least one analysis parameter calculated from the spatially and spectrally resolved image data; - Performing a fault detection process in which, independent of the analysis of the image data and the calculated analysis parameter, the presence of a fault in image acquisition is detected and a fault state of the image acquisition is determined; and - Generating a user output according to the fault state, based on the analysis parameter.

21. Method for medical imaging, in particular carried out with a medical imaging device (10) according to any one of claims 1 to 19, comprising the steps: - Performing an image acquisition of an image area (18) in which spatially and spectrally resolved image data are generated that include both spatial and spectral information; - Creating an analysis of the spatially and spectrally resolved image data, based on spatial and spectral information and relying on at least one analysis parameter calculated from the spatially and spectrally resolved image data; - Performing a fault detection process in which, independent of the analysis of the image data and the calculated analysis parameter, the presence of a fault in image acquisition is detected and in which a fault condition of the Image capture is determined; and - Generating a user output according to the fault state, based on the analysis parameter.

Images