Medical imaging device, medical system, method for operating a medical imaging device, and method of medical imaging

EP4463873A2Pending Publication Date: 2024-11-20KARL STORZ SE & CO KG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
EP2023706347
Authority / Receiving Office
EP · EP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-02-22
Filing Date
2023-02-20
Publication Date
2024-11-20

AI Technical Summary

Technical Problem

Current medical imaging devices lack the capability to provide real-time, high-resolution spatial and spectral data effectively, which hampers the safety, quality, and efficiency of diagnostic and therapeutic procedures, especially in minimally invasive surgeries.

Method used

A medical imaging device equipped with a spatially and spectrally resolving image capture unit, coupled with an image analysis and assessment unit, generates and analyzes image data to provide automated evaluations and recommendations for diagnostic and therapeutic actions, enhancing the precision and efficiency of medical procedures.

Benefits of technology

The device improves the safety and quality of medical procedures by providing real-time, high-resolution spatial and spectral data, enabling automated assessments and recommendations that support both experienced and inexperienced users, thereby optimizing diagnostic and therapeutic outcomes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 1.1
    Figure 1.1
Patent Text Reader

Abstract

The invention relates to a medical imaging device (10), comprising: a spatially and spectrally resolving image capturing unit (12) which comprises at least one optical unit (14) and at least one image capturing sensor system (16) coupled to the optical unit (14), these being configured to create image data comprising spatial and spectral information of an object region (18); an image analysis unit (20) which is configured to compile an analysis of the image data on the basis of spatial and spectral information, wherein the analysis comprises at least one evaluation comprising at least one evaluation parameter relating to an object partial region (22) of the object region (18); an assessment unit (24) which is configured, according to the stipulation of the analysis and on the basis of information relating to a diagnostic and / or therapeutic action which is to be carried out and / or has been carried out, to establish an assessment, on the basis of the evaluation parameter, of an attribute of the object partial region (22) relevant to the diagnostic and / or therapeutic action; and an output generation unit (26) which is configured to generate an output based on the assessment. The invention also relates to a medical system (56) having a medical imaging device (10) and to associated methods.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Medical imaging device, medical system, method for operating a medical imaging device and method for medical imaging

[0002] The invention relates to a medical imaging device, a medical system, a method for operating a medical imaging device and a method for medical imaging.

[0003] Medical imaging devices, such as endoscopic devices, that operate multispectrally or hyperspectrally are known from the prior art. An object to be imaged, such as a site for a minimally invasive procedure, is acquired with spectral resolution, so that individual spectral information is obtained for each of the different acquired pixels. For example, DE 10 2020 105 458 A1 discloses an imaging device that has an adjustable spectral filter that allows images to be acquired in different spectral ranges to generate multispectral image data.

[0004] In principle, multispectral or hyperspectral image data can be acquired using different methods. In the so-called whiskbroom approach, the object to be imaged is scanned point by point, obtaining a spectrum for each point. In the so-called pushbroom method, a complete image line is acquired at a time. By using a two-dimensional detector, a spectrum can then be acquired for each point in the image line. It is also possible to acquire multiple images sequentially through different spectral filters, in a so-called staring approach. Each individual image contains complete spatial information for a single spectral range.There are also approaches that use suitable optical elements such as optical slicers, lenses, and prisms to decompose a two-dimensional multicolor image into several individual spectral images that are simultaneously acquired on different detectors or detector areas. This is sometimes referred to as the snapshot approach.

[0005] Imaging techniques are of central importance in the medical field, particularly in connection with minimally invasive surgical procedures, but also generally for diagnostics and for assessing the success or quality of a procedure. For example, a site can be observed during a minimally invasive procedure when tissue is to be welded or has been welded, for example, using an electrosurgical device such as a high-frequency coagulation instrument. Other examples of diagnostic and / or therapeutic actions before, during, or after which observation using imaging techniques is appropriate include the observation and assessment of ligatures, clips, and stapler insertions, tissue sampling, or various surgical procedures in general.

[0006] The invention is particularly based on the object of providing an imaging device and an imaging method by means of which the safety, quality and / or efficiency of diagnostic and / or therapeutic actions can be increased.

[0007] This object is achieved according to the invention by the features of the independent claims. Further developments of the invention can be found in the dependent claims.

[0008] An imaging device according to the invention can comprise a spatially and spectrally resolving image acquisition unit, which comprises at least one optical system and at least one image acquisition sensor coupled to the optical system, which are configured to generate image data of an object region comprising spatial and spectral information. Furthermore, the imaging device can comprise an image analysis unit configured to create an analysis of the image data based on spatial and spectral information, wherein the analysis comprises at least one evaluation comprising at least one evaluation parameter relating to an object sub-region of the object region.Furthermore, the imaging device may comprise an assessment unit configured to generate, in accordance with the analysis and based on information regarding a diagnostic and / or therapeutic action to be performed and / or performed, an assessment of an attribute of the object subregion relevant to the diagnostic and / or therapeutic action, which is based on the evaluation parameter. Furthermore, the imaging device may comprise an output generation unit configured to generate an output based on the assessment.

[0009] According to one aspect, a medical system is provided that includes the imaging device and a medical device. One aspect also relates to a method for operating a medical imaging device that includes a spatially and spectrally resolving image acquisition unit that includes at least one optical system and at least one image acquisition sensor coupled to the optical system, which is configured to generate image data of an object region that includes spatial and spectral information. This can be an imaging device according to the invention. The method can include a step of at least partially automated, in particular automated, generation / acquisition of image data of the object region that includes spatial and spectral information.The method may comprise a step of at least partially automated, in particular automated, creation of an analysis of the image data based on spatial and spectral information, wherein the analysis comprises at least one evaluation comprising at least one evaluation parameter relating to an object sub-area of ​​the object region. The method may further comprise at least partially automated, in particular automated, creation, in accordance with the analysis and on the basis of information relating to a diagnostic and / or therapeutic action to be performed and / or performed, of an assessment of an attribute of the object sub-area relevant to the diagnostic and / or therapeutic action, which is based on the evaluation parameter. The method may further comprise at least partially automated, in particular automated, outputting of an output based on the assessment.

[0010] Furthermore, one aspect relates to a method for medical imaging. The method can be carried out using an imaging device according to the invention and / or using a medical system according to the invention. The method can comprise a step of at least partially automated, in particular automated, generation / capture of image data of the object region, which comprises spatial and spectral information. The method can comprise a step of at least partially automated, in particular automated, creation of an analysis of the image data based on spatial and spectral information, wherein the analysis comprises at least one evaluation comprising at least one evaluation parameter relating to an object sub-region of the object region.The method may further comprise an at least partially automated, in particular automated, generation, in accordance with the analysis and based on information regarding a diagnostic and / or therapeutic action to be performed and / or performed, of an assessment of an attribute of the object sub-area relevant to the diagnostic and / or therapeutic action, which is based on the evaluation parameter. The method may further comprise an at least partially automated, in particular automated, output of an output based on the assessment.

[0011] These features allow a high degree of reliability to be achieved when planning, implementing, and / or assessing diagnostic and / or therapeutic actions. Furthermore, a high degree of quality can be achieved. Furthermore, the efficiency of planning, implementing, assessing, and / or training such diagnostic and / or therapeutic actions can be increased. By generating and outputting an assessment based on an analysis of image data, at least partially automatically, a user can be supported in a targeted and effective manner. Even users with comparatively little experience can be shown options for diagnostic and / or therapeutic actions for therapy and / or diagnosis, or they can be supported and guided in doing so, thus optimizing the outcome for the patient.

[0012] The imaging device can be a microscopic, macroscopic, and / or exoscopic imaging device. The imaging device can be configured as and / or comprise a microscope, macroscope, and / or exoscope. In some embodiments, the imaging device is an endoscopic imaging device. The imaging device can be an endoscope device. It can comprise and / or be configured as an endoscope and / or an endoscope system and / or form at least a part and preferably at least a major part and / or a main component of an endoscope and / or an endoscope system. “At least a major part” can mean at least 55%, preferably at least 65%, more preferably at least 75%, more preferably at least 85%, and most preferably at least 95%, in particular with reference to a volume and / or a mass of an object.

[0013] In some embodiments, the imaging device is configured to be insertable into a cavity for inspection and / or observation, for example into an artificial and / or natural cavity, such as the interior of a body, a body organ, tissue, or the like. The imaging device can also be configured to be insertable into a housing, casing, shaft, pipe, or other, particularly artificial, structure for inspection and / or observation. The imaging device and in particular the optics and / or the image acquisition sensor system can be configured for multispectral and / or hyperspectral imaging, specifically for capturing and / or generating multispectral and / or hyperspectral image data. Multispectral imaging orMultispectral image data can refer in particular to imaging in which at least three spectral bands are and / or can be independently detected. Hyperspectral imaging or hyperspectral image data can refer in particular to imaging in which at least 20, at least 50, or even at least 100 spectral bands are and / or can be independently detected. The imaging device can operate according to the pushbroom principle and / or the whiskbroom principle and / or the staring principle and / or a snapshot principle.

[0014] In some embodiments, the imaging device comprises a white-light camera and / or sensors for white-light image acquisition. The imaging device can be configured for white-light imaging in addition to spectrally resolved imaging. Separate optics and / or shared optics can be used for this purpose. The white-light imaging and the spectrally resolved imaging can be performed simultaneously or alternately, or at times simultaneously and at times sequentially.

[0015] For some applications, it can be advantageous to be able to use a high spectral resolution. Hyperspectral imaging is then a suitable option. This can be combined with white-light imaging. This enables real-time observation via a white-light image, even if the acquisition of spectrally resolved image data is essentially only real-time, meaning that, for example, several seconds are required to create a spectrally resolved image.

[0016] For some applications, it may be advantageous to generate spectral image data in real time. This includes, for example, generating a spectrally resolved image in less than one second or even multiple times per second. In this case, it may be appropriate to use multispectral imaging. A possibly lower spectral resolution is then offset by a higher frame rate. Depending on the application, it may be sufficient to consider only a few different spectral ranges and / or wavelengths, for example, two or three or four, or generally fewer than ten. In this case, additional white light imaging can optionally be omitted. Spectrally resolved image data that is acquired in real time ordeliver several images per second, can also be used for surveillance purposes, whereby it is not necessarily necessary to create an image for a user to display, but the image data can also be processed in the background.

[0017] The medical imaging device can have at least a proximal section, a distal section and / or an intermediate section. The distal section is designed in particular to be introduced into and / or located in a cavity to be examined in an operating state, for example during the diagnostic and / or therapeutic action. The proximal section is designed in particular to be arranged outside the cavity to be examined in an operating state, for example during the diagnostic and / or therapeutic action. “Distal” should be understood in particular to mean facing towards a patient and / or facing away from a user during use. “Proximal” should be understood in particular to mean facing away from a patient and / or facing away from a user during use. In particular, proximal is the opposite of distal.The medical imaging device has, in particular, at least one, preferably flexible, shaft. The shaft can be an elongate object. Furthermore, the shaft can at least partially and preferably at least largely form the distal section. An “elongate object” is to be understood, in particular, as an object whose main extent is at least a factor of five, preferably at least a factor of ten, and particularly preferably at least a factor of twenty greater than a greatest extent of the object perpendicular to its main extent, i.e., in particular, a diameter of the object. A “main extent” of an object is to be understood, in particular, as its longest extent along its main extension direction.A “main extension direction” of a component is to be understood in particular as a direction which runs parallel to a longest edge of a smallest imaginary cuboid which just completely encloses the component.

[0018] The image acquisition unit can be arranged at least partially and preferably at least largely in the region of the proximal section and / or form this. In other embodiments, the image acquisition unit can be arranged at least partially and preferably at least largely in the distal section and / or form this. Furthermore, the image acquisition unit can be arranged at least partially distributed between the proximal section and the distal section. The image acquisition sensor system has, in particular, at least one image sensor. Furthermore, the image acquisition sensor system can also have at least two and preferably more image sensors, which can be arranged one behind the other.Furthermore, the two and preferably multiple image capture sensors can have spectral detection sensitivities that differ from one another, so that, for example, a first sensor is particularly sensitive or comparatively more sensitive than the other sensors in a red spectral range, a second sensor in a blue spectral range, and a third sensor in a green spectral range. The image sensor can be designed, for example, as a CCD sensor and / or a CMOS sensor.

[0019] The optics of the image capture unit can comprise suitable optical elements such as lenses, mirrors, gratings, prisms, optical fibers, etc. The optics can be configured to guide object light coming from the object area to the image capture sensor system, for example, to focus and / or project it. The object light can, in particular, originate from illumination of the object area.

[0020] The image acquisition unit is particularly configured to generate at least two-dimensional spatial image data. The image acquisition unit can be spatially resolving in such a way that it delivers a resolution of at least 100 pixels, preferably of at least 200 pixels, preferably of at least 300 pixels, and advantageously of at least 400 pixels in at least two different spatial directions. The image data is preferably at least three-dimensional, with at least two dimensions being spatial dimensions and / or with at least one dimension being a spectral dimension. A plurality of spatially resolved images of the object region can be obtained from the image data, each of which is assigned to different spectral bands. The spatial and spectral information of the image data can be such that an associated spectrum can be obtained for a plurality of spatial pixels.

[0021] In some embodiments, the image acquisition unit is configured to continuously generate updated image data. For example, the image acquisition unit may be configured to generate the image data substantially in real time, which may include, for example, generating updated image data at least every 30 seconds, in some cases at least every 20 seconds, and in some cases even at least every 10 seconds or at least every 5 seconds.

[0022] The object region can be an image region that includes at least a part and / or section of an imaged object. The object region can relate to tissue and / or organs and / or a part of a patient's body. For example, the object can be a site. The object region can relate to a site.

[0023] The imaging device can comprise an illumination device that includes at least one illuminant configured to illuminate and / or illuminate the object region in at least one operating state. The illuminant can comprise a white light source, a particularly tunable monochrome light source, a laser, a white light laser, at least one light-emitting diode and / or a light-emitting diode array, at least one laser diode and / or a laser diode array, or the like. The illumination device can be formed integrally with the image capture unit. In particular, the illumination device can utilize individual or all components of the optics of the image capture unit and / or have separate illumination optics. An illumination light beam can be guided and / or guideable, at least in sections, coaxial with a measurement light beam.

[0024] The analysis of the image data is based in particular on both spatial and spectral information. The evaluation can be based on processing both spatial and spectral information to determine the evaluation parameter. The object sub-area can be selected at least partially and in particular automatically. Alternatively or additionally, the object sub-area can be selected and / or defined by a user. For example, it can be provided that the user selects the object sub-area by clicking, drawing, staking, tapping or the like, for example starting from a representation of the object area. The evaluation parameter can be obtained in particular from spectral information assigned to the object sub-area using a mathematical rule. In this case, the evaluation parameter can in particular assume a continuous value range of 0-100% or a discrete value range 0, 1, 2, ...The mathematical rule may comprise an algebraic calculation rule and / or a machine learning method, e.g., a KL model. The evaluation parameter is based, in particular, on a spectral value for a specific wavelength and / or a specific wavelength range and / or a specific spectral band.

[0025] The diagnostic and / or therapeutic action to be performed and / or performed may be planned and / or currently being performed and / or already partially or completely completed. The diagnostic and / or therapeutic action may comprise at least one diagnostic step and / or treatment step and / or therapy step. The diagnostic and / or therapeutic action may alternatively or additionally be related to the commissioning, maintenance, and / or adaptation of a medical device and / or the imaging device itself. For example, it may be sterilization and / or cleaning, such as an at least partially automated or automatic cleaning of a window, such as a distal end window and / or illumination window, of the imaging device. In some embodiments, the diagnostic and / or therapeutic action is a diagnosis and / or an intervention and / or a treatment.The diagnostic and / or therapeutic procedure can be invasive and / or microinvasive. For example, the diagnostic and / or therapeutic procedure can include vessel sealing, tubal sterilization, welding and / or joining tissue structures, such as anastomosis formation, ligation, insufflation, gas humidification, clip application, suturing or stapling, exposing certain structures such as vascular structures, bile ducts, nerve cords, lymph node tracts, fascia, etc., removing calculi such as kidney stones, gallstones, etc., tissue denaturation, suctioning of secretions, treatment of stenoses, or the like.

[0026] The information, hereinafter also referred to as "assessment information," which forms the basis for the assessment in addition to the analysis, is particularly different from the spatial and spectral information. The assessment information may be such that it cannot be obtained from the spatial and spectral information. The assessment is based in particular on the evaluation and additionally includes consideration of the specific circumstances during the planning, implementation, or evaluation of the diagnostic and / or therapeutic action. These specific circumstances may also arise, in particular, from prior knowledge, which is acquired, for example, through device parameters and / or AI detection in the white light image and / or preoperative imaging. The assessment information may be present in a database and / or originate from artificial intelligence or be obtainable from a neural network.The assessment information may be based on empirical values ​​and / or previous results and may include, for example, lookup tables, value ranges, thresholds, categorical data, image data, spectral data, etc., associated with a respective diagnostic and / or therapeutic action. In some embodiments, the assessment information is alternatively or additionally associated with the status of the diagnostic and / or therapeutic action, in particular with regard to whether the action is planned and / or is currently being performed and / or has been performed.

[0027] In some embodiments, the relevant attribute relates to the suitability and / or unsuitability of the object sub-area for the diagnostic and / or therapeutic action to be performed and / or performed. Alternatively or additionally, the attribute may relate to a quality and / or condition of the object sub-area or the object depicted therein and / or the treatment result. The attribute may be categorical and / or metric and / or numeric.

[0028] In some embodiments, the output is a digital output and / or an analog output. The output may be directed to a user and / or a device, or may include information that is processable by a user and / or a device.

[0029] The imaging device may comprise a control unit. The control unit may comprise suitable control electronics and / or a computer. In some embodiments, the control unit comprises at least one processor, computer-readable memory, an operating system, and / or suitable inputs and outputs. The control unit includes at least one control program. The image acquisition unit, the image analysis unit, the assessment unit, and / or the output generation unit may be at least partially implemented by the control unit. In particular, functions thereof may be implemented by the control program or be part of it.The imaging device, and in particular the control unit, can each comprise at least one processor and / or an associated memory with program code that implements the described functions and steps, and / or an associated main memory and / or associated connections and / or data interfaces, and / or an electronic circuit for implementing the functional units mentioned herein and / or for carrying out the method steps mentioned herein. One or more processors, memories, main memories, connections, data interfaces, and / or circuits can also be associated with one or more functional units and / or implement one or more method steps.

[0030] The imaging device may comprise an output unit configured to output the output and / or user output generated in accordance with the output to a user. The output unit may be configured to output a visual output and / or an audio output and / or a haptic output and / or any other output perceivable by a user. For this purpose, the output unit may comprise suitable components, such as one or more lamps, illuminators, loudspeakers, screens, vibrators, or the like. The output unit may comprise a computer and / or processor and / or memory and / or RAM and / or ports and / or a data interface for receiving, processing, and outputting unprocessed, preprocessed, and / or processed output data. The output generation unit may be connected to the output unit via an interface.The generated output can be processed and / or output by the output unit.

[0031] The imaging device may comprise a display unit configured to display an image, in particular a moving image, for a user. The display unit may be part of the output unit and / or form the output unit. The displayed image may be based on the image data. The display unit may comprise a screen and / or control electronics. The display unit may comprise a computer and / or processor and / or memory and / or RAM and / or ports and / or a data interface for receiving, processing, and outputting unprocessed, preprocessed, and / or processed image data and / or display data. The output generation unit may be connected to the display unit via an interface. The generated output may be processed and / or output by the display unit.

[0032] The imaging device may comprise an audio output unit configured to output an audio signal. The audio output unit may be part of the output unit and / or form the output unit. The audio output unit may comprise a loudspeaker and / or control electronics. The audio output unit may comprise a computer and / or processor and / or memory and / or RAM and / or ports and / or a data interface for receiving, processing, and outputting unprocessed, preprocessed, and / or processed output data. The output generation unit may be connected to the audio output unit via an interface. The generated output may be processed and / or output by the audio output unit.

[0033] The imaging device may comprise a user interface configured to allow a user to provide input. Such input may be used to operate the imaging device and / or the medical device and / or the display unit. The user interface may comprise input means such as buttons, rotary switches, a touch display, a mouse, a keyboard, voice input, at least one foot switch, or the like.

[0034] The output unit and / or the audio output unit and / or the display unit and / or the user interface can be integrated.

[0035] The imaging device may comprise an imaging device. The imaging device may be, for example, an endoscope. The imaging device may at least partially comprise the imaging unit, in particular at least the optical and / or sensory components of the imaging unit.

[0036] The imaging device may comprise a lighting device. The lighting device may comprise at least part of the lighting unit, in particular at least its electronic and / or optical components.

[0037] The imaging device may comprise an output device. The output device may comprise at least part of the output unit, in particular at least part of the display unit and / or the audio output unit, for example, their electronic and / or mechanical components such as a screen, a loudspeaker, etc.

[0038] The imaging device may include a control unit. The control unit may at least partially or completely include the control unit, in particular its electronic components.

[0039] Components of the user interface can be distributed among the different devices or located entirely within one of the devices. The imaging device can be connected to the illumination device via an electronic and / or optical interface. The imaging device and / or the illumination device and / or the output device can be connected to the control device via a preferably electronic and / or optical interface.

[0040] The aforementioned methods can each be carried out, in particular, within the framework of a simulation and / or demonstration and / or during a test and / or a trial run remote from the patient. The diagnostic and / or therapeutic action to be performed and / or performed can be real and / or virtual. In particular, the performance of this action is not part of the aforementioned methods. The aforementioned methods can each be carried out with computer support. For example, one or more of the at least partially automated steps can comprise calculations that are performed by at least one processor and / or that are based on the execution of program code.

[0041] The medical device may be a device, instrument, tool, etc., particularly a diagnostic and / or therapeutic and / or surgical device. In some embodiments, the medical device is a laparoscopic instrument. For example, the medical device may comprise and / or form an insufflator, a light source, an anesthesia device, an energetic system, and / or an energetic device such as an RF instrument, a laser instrument, a motorized instrument, a medical robot, an endoscope, an endoscope system, a suturing device or suturing system, a light source or illumination system, or the like. The medical device may be arranged at least partially in the object area before, during, and / or after the diagnostic and / or therapeutic action.The imaging is then used, among other things, to observe the medical device, for example a tool of the same, before, during and / or after the impact on body parts in the object area.

[0042] In some embodiments, the medical device is a tissue-coagulating, in particular vessel-sealing or tissue-welding, instrument. The medical device can be configured to deliver energy into tissue in a targeted manner. The medical device can comprise an RF generator. Alternatively or additionally, the medical device can comprise a laser, a radiation source, and / or another energy source. An energy output of the medical device can be adjustable, in particular with regard to an energy quantity, a power, a temporal and / or spatial output profile, an output frequency, an output waveform, or the like.

[0043] A connection between the imaging device and the medical device can be easily established, in particular if the medical imaging device comprises a device interface to which the medical device can be connected and / or connected, in particular detachably and / or tool-free. The device interface can be wired and / or wireless. The device interface can be configured to establish a connection to different medical devices, for example selectively or simultaneously. The imaging device can have a device recognition unit that automatically recognizes the type and / or model of the medical device. The recognition can be taken into account for the assessment. For example, the relevant attribute can depend on the type and / or model of the recognized medical device.

[0044] According to one aspect, the imaging device comprises a recommendation unit configured to generate a recommendation for a user based on the assessment of the attribute of the object sub-area. The output generation unit may be configured to generate the output based on the recommendation. By generating a recommendation, a user may be assisted in making a more accurate situational assessment. For example, a user may indirectly draw on the experience of other clinicians, testers, surgeons, diagnosticians, etc., because their experience is reflected in the recommendation.

[0045] The recommendation may be a recommended course of action for performing a diagnostic and / or therapeutic action on a patient. This may include a suggestion to perform a specific action, to select one of several actions, not to perform a specific action and / or a series of specific actions, to perform actions in a specific order or not, and the like. The recommended course of action may be categorical, in particular nominally scaled. The recommended course of action may also be ordinally scaled. In some embodiments, the recommended course of action may include multiple alternatives, which may in particular be sorted according to a recommended ranking. By using a recommended course of action, a user may be encouraged to perform a diagnosis and / or treatment in a specific way, thereby achieving a high degree of efficiency in planning and implementation.In addition, the associated chances of success can be increased or the risk of errors minimized because a situation-adapted recommendation can be generated depending on the outcome of the assessment.

[0046] A user can be supported effectively and specifically, especially if the recommendation concerns the selection and / or setting and / or handling of a medical device other than the imaging device. For example, the recommendation unit can be configured to suggest to the user, based on the assessment of the attribute, which therapeutic and / or diagnostic action appears medically appropriate and to suggest suitable equipment, a suitable methodology, and / or a suitable configuration for this purpose.

[0047] In some embodiments, the recommendation relates to whether and / or in which area the object area should be surgically affected.

[0048] Alternatively or additionally, the recommendation refers to whether and / or in which area of ​​the object area should not be surgically manipulated. This can make it easier for a user to quickly and reliably identify medically relevant tissue structures, organs, body parts, tissue sections, etc., guide them to work in an area that promises informative diagnostics and / or successful treatment, and prevent them from working in an area that could result in an inadequate diagnosis and / or unsuccessful treatment.

[0049] The recommendation can be generated particularly precisely and / or adequately if it is based on a comparison of the assessment with a database. The database can contain data records on various diagnostic and / or therapeutic actions to be performed and / or performed, as well as, in some embodiments, associated recommendations. The recommendation can be based on the selection of a specific data record from the database, which operates according to a best-match algorithm. In some embodiments, the recommendation can also be based on a combination of multiple data records. The recommendation can therefore extend beyond the diagnostic and / or therapeutic situations contained in the database.In particular, the recommendation can be determined in a multi-stage process in which a first partial assessment is used to recommend a specific diagnostic and / or therapeutic action. In which, based on the selected diagnostic and / or therapeutic action and a second partial assessment for this action, situation-appropriate settings, handling recommendations for a medical device used, recommendations regarding suitable and / or unsuitable areas of the body region under consideration, etc. are then recommended. Furthermore, it can be provided that the assessment is carried out in several stages. In particular, a first database comparison can be carried out based on a first partial assessment. The result of the first database comparison can form the basis for a second partial assessment. The second partial assessment can in turn be used for a second database comparison.In this way, the recommendation can be generated step by step and based on combinations of data records that are not explicitly stored in the database itself.

[0050] A high degree of safety can be achieved, in particular, if the assessment includes the detection and / or classification of risk structures. The recommendation can include information regarding the detection and / or classification of risk structures. This can also reduce the likelihood of treatment errors. A risk structure can be identified as a sub-area of ​​the object area that should not be therapeutically influenced, specifically not with a medical device currently in use and / or connected to the imaging device. The risk structure can, for example, be an area in which no incision and / or tissue coagulation should be performed, for example due to fear of bleeding. Depending on the type of diagnostic and / or therapeutic action, risk structures can include different body parts, tissue types, vessels, organs, regions, etc.Specifically, for example, those that a competent user and / or physician and / or clinician would identify as unsuitable for the action, had they recognized them. The classification can be two-level, such as "suitable" and "unsuitable," but also multi-level, in particular at least three-level.

[0051] According to one aspect, the imaging device comprises an evaluation unit configured to assign at least one rating to the recommendation depending on a user's behavior in response to the recommendation prompted by the recommendation. By thus taking into account how a user behaved and, for example, whether or not they followed a recommended action, the quality of the recommendation can be assessed. The evaluation unit can further be configured to transmit the rating to a recommendation database. This can provide a learning system that can utilize human input to generate future recommendations of higher quality. The recommendation database can be part of the aforementioned database or different from it. The user's behavior can be an acceptance and / or rejection and / or adaptation of a recommended action.

[0052] Supporting the user during a diagnosis, an intervention and / or a treatment can be particularly effective if the recommendation unit is configured to adapt the recommendation essentially in real time depending on continuously updated image data.

[0053] For example, an updated recommendation can be determined for each updated image data. This allows the recommendation to be adjusted if a viewing angle and / or image section changes, if the situs changes, if the medical device is moved and / or modified, etc.

[0054] According to one aspect, the imaging device comprises a parameter determination unit configured to determine at least one control parameter for controlling a medical device based on the assessment of the attribute of the object subregion. The output generation unit can be configured to generate the output based on the at least one control parameter. By determining a control parameter, a medical device in use can be used particularly effectively. A user can be assisted in using suitable parameters appropriate to the situation. This allows even a less experienced user to use the medical device without having to know all the details regarding its optimal setting.The control parameter can relate to a flow rate and / or gas composition of a gas-humidifying insufflator, brightness and / or spectral settings of a light source, a delivery rate of an anesthesia device, parameters for setting a robot such as position, orientation, feed rate, gripping force, etc., optical setting parameters such as focus, aperture, exposure time, color contrast enhancements in the parallel white-light video, etc., a shape and / or size of a needle used, a thread material, a closing force for a stapler, cleaning parameters for cleaning a window, or the like. The control parameter can be read from at least one list, table, database, or the like. Alternatively or additionally, the control parameter can be calculated by a machine learning algorithm, such as based on an RNN model, an LSTM model, according to a random forest method, or the like.

[0055] If the imaging device includes a device interface, this can be configured to transmit the at least one control parameter to the medical device. This allows suitable parameters to be transferred to the medical device efficiently. Instead of manual input or the use of a program or parameter set stored in the medical device independently of the situation, a parameter set adapted to the individual diagnostic or therapeutic situation can be used.

[0056] In some embodiments, the control parameter can be transmitted to the medical device automatically and / or without user intervention, in particular at least substantially in real time. For this purpose, it can be provided that the parameter determination unit and the output generation unit can operate in an automatic mode, in particular one that can be activated perceptibly by the user. This can ensure that optimized parameters are reliably used.

[0057] Alternatively or additionally, the output can include a suggestion for a user to automatically forward at least one control parameter to a medical device that can be controlled using the control parameter. This can relieve the user's workload and allow them to focus more on the diagnostic and / or therapeutic action rather than on setting up the medical device. The suggestion can, for example, include a parameter set. In this way, various settings on the medical device can be made automatically with simple user approval. At the same time, it is ensured that no settings are made without user approval.

[0058] The user interface can be configured to receive at least one user input, by means of which the user can accept and / or reject and / or adapt the suggestion. The user can be brought to the level of a highly experienced doctor or clinician in a simple and intuitive manner by suggesting suitable control parameters. Nevertheless, the decision as to whether and in what manner the medical device should actually be controlled can remain with the user. Furthermore, this can enable the user to make adjustments that they deem necessary based on their individual assessment. The suggestion can comprise a display of a parameter set, a graphic representation, an overlay, particularly in data glasses, an audio description of the settings to be made, or the like.

[0059] Situation-dependently adapted and thus optimized settings of the medical device can be provided reliably and effectively, especially if the parameter determination unit is configured to adapt the at least one control parameter essentially in real time depending on continuously updated image data. In some embodiments, the output may include an overlay and / or display and / or representation of a current control parameter or parameter set, which the user can use to obtain information about the current control parameter.

[0060] Furthermore, it may be provided that the user can optionally interrupt an automated transmission of the control parameter, for example if he considers an updated control parameter to be unsuitable.

[0061] The at least one control parameter can be configured to define an operating mode of the medical device. The at least one control parameter can comprise a parameter set. For example, the control parameter can cause a coagulation instrument to be set for a specific tissue entity. In this case, the assessment includes, for example, whether the object sub-area is suitable for tissue coagulation and what type of tissue it is and / or what its surroundings are like. Suitable parameters for the coagulation instrument can then be determined for the thus recognized scenery, for example by comparing them with a database. These parameters can then be transmitted to the instrument automatically before the instrument is used in the object sub-area and / or after user approval in order to adjust it accordingly.It can also be provided that, as part of the assessment, a condition such as tissue moisture is determined and an assessment is made as to whether this is sufficient or whether additional humidification is required. Based on this, optimized setting parameters for a gas humidification device and / or an insufflator can be determined. Alternatively or additionally, the at least one control parameter can be configured to effect a safety-relevant shutdown of the medical device. This can be expedient, for example, when critical anatomical structures are present in a coagulation area, such as a patient's ureter. The device interface and / or the output generation unit can be configured to automatically transmit the control parameter to the medical device, at least when it effects the safety-relevant shutdown.Alternatively, a warning can be issued to the user, upon which the user authorizes the transmission of the control parameter that causes the shutdown to the medical device. In some embodiments, the assessment can include assessing the safety relevance of a performed and / or upcoming diagnostic and / or therapeutic action. If it is assessed that the patient's safety is at risk, for example due to an impending incision and / or an impending impact on an area that should not be impacted accordingly, the control parameter that causes the safety-relevant shutdown can be generated. In some embodiments, this is automatically output to the medical device even if the user can and / or must authorize other, non-safety-relevant control parameters before they are output to the medical device.

[0062] A simple, reliable, and user-friendly review of the success of a performed diagnostic and / or therapeutic action can be enabled, in particular, if the assessment comprises an evaluation of the quality of a diagnostic examination and / or a treatment result. The output can comprise information regarding the quality of the diagnostic examination and / or the treatment result. The assessment can comprise at least a comparison with a database containing information on different examination and treatment results. The assessment can comprise a binary and / or categorical, in particular nominally or ordinally scaled, quality parameter. Alternatively or additionally, the assessment can comprise a continuous quality parameter and / or be metric. The quality assessment can relate to the suitability of recorded image data for making a diagnosis.Furthermore, the assessment of quality can relate to the success of a performed procedure. For example, it can be assessed whether tissue coagulation is sufficient to achieve the desired effect, such as complete and secure vessel sealing, whether a suture and / or one or more staples were correctly applied, whether a structure was successfully exposed, whether a ligation was correctly performed, whether a biopsy provided the required tissue samples, whether bipolar tubal sterilization was successful, or the like. Information regarding quality can be provided to the user during and / or immediately after the diagnostic and / or therapeutic action has been performed. This allows the user to, if necessary,react to the quality assessment and, for example, completely seal a partially sealed vessel, supplement an inadequate suture line, take additional tissue samples, select changed settings for image acquisition, etc.

[0063] The assessment can be based on at least two different sets of image data and comprise a comparison of the attribute of the object sub-region at two different points in time. In some embodiments, the attribute is determined before and after performing a diagnostic and / or therapeutic action. Comparing the attribute before performing the diagnostic and / or therapeutic action with the attribute afterward allows any changes to be reliably detected. A first set of image data can serve as a reference against which a second set of image data is compared. The assessment can also comprise a comparison series in which gradual changes in the attribute are observed. In some embodiments, the comparison of different sets of image data can serve to evaluate the quality of a diagnostic examination and / or a treatment result.The different sets of image data can be acquired according to image tracking. In particular, a later acquired set of image data is based on tracking according to an earlier acquired set of image data. The tracking can be based on the spatial information and / or on a white-light image. The tracking can occur continuously, in particular also between the recordings that result in the at least two different sets of image data. If tracking is used, it can be performed, for example, using an optical flow algorithm, a landmark-based algorithm, or another suitable algorithm.Alternatively or additionally, a specific image region, such as a section of a medical device and / or a characteristic body part, particularly as determined during image segmentation, can be detected in the different sets of image data, and based on this detection, these can be related to their spatial coordinates. In some embodiments, a localization of a viewed object sub-region can be performed based on the respective spectral information.

[0064] According to one aspect, the analysis of the image data comprises image segmentation. This allows objects to be detected and / or tracked at least partially automatically, for example, as part of a tracking process. In particular, the analysis unit can be configured to use the image segmentation to detect a medical device that is located at least partially in the object region. The image segmentation can be performed based on the spatial information and / or the spectral information of the image data. The image segmentation can be performed with respect to the spatial coordinates and / or according to a spectral criterion, for example, according to values ​​in one or more specific spectral ranges.For example, if spectral ranges are selected for which high or low absorption values ​​indicate the presence of water and / or fat and / or protein and / or hemoglobin, image segmentation can be performed according to these spectral ranges. Spatial regions can then be automatically detected in which a certain concentration of water and / or fat and / or protein and / or hemoglobin is present.

[0065] Alternatively or additionally, the analysis of the image data includes tissue recognition. This allows the observed scene to be evaluated particularly accurately and the assessment to be carried out particularly precisely. Tissue recognition can be performed, in particular, on the basis of image segmentation. For example, structures such as organs, vessels, nerves, fascia, and / or body parts can be recognized. Tissue recognition can include a type, extent, wall thickness, type, and / or composition of the tissue. Furthermore, tissue recognition can also include a number, for example, a number of blood vessels, nerves, fascia, and the like. Tissue recognition can be based on spatial and / or spectral information.Tissue detection can take place in several stages and, for example, comprise a first detection, in particular a coarse detection, based on a segmentation of a white light image and / or an image for a single spectral band, and a second detection, in particular a fine detection, based on additionally evaluated spectral information. In some embodiments, the analysis of the image data can comprise the detection of at least one medical device. This allows the imaged situation to be assessed with high accuracy. The detection of the at least one medical device can, in particular, comprise information about a type and / or an orientation and / or a position and / or a state of the medical device. A KL algorithm can be used to determine the type and / or orientation and / or position and / or state of the medical device.For example, the position of the medical device can be detected using a segmentation model such as ll-Net with a ResNet50 backbone. 2D convolutions and / or 3D convolutions can be used. Alternatively or additionally, a detection model such as a YOLO algorithm can be used, particularly when information about a medical device type and / or a comprehensive bounding box is required, specifically when only such information is required or when this information is sufficient.

[0066] Models and / or algorithms for image segmentation and / or tissue recognition and / or medical device detection can be trained using annotated data. A trained model created in this way can then be applied to new image data that was not used for training.

[0067] The analysis may include mapping the image data to three-dimensional data relating to the object region. The image segmentation and / or tissue detection and / or detection of a medical device may be linked to a mapping, for example a 2D mapping and / or a 3D mapping, based on existing spatial data of the object region and / or on spatial data of the object region obtained by imaging without spectral information, such as white light imaging. In some embodiments, such mapping may include the use of stereoscopic image data. Alternatively or additionally, mapping may be performed using a partially or fully simulated spatial model, in particular a 3D model. Spatial data may also originate from a tracked system.For example, the image data can be compared with data provided by the medical device and / or a robot manipulating it, which data allows conclusions to be drawn about a location, in particular a position and / or orientation, of the medical device. In principle, mapping can be carried out using spatial data that is / were acquired in real time, substantially in real time and / or in advance. In some embodiments, verification can be carried out using pre-operative data. For example, image data, in particular spatial and / or spectral image data, can first be acquired at a specific position and / or in a specific viewing direction in order to calibrate and / or adjust the imaging device. This step is optional. Furthermore, spatially and spectrally resolved image data of the object region can be generated using the imaging device. The analysis of this data can, if necessary,The results of the aforementioned adjustment and / or calibration step can be used as a basis. Tissue structures and / or sections of a medical device identified during an analysis of the image data can then be compared or verified with pre-operative data relating to the object area. This makes it possible to check, before tracking identified structures, whether the corresponding detection is correct based on the spatially and spectrally resolved image data, for example whether an organ, a vessel, a body part, a nerve, a fascia, etc. was correctly identified. Furthermore, this makes it possible to perform tracking reliably even if the tracked structure is only partially represented in the spatially and spectrally resolved image data, since the assignment of this partial representation is based on the use of additional data.

[0068] In general, in some embodiments, the attribute may include information about anatomical and / or physiological characteristics of the object subregion. For example, the attribute may include a content of a specific substance, in particular a water content, a hemoglobin content, a fat content, a protein content, or the like; a tissue coagulation state, in particular a vascular occlusion state; a geometry, coordinates, directional information, an amount and / or direction of blood flow, a blood flow state, a humidity and / or dehydration state, or the like.

[0069] Alternatively or additionally, the analysis may include information about anatomical and / or physiological characteristics of the object subregion. For example, the evaluation and in particular the evaluation parameter may include a content of a specific substance, in particular a water content, a hemoglobin content, a fat content, a protein content, or the like, a degree of tissue coagulation, a geometry, coordinates, directional information, an amount and / or direction of blood flow, a blood flow state, a humidity and / or a degree of dehydration, or the like. In this case, the attribute may be a classification and / or categorization based on the evaluation or the evaluation parameter.For example, the analysis may provide a metric and / or numerical evaluation parameter, such as a salary amount, a volume, a length, an angle, or the like, and the attribute may be based on a comparison of the evaluation parameter with one or more value ranges, each of which is assigned a category and / or designation, in particular a nominal and / or ordinal one. The assignment may be based on information from a database and / or from artificial intelligence and / or using an appropriately trained neural network. In this regard, reference is also made to the above description.

[0070] According to one aspect, the imaging device comprises a smoke detection unit and / or a contamination detection unit configured to detect the presence and / or absence of smoke and / or contamination in the object region and / or on an optical window. This allows for automatic determination of whether the prerequisites for successful or qualitatively satisfactory imaging are met. Smoke generated, for example, during energy input, such as during tissue coagulation, and covering parts of the object region can be detected. The presence of contamination on windows caused by tissue residues, blood, fluids, particles, and the like can be detected.The smoke detection unit and / or the contamination detection unit can be configured to perform smoke detection and / or contamination detection based on a white-light image using a KI algorithm, for example, a binary classification algorithm, such as ResNet 50. For this purpose, the object area can be imaged. It may also be expedient to adjust the focus appropriately to direct it more toward potentially dirty windows. Smoke detection and / or contamination detection can be performed based on the analysis. In particular, in some embodiments, smoke detection and / or contamination detection can be based on spectral information, for example, on absorption values ​​in characteristic spectral bands that allow conclusions to be drawn about the presence of certain substances.It can be provided that smoke detection and / or pollution detection only takes place for a specific spatial selection of image data, for example for an area for which further image data is to be recorded.

[0071] Smoke and contamination can be efficiently and automatically taken into account for generating an output or for an automatic assessment, in particular if the assessment unit is configured to take into account the presence and / or absence of smoke and / or contamination in the object area and / or on a window, as detected by the smoke detection unit and / or the contamination detection unit, when assessing the attribute. For example, a quality value can be assigned to the attribute, which depends on a degree of contamination and / or a degree of impairment of image capture by smoke. In this case, the output can be provided with the quality value or a parameter derived therefrom, in particular to indicate a possible impairment for a user.Alternatively or additionally, the image acquisition unit can be configured to generate image data of the object area again at a time after the image data of the object area has been generated, if the smoke detection unit and / or the contamination detection unit has detected the presence of smoke and / or contamination in the object area and / or on a window based on the image data of the object area. A waiting time applied until this time can be fixed, user-specifiable, and / or adaptive.

[0072] A high degree of ease of use and / or intuitive comprehensibility of information directed at a user can be achieved in particular if the output comprises at least one overlay directed at a user that is superimposed on a visual representation of the object area. For example, the display unit can generate a visual representation of the object area based on spatial and / or spectral image data. This visual representation can be based on the spatially and spectrally resolved image data and / or on image data generated separately, for example, white light image data and / or real-time image data and / or image data that do not include spectral information or that are not multispectral or hyperspectral. The overlay can be superimposed on this representation in part or in its entirety. In some embodiments, the overlay can convey spectrally and spatially resolved information.For example, coloring can occur based on spectral data. The overlay can be generated at a different or the same update rate as the display. In particular, it can be provided that a real-time image containing no spectral information is superimposed with an image that is essentially updated in real time and contains spectral information. Alternatively or additionally, the overlay can contain information determined based on the spatially and spectrally resolved image data, but from which the image data itself is not apparent, for example, categorical information, instructions, selected and / or selectable icons, predefined images, text, coloring, warnings, etc.

[0073] Areas of interest to the user, such as specific body parts, structures, tissue types, vessels, etc., can be identified particularly intuitively if the display includes highlighting and / or coloring at least parts of a visual representation of the object area. For example, the coloring can be done in at least two colors, expressing categories such as "suitable" and "unsuitable" and / or "safe" and "dangerous" and / or "adequate quality" and "poor quality." The coloring can be based on a color scale. The color scale can be multi-level, in particular at least three-level, at least five-level, or at least ten-level, or it can be continuous.

[0074] Alternatively or additionally, a visual output can be provided via colored lamps and / or a separate display, in some embodiments via a display of the medical device. In this respect, the output unit can be at least partially formed by the medical device. The device interface, for example, can serve as the interface between the output generation unit and the output unit.

[0075] An easily perceptible output that can be registered by the user even if they temporarily look away from a display can be achieved in particular if the output comprises at least one audio output directed at a user. The audio output can be output via the audio output unit. The audio output can comprise a tone, a tone sequence, a noise, a voice output, music, or any combination thereof. In some embodiments, the audio output can comprise a warning, for example, if a diagnostic and / or therapeutic action is about to be performed incorrectly and / or is being performed incorrectly and / or has been performed incorrectly. In particular, a warning can be output if the user is about to work with the medical device in an unsuitable area, which could potentially pose health risks to a patient.

[0076] The output may relate to a tissue classification, in particular a classification according to the type and / or nature of the tissue. This allows the user to quickly and easily identify which tissue types and / or tissue structures are present in the object area.

[0077] A user can be guided intuitively during a diagnosis and / or treatment, in particular if the output includes the suitability and / or unsuitability of at least one partial area of ​​the object area for the diagnostic and / or therapeutic action to be carried out. For this purpose, the output can include a display as described above. In this way, the user can be shown, for example by means of colored markings, arrows, visual highlights or the like, which areas of the object area are suitable and / or unsuitable and / or susceptible to injury and / or not or not fully identifiable / classifiable. Based on this, the user can carry out the diagnostic and / or therapeutic action in a targeted manner at a suitable location or generally in a suitable and safe manner.

[0078] In some embodiments, the output may relate to the quality of a result of the performed diagnostic and / or therapeutic action. The output may be binary and classify the quality, for example, as "OK" or "NOT OK." It may also be provided that multiple quality levels are used, for example, a point scale, a grade scale, or a continuous scale. If, for example, the therapeutic action was tissue coagulation in the object sub-area, it may be output after its implementation whether the tissue coagulation in the object sub-area was complete and / or sufficient or not. If, for example, the therapeutic action was tissue humidification using an insufflator that allows gas humidification of tissue, it may be output after humidification has taken place whether the humidification was sufficient or not.The result whose quality is being assessed can also be an intermediate result and / or a continuously updated result or a state that changes over time, such as in the above-mentioned example of tissue hydration. Treatment errors and / or injuries to the patient can be avoided, in particular, if the output includes a warning directed at a user. The warning can be a visual warning and / or an audio warning and / or a haptic warning. For example, if it is assessed that the user is about to inadvertently damage and / or sever a vessel and / or a nerve and / or fascia, the warning can be issued to draw the user's attention. This can be particularly helpful in microinvasive procedures, as the site cannot then be viewed directly by the user, but only an image of it can be observed.

[0079] The analysis can include a comparison of normalized and / or standardized spectral data. This allows generated image data to be evaluated particularly efficiently. Furthermore, effects resulting from different sized areas of observation can be eliminated if necessary. The normalized and / or standardized data can be normalized and / or standardized spectra. Normalization and / or standardization can include division by a value of the spectrum at a specific wavelength, normalization to an area under the curve, subtraction of a background, etc. During test experiments, typical spectra can be determined that characterize specific tissue types that have been therapeutically influenced in a specific way. For example, the absorption spectrum of coagulated tissue depends on the degree of coagulation and the parameters used for coagulation.If image data are generated before and after a targeted energy input into the tissue and the spectral data are compared, deviations due to different amounts of tissue observed can be compensated for by normalizing and / or standardizing the spectra to be compared.

[0080] In some embodiments, the analysis may comprise a comparison of spectral data for different spectral ranges. This can increase the accuracy of an assessment by comprehensively taking characteristic spectral properties into account. For example, values ​​of different spectra at multiple wavelengths and / or in multiple wavelength ranges can be compared with one another. Furthermore, it can be provided that curve shapes are compared with one another. Curves based on models can be fitted partially or completely to spectra obtained from the image data. Models fitted in this way and / or their underlying parameters can then be compared with one another. For example, different amino acids have different absorption spectra, with many showing pronounced absorption in the 280 nm range.If only this single wavelength were considered, it might be difficult to differentiate between amino acids or specific proteins, in which certain amino acids occur with varying frequencies. However, if data for multiple wavelengths are considered, characteristic absorption shoulders, for example, can be included in the assessment.

[0081] Generally speaking, the spectral information can in many cases relate to the absorption of light. The spectral information can represent an absorption spectrum. Alternatively or additionally, the spectral information can also relate to the emission of light, for example, fluorescence. In general, almost any spectral measurement can be used. The illumination device can function as a measurement light source for an absorption measurement and / or as an excitation light source for an emission measurement. In some embodiments, the imaging device can be operable in different measurement modes, in which, for example, the image acquisition unit and / or the illumination unit are controlled differently to implement different measurement principles.

[0082] Various energy-injecting diagnostic and / or therapeutic actions have already been mentioned several times. Generally speaking, the diagnostic and / or therapeutic action to be performed and / or performed may comprise an energy injection into tissue, in particular a thermal and / or electrosurgical and / or radiation-based energy injection. The medical device may accordingly be an energy-injecting instrument, for example, a coagulation instrument, an HF instrument, a laser instrument, an electrosurgical device, an electroscalpel, an electrocautery device, and / or a vessel sealing device, etc.

[0083] The evaluation parameter can contain information regarding tissue coagulation, in particular regarding vessel sealing, tubal sterilization, and / or the welding and / or connection of tissue structures, such as anastomosis formation. If the output relates to a control parameter, this can, in particular in this case, be a process parameter such as power, voltage, current, resistance, phase shift, tissue shrinkage, temperature, gripping force, or the like.

[0084] A particularly versatile image acquisition unit can be provided and / or a high degree of flexibility and imaging quality can be achieved in particular if the image acquisition unit is multimodal and can be operated in at least a first mode in which spatially and spectrally resolved image data can be generated, and in at least a second mode in which only spatially resolved image data can be generated. The first mode can relate to the generation of image data as described above. The second mode can relate to white light image acquisition and / or image acquisition without spectral information. The image acquisition unit can have separate optics and / or image acquisition sensors, between which it is possible to switch depending on the selected mode and / or which can be used simultaneously, or can have a common optics and / or a common image acquisition sensor.

[0085] An accurate, automatically generated assessment and / or recommendation can be obtained, in particular, if the assessment is additionally based on patient-specific data that are different from the image data. The patient-specific data can be obtained from a database. The patient-specific data can reflect relationships between patient data, such as age, gender, weight, medical history, etc., and expected spatially and spectrally detectable properties, e.g., an expected water content in a specific organ of a patient with a specific medical history, an expected vessel diameter for a specific vessel of a patient of a specific gender and age, etc., whereby these examples serve merely as illustrations.

[0086] In some embodiments, the image acquisition unit can be stereoscopic. The stereoscopic design of the image acquisition unit can relate to multispectral or hyperspectral image acquisition and / or image acquisition that only delivers spatially resolved image data. In the case of stereoscopic image acquisition, the spatial data can be three-dimensional. Specifically, it can be data that describes a surface in three-dimensional space. The image data can then be four-dimensional and include, for example, three spatial coordinates and one spectral coordinate. The invention further relates to a control device comprising a processor and a memory, wherein program code is stored on the memory, which, when executed by the processor, causes a method according to the invention to be carried out.

[0087] The invention further relates to program code which, when executed by a processor and / or a control device and / or a computer, is designed to effect the implementation of a method according to the invention.

[0088] Furthermore, the invention relates to a computer program product comprising a computer-readable medium and program code according to the invention stored on the computer-readable medium.

[0089] The device according to the invention and the method according to the invention are not intended to be limited to the application and embodiment described above. In particular, the device according to the invention and / or the method according to the invention may have a number of individual elements, components, units, and method steps that differs from the number stated herein to fulfill a functionality described herein. Furthermore, in the value ranges specified in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily.

[0090] It is particularly noted that all features and properties described with reference to the device, as well as procedures, are transferable to the method according to the invention and can be used within the meaning of the invention and are considered to be included in the disclosure. The same applies in reverse. This means that structural features mentioned with reference to the method, i.e., features related to the device, can also be considered, claimed, and included in the disclosure within the scope of the device claims.

[0091] The present invention is described below by way of example with reference to the attached figures. The drawings, the description and the claims contain numerous features in combination. A person skilled in the art will expediently also consider the features individually and use them sensibly in combination within the scope of the claims. If there is more than one example of a particular object, only one of them will be provided with a reference symbol in the figures and the description. The description of this example can be applied accordingly to the other examples of the object. If objects are named in particular using numerical words such as first, second, third object etc., these serve to name and / or assign objects. Accordingly, for example, a first object and a third object may be included, but not a second object.However, using number words, it might also be possible to derive a number and / or order of objects.

[0092] They show:

[0093] Fig. 1 is a schematic representation of a medical system with a medical imaging device;

[0094] Fig. 2 is a schematic structural diagram of the medical imaging device;

[0095] Fig. 3 is a schematic flow diagram of a method for operating the imaging device or a method for medical imaging;

[0096] Fig. 4 is a schematic representation of an imaged object area;

[0097] Fig. 5 a schematic representation of the object area with an overlay of detected tissue structures;

[0098] Fig. 6 a schematic representation of the object area with different overlays of recognized tissue structures;

[0099] Fig. 7 a schematic representation of the object area after performing a therapeutic action with different overlays;

[0100] Fig. 8 is a schematic representation of the object area in the event of smoke formation in the object area;

[0101] Fig. 9 is a schematic representation of the object area in the presence of contamination; Fig. 10 is a schematic representation of absorption curves of tissue with different degrees of coagulation;

[0102] Fig. 11 a schematic representation of the object area after carrying out a therapeutic action with various overlays;

[0103] Fig. 12 is a schematic flow diagram of a method for operating the medical imaging device;

[0104] Fig. 13 is a schematic flow diagram of a medical imaging method;

[0105] Fig. 14 a first alternative medical system;

[0106] Fig. 15 a second alternative medical system; and

[0107] Fig. 16 a third alternative medical system.

[0108] Fig. 1 shows a schematic representation of a medical system 56 with a medical imaging device 10 and a medical device 30. A schematic structural diagram of the imaging device 10 and the medical device 30 is shown in Fig. 2. Reference is made in part to both figures below.

[0109] In the exemplary case, the imaging device 10 is an endoscopic imaging device, specifically an endoscope device. Alternatively, the imaging device 10 could be an exoscopic, microscopic, or macroscopic imaging device. The medical imaging device is intended for examining a cavity.

[0110] In the illustrated case, the medical device 30 is a bipolar electrosurgical instrument. The medical device 30 is configured to specifically introduce energy into tissue in order to coagulate it, for example, for vascular occlusion. This embodiment is to be understood purely as an example. Other types of energy introduction may be provided, as well as other types of medical devices in general, such as surgical, diagnostic, imaging, interventional, anesthetic, or other medical instruments and / or devices.

[0111] The imaging device 10 comprises, for example, a medical imaging device 58. In the illustrated case, this is an endoscope. Furthermore, the medical imaging device 10 can include an illumination device 60. In the illustrated case, the illumination device 60 is connected to the imaging device 58 via a light guide. Illumination light can thus be guided to the imaging device 58 and directed from there onto an object to be imaged, in particular a site.

[0112] The imaging device 10 has, for example, a control unit 62. The control unit is connected to the imaging device 58 via a cable and / or an optical line and / or a light guide.

[0113] The imaging device 10 and in particular the imaging device 58 has, by way of example, one or more windows 52 through which illumination light can be coupled out and / or object light can be coupled in.

[0114] The imaging device 58 has a distal section 66. The distal section 66 is designed to be inserted into a cavity in an operating state. The distal section 66 faces a patient in the operating state. The distal section 66 faces away from a user in the operating state. Furthermore, the medical imaging device 58 has a proximal section 68. The proximal section 68 is arranged outside a cavity in the operating state. The proximal section 68 faces away from the patient in the operating state. The proximal section 68 faces the user in the operating state.

[0115] The imaging device 58 has a handle 70. The handle 70 is configured, by way of example, for handling by the user. Alternatively or additionally, the handle can be configured for attachment and / or connection to a medical robot. In some embodiments, the imaging device 58 can also be formed integrally with a robot. A position and / or orientation of the imaging device 58 relative to the patient can be changed, for example, by handling by the user and / or by suitable movement of the robot. The medical system 56 has a display device 72. The display device 72 can be part of the imaging apparatus 10. The display device 72 can be a separate display, such as a screen or the like. In other embodiments, the display device 72 can also be integrated into the imaging device 58 and / or the control device 62.

[0116] The imaging device 10 has a device interface 44, via which the medical device 30 can be connected to the imaging device 10. In the illustrated case, the device interface 44 is part of the control unit 62. The device interface 44 is wired in the illustrated case. The device interface 44 is detachable. Furthermore, the device interface 44 is configured for connection to various medical devices. The device interface 44 can comprise a plug-in socket and / or electrical connections. In some embodiments, the device interface 44 can also be partially or completely wireless, i.e., the medical device 30 can then be connected wirelessly, for example via a radio connection, to the imaging device 10, and the devices 10, 30 can have correspondingly suitable antennas.

[0117] The imaging device 10 has a user interface 46 through which the user can make inputs. The user interface 46 may include multiple control elements that may be attached to different components of the imaging device 10 and / or the medical system 56. In the illustrated case, the imaging includes

[0118] The imaging device 10 has a spatially and spectrally resolving image acquisition unit 12, which has at least one optical system 14. The image acquisition unit 12 further has an image acquisition sensor system 16 coupled to the optical system 14. The optical system 14 and the image acquisition sensor system 16 are configured to generate image data of an object region 18. The object region 18 is shown in Fig. 1 as an example on a display of the display device 72. The image data comprises both optical and spectral information. In this case, the image data correspond to two-dimensional spatial data that define spatial pixels, as well as spectral data that is assigned to the individual pixels. A spectrum is thus obtainable from the image data for each pixel. Furthermore, a two-dimensional image is obtainable from the image data for each spectral band. The image data correspond to a multispectral or hyperspectral data cube.The optics 14 comprises optical elements (not shown) that collect object light and guide it to the image capture sensor 16. The image capture sensor 16 comprises a CMOS or CCD sensor (not shown). The optics 14 and the image capture sensor 16 are arranged together in a pushbroom arrangement. In other embodiments, a whiskbroom arrangement, a staring arrangement and / or a snapshot arrangement is used. In the present case, the image capture unit 12 is configured for hyperspectral image capture, and the imaging device 10 is accordingly a hyperspectral imaging device. With regard to different methods of hyperspectral imaging and the components required for this, reference is made to the specialist article "Review of spectral imaging technology in biomedical engineering: achievements and challenges" by Quingli Li et al.Published in Journal of Biomedical Optics 18(10), 100901, October 2013, and reference is made to the article "Medical hyperspectral imaging: a review" by Guolan Lu and Baowei Fei, published in Journal of Biomedical Optics 19(1), 010901, January 2014. In other embodiments, the imaging device 10 can also be multispectral. Multiple spectral ranges can be viewed, for example, through filters that can be selectively inserted into an object light beam path and / or through sequential illumination with different wavelengths.

[0119] The image capture unit 12 is at least partially contained in the imaging device 58. Parts of the optics 14 and / or the image capture sensor system 16 can be contained in the control unit 62. For example, object light can be guided to the image capture sensor system 16 via a fiber optic cable, and the latter can be arranged in the control unit 62. In other embodiments, the entire image capture sensor system 16 is contained in the imaging device 58, and only data is transmitted to the control unit 62.

[0120] The imaging device 10 further comprises an image analysis unit 20. This is configured to perform an analysis of the image data. The analysis is based on both spatial and spectral information. The analysis comprises at least one evaluation, which includes at least one evaluation parameter relating to an object sub-area 22 of the object area 18. This will be discussed in more detail below.

[0121] The imaging device 10 also includes an assessment unit 24. This is configured to generate an assessment of an attribute of the object subregion 22 relevant to the diagnostic and / or therapeutic action, based on the analysis and on information relating to a diagnostic and / or therapeutic action to be performed and / or performed. This will be discussed in more detail below.

[0122] The imaging device 10 further includes an output generation unit 26 configured to generate an output based on the assessment.

[0123] In the exemplary case, the imaging device 10 comprises an output unit 74. The output unit 74 can comprise the display device 72 and / or other output devices. In the illustrated case, the output unit 74 comprises, in addition to the display device 72, a loudspeaker 76 and an additional display 78, which are formed, for example, on the control unit 62. In other embodiments, the output unit 74 has an interface for connecting one or more output devices. The output generation unit 26 can also comprise the output unit 74 or its components.

[0124] The imaging device 10 comprises a control unit 11. This comprises a processor, a main memory, a storage, and correspondingly configured circuits. Program code is stored in the memory, which, when executed by the processor, causes the methods described herein to be carried out or implements functionalities of the described units.

[0125] In the present exemplary embodiment, the imaging device 10 is multimodal. It can be operated in both a white-light mode and a multispectral or hyperspectral mode. In white-light mode, only a white-light image containing no spectral information is acquired by the image acquisition unit 12. In multispectral or hyperspectral mode, however, the described spatially and spectrally resolved image data are generated. In the present case, the image acquisition unit 12 is configured to use both modes simultaneously using multiple optics and image acquisition sensors. Image acquisition with spectral information often takes more time than pure white-light imaging or imaging without spectral information. Therefore, in the present case, it is provided that images from the two modes can be combined.Spatial information based on white-light images is therefore subject to a higher frame rate or acquisition rate than spatially and spectrally resolved image data. The former can be generated and / or output in real time, the latter essentially in real time. However, it is understood that the functions, methods, and method steps described below can also be based solely on the spatially and spectrally resolved image data acquired using multispectral and / or hyperspectral image acquisition.

[0126] Depending on the application, it may be advantageous to combine hyperspectral imaging with parallel and / or alternating white-light imaging, or to use multispectral imaging and, if necessary, to use this without white-light imaging. Multispectral imaging often allows for a higher frame rate, meaning that real-time white-light imaging may be unnecessary. Hyperspectral imaging is often characterized by a lower frame rate but can deliver higher spectral resolution. In this case, a combination with white-light imaging is recommended to still have real-time image data available. Multispectral image data can also be used alternatively or additionally to monitor specific parameters or image areas in the background, possibly without displaying a corresponding image to the user.Especially in this case, a combination with white light imaging can also be advantageous for multispectral imaging.

[0127] An exemplary application of the medical system 56 is described below using the example of a vessel sealing procedure to be performed. However, it is understood that appropriate adaptations can be made for other diagnostic and / or therapeutic actions and / or one or more other medical devices 30 can be used. Reference is repeatedly made to Fig. 3, which shows a schematic flow diagram of a method for operating the imaging device 10 or the medical system 56. Fig. 3 also illustrates, in an analogous manner, a method for medical imaging, performed using the imaging device 10 or the medical system 56.

[0128] Fig. 4 shows a schematic representation of an imaged object region 18. The object region 18 is being observed during a microinvasive procedure. For this purpose, the distal section 66 of the imaging device 58 is inserted into a cavity. Located in the cavity are various native structures 80, 82 as well as a vessel 84, for example a blood vessel, which is to be sealed. The sealing is to be achieved by means of the medical device 30 through tissue coagulation at a suitable location. In a step S1, spatially and spectrally resolved image data of the object region 18 are generated. For this purpose, the object region 18 is illuminated, and object light coming from the object region 18 is then detected. In this case, the detected spectral information relates to light absorption. An absorption spectrum can be obtained for each pixel accordingly.

[0129] In a step S2, the image data is analyzed by the image analysis unit 20. The analysis comprises image segmentation and a comparison of spectral information with information stored in a database 32 and relating to properties of various structures, tissue entities, etc. Regions identified during image segmentation can be analyzed for their spectral properties. The database 32 stores, for example, which tissue entities are typically characterized by which absorption spectra and / or exhibit certain absorption values ​​for certain spectral bands, in particular relative to other spectral bands. As part of the analysis, an evaluation is created that includes at least one evaluation parameter. The evaluation parameter can, for example, indicate which tissue entity is involved in the corresponding image segment. As part of the analysis, the medical device 30 is also identified.

[0130] In an optional step S3, an overlay 54 is generated based on the evaluation, which can be displayed to the user. As illustrated in Fig. 5, the various detected tissue entities are highlighted differently, for example, by displaying a colored overlay, a texture, a label, or a combination. This makes it easy for the user to understand which subregions, instruments, organs, vessels, nerves, fascia, etc. are located in the object area 18.

[0131] Steps S2 and S3 can be performed repeatedly. As mentioned, a real-time white-light image can be used as the basis. The displayed highlights of different tissue structures, based on spectral information, can be updated essentially in real time.

[0132] In this case, the detection of image regions involves tracking using a suitable tracking algorithm. The corresponding highlighting can also be tracked, even if no updated spectral image has yet been recorded and / or analyzed. For example, if the vessel 84 is detected and highlighted as such, but then moves within the object area and / or relative to the imaging device 58, the highlighting can also be tracked using a tracking method based on a real-time white light image. If a new spectral image is available, its analysis can be used to check whether the current highlighting is still correct or needs to be adjusted.

[0133] In some embodiments, recognition is also based on 3D data, which is obtained, for example, from database 32 and / or acquired using an additional detection step. If, for example, the imaging device 58 is attached to a robot, the robot can first move across the object area in a targeted manner to capture multiple 2D images from which a 3D model can be generated. The 3D data can be compared with the current image data, allowing structures to be recognized even if they are only partially visible and / or if they are temporarily obscured due to a changed viewing angle or due to movements in the object area 18.

[0134] In a step S4, an assessment is generated by the assessment unit 24. This assessment is based on the evaluation parameter and additional information, also referred to herein as assessment information. The assessment information can be obtained from the database 32 and / or stored in the imaging device 10. In this case, the assessment information includes details about which tissue entities are suitable and unsuitable for the planned therapeutic action. The assessment information can be based on empirical values ​​​​of various physicians, clinicians, scientists, etc., as well as on various studies, medical findings, or the like.

[0135] The imaging device 10 comprises a recommendation unit 28. The recommendation unit 28 can be part of the assessment unit 24. The recommendation unit 28 creates a recommendation for the user. This can occur in a step S5. The recommendation unit 28 determines, on the basis of the assessment and using information from the database 32 and / or a recommendation database 40, that an object subregion 22 is particularly well suited for the planned therapeutic action. The recommendation database 40 can be integrated into the database 32 or formed separately from it. The database 32 and / or the recommendation database 40 contains information about characteristics of tissue entities, patient data, medical devices, diagnostic methods, therapeutic methods, etc., based on which the attributes determined during the assessment can be classified and / or sorted.For example, the attributes assigned to the different image regions can be sorted according to their suitability or unsuitability for the therapeutic action to be performed. Based on this, the recommendation unit 28 can determine an object sub-region 22. At this location, vessel sealing is likely to be successfully performed. The recommendation unit 28 thus recommends the object sub-region 22 as the target location for tissue coagulation. Furthermore, the recommendation unit 28 classifies the structures 80, 82 as risk structures 34, 36 because an inadvertent use of the medical device 30 in the area of ​​these structures 80, 82 would be particularly dangerous.

[0136] Based on the assessment and the recommendation, the output generation unit 26 generates an output in the form of further overlays 54, as shown by way of example in Fig. 6. These include, for example, a highlighting of the object sub-area 22 to identify it as the preferred region for treatment, as well as coloring, markings, or the like to identify the risk structures 34, 36.

[0137] In other embodiments, the recommendation may also include advising the user to use or not use a specific medical device. For example, a user may select which treatment outcome is desired before a medical device is connected and / or inserted. Based on the recommendation, the user may then select the appropriate medical device from among several possible medical devices. This selection may, in particular, also include adapting a medical device by attaching a specific instrument tip or attachment, or the like.

[0138] In the present case, it is additionally provided that the output includes a warning directed at the user. For this purpose, for example, a display 54 in the form of a traffic light symbol is used, which turns red if the user is about to act on a risk structure 34, 36. As long as the user moves the medical device 30 in non-critical areas and / or towards the object sub-area 22, the traffic light symbol is displayed in green. Additionally or alternatively, a warning tone or another audio signal can also be output by the output unit 74, in particular by the loudspeaker 76, if incorrect treatment is imminent. The warning can also relate to processes during the implementation of the diagnostic and / or therapeutic action. For example, a warning can be output if the user grasps tissue in the wrong place and / or with an inappropriate gripping force.

[0139] The imaging device 10 includes an evaluation unit 38. This determines a rating of a generated recommendation based on observed user behavior. For example, if the object area 22 is recommended for treatment and the user follows the recommendation, the recommendation is rated as good. If, however, the user selects a different area for treatment, the recommendation is rated as poor. The rating can be based on additional parameters, such as the user's experience and / or treatment success. A data set associated with the recommendation can then be transmitted to the recommendation database 40 and used for future recommendations. A learning system is thus provided.

[0140] The imaging device 10 further comprises a parameter determination unit 42. The parameter determination unit 42 is configured to determine at least one control parameter for controlling the medical device 30 based on the assessment of the attribute of the object subregion. The output then includes this control parameter. The control parameter can be transmitted to the medical device 30 via the device interface 44.

[0141] The parameter determination unit 42 determines suitable parameters for the object sub-area 22 for controlling the medical device 30. This can be done in a step S5. On the basis of the spectral and spatial image data available for the object sub-area 22, as well as taking into account empirical values, manufacturer information, etc., which are stored in the database 32 and / or the recommendation database 40, the parameter determination unit 42 can determine process-relevant parameters that promise a successful treatment. The process and its effectiveness can thus be optimally adapted to the tissue to be treated. The process-relevant parameters include, among others, the specified size and temporal course (mode) of the introduced energy via the applied voltage and / or the type of process control and / or via measured process parameters such asPower, voltage, current, resistance, phase shift, tissue shrinkage (thickness), temperature, gripping forces (tissue compression), etc. The output generation unit 26 is configured to generate an output of the parameters determined to be suitable for the user. These can be displayed, for example, as an overlay 54 in a box and / or as a list. The user can confirm the acceptance of the parameters by entering them or adjust them as needed. The corresponding control parameters are then automatically transmitted to the medical device 30 via the device interface 44.

[0142] In automatic mode, control parameters can be adjusted even without user acknowledgement. If the user activates automatic mode, the control parameters of the medical device 30 are automatically adjusted during operation.

[0143] In the present case, the parameter determination unit 42 also determines at least one control parameter that defines an activation state of the medical device 30. A two-stage safety concept can thus be implemented. In addition to warnings issued to the user, an automatic shutdown of the medical device 30 can occur based on such a control parameter, for example, if gross misuse occurs, an image signal suddenly interrupts, the object area shifts significantly, heavy bleeding is detected, or the like. An emergency shutdown can then occur even without user intervention to prevent further consequences for the patient.

[0144] Both the suggestion of parameters to the user and automatically transmitted parameters can be adjusted essentially in real time when this is indicated based on the assessment of updated image data.

[0145] After the user has performed the therapeutic action in the object sub-region 22, in particular after selecting and / or adjusting the suggested parameters, another image can be recorded in a step S6. The actual performance of the diagnostic and / or therapeutic action is not a component of the methods described herein. As shown in Fig. 7, the tissue in the object sub-region 22 has changed due to the performed therapeutic action. This will be discussed further below. In an optional step S7, the quality of the obtained image data can be assessed. In the present case, the imaging device 10 comprises a smoke detection unit 48 and a contamination detection unit 50. These are configured to detect smoke and contamination, respectively. As shown in Fig.As shown in Figure 8, smoke, symbolized by the cloud shown, may have been generated in the subject area due to the diagnostic and / or therapeutic procedure performed, for example, due to vaporized and / or burned tissue. This smoke prevents the acquisition of high-quality images.

[0146] Furthermore, contamination on one or all of the windows 52 can lead to reduced image quality. As schematically illustrated in Fig. 9, this can result in image artifacts that can impair the quality of the image data.

[0147] The smoke detection unit 48 is configured to detect the presence of smoke based on the image data. For this purpose, the smoke detection unit 48 uses a KI algorithm, such as binary classification with ResNet 50. If smoke is detected, a corresponding output can be sent to the user. Furthermore, the smoke detection unit 48 is configured to generate a waiting time. After the waiting time has elapsed, it can be expected that the smoke will have dissipated and that image data of satisfactory quality can be generated again. In other embodiments, the smoke detection unit 48 can also examine all recorded image data and release each one, or flag it as possibly containing smoke, or not release it.

[0148] Contamination detection unit 50 operates in a similar manner. If contaminants are detected that could significantly impair image capture, the user can be informed. In some embodiments, imaging device 10 can also have a cleaning mechanism that can automatically clean windows 52 as needed. In some embodiments, parameters for cleaning can be determined by parameter determination unit 42 and passed on to the cleaning mechanism automatically and / or upon user approval. The cleaning mechanism can be a medical device within the meaning of this disclosure. It is understood that the detection of smoke and / or contaminants can be performed at any time, for example, at regular intervals and / or image capture intervals, upon user request, before each image capture, etc.

[0149] In step S8, the image data of the treated object area are analyzed. In this case, absorption values ​​in different spectral ranges obtained for pixels located in the object sub-area 22 are used as evaluation parameters.

[0150] Fig. 10 shows schematic absorption curves in the visible and near-infrared range of tissue coagulated with different parameters. The inventors have found that the absorption curves for less and more coagulated tissue behave differently in different spectral ranges. In a short-wave range, which is indicated by line a1 in Fig. 10, the absorption decreases with increasing coagulation. In a long-wave range, which is indicated by line a2 in Fig. 10, the absorption increases with increasing coagulation. The spectra in Fig. 10 are normalized, in this case by division by the respective absorption value at line n. As described below, this can be used to analyze and assess whether a coagulation treatment was completed with sufficient quality.

[0151] The analysis is based on a before-and-after comparison, with image data from before the diagnostic and / or therapeutic procedure serving as a reference. The before-and-after comparison involves calculating the difference, which indicates the extent to which absorption has changed in the spectral ranges under consideration. To compensate for effects attributable to varying amounts of tissue in the subject area or other deviations, the spectra to be compared are normalized, as illustrated in Fig. 10. In the present case, for example, the differences in lines a1 and a2 are determined as evaluation parameters.

[0152] Database 32 stores the ratios, differences, and / or other relationships between these evaluation parameters that can provide information about the quality of coagulation. The assessment unit 24 uses this information as assessment information and, based on it, evaluates the evaluation parameters obtained from the analysis. In the present case, this assessment determines a binary attribute that can assume the values ​​"coagulation complete" and "coagulation incomplete." As mentioned above, the attribute can also refer to a grading scale, for example, and / or assume continuous values.

[0153] Based on the attribute, the output generation unit 26 generates an overlay 54 as output, from which the user can determine the attribute and thus the quality of the coagulation. This is illustrated in Fig. 11 by the overlay of the word "OK!". In other embodiments, the object region 22 can also be colored, for example, green for good quality and red for poor quality.

[0154] In the described case, the spectra of all pixels in the object sub-area 22 can be averaged to perform the analysis. In some embodiments, a similar attribute can also be determined point-by-point, for example, for each pixel of the object area 22. The output can then include a corresponding point-by-point coloring, from which the user can possibly determine where the coagulation was successful or inadequate.

[0155] It goes without saying that in addition to automated detection of the object area, the user can also select and / or specify the object area. For example, they can highlight areas for which they would like to have the degree of coagulation assessed.

[0156] If necessary, further diagnostic and / or therapeutic intervention can be planned and / or performed, for example, to achieve the desired degree of coagulation through follow-up treatment. A further assessment can then be performed.

[0157] Aspects of the above description can also be summarized or described as follows. Reference is first made to Fig. 12. This shows a schematic flow diagram of a method for operating the medical imaging device. A step S11 comprises an at least partially automated generation / acquisition of image data. A step S12 comprises an at least partially automated creation of an analysis of the image data based on spatial and spectral information, wherein the analysis comprises at least one evaluation comprising at least one evaluation parameter relating to an object sub-region 22 of an object region 18.A step S13 comprises an at least partially automated generation, in accordance with the analysis and based on information regarding a diagnostic and / or therapeutic action to be performed and / or performed, of an assessment of an attribute of the object sub-area 22 that is relevant to the diagnostic and / or therapeutic action, which assessment is based on the evaluation parameter. A step S14 comprises an at least partially automated output of an output based on the assessment.

[0158] Reference is made below to Fig. 13. This shows a schematic flow diagram of a method for medical imaging. A step S21 comprises an at least partially automated generation / acquisition of image data. A step S22 comprises an at least partially automated creation of an analysis of the image data based on spatial and spectral information, wherein the analysis comprises at least one evaluation comprising at least one evaluation parameter relating to an object sub-region 22 of an object region 18. A step S23 comprises an at least partially automated creation, in accordance with the analysis and on the basis of information relating to a diagnostic and / or therapeutic action to be performed and / or performed, of an assessment of an attribute of the object sub-region 22 relevant to the diagnostic and / or therapeutic action, which is based on the evaluation parameter.A step S24 comprises an at least partially automated output of an output based on the assessment.

[0159] Fig. 14 shows a first alternative medical system 56'. Like reference numerals denote corresponding objects, with the reference numerals in Fig. 14 being provided with inverted commas for differentiation. The medical system 56' comprises a medical imaging device 10', which is fundamentally constructed like the imaging device 10 described above. Therefore, the individual components are not identified by reference numerals.

[0160] The medical system 56' comprises a medical device 30'. This is connected and / or connectable to the imaging device 10'. The medical device 30' is an insufflator. The medical device 30' is configured to perform gas humidification of tissue during a procedure. The imaging device 10' comprises a parameter determination unit configured to determine control parameters for the medical device 30'. These can be forwarded to the medical device 30' as described above, in particular automatically. In the present case, the control parameters include a type and / or composition of the gas used, typically carbon dioxide, a degree of humidification of the gas, a gas flow, a pressure, a humidification interval, etc.Using the imaging device 10', it can be assessed whether a current setting of the medical device 30' is suitable for moistening the tissue or whether the setting needs to be changed. For this purpose, evaluation parameters and attributes can be obtained from the obtained spatially and spectrally resolved image data, similar to those described above, which indicate the degree of moistening of the tissue. In this case, the water content of the tissue is determined based on the obtained spectra. This value is compared with stored values ​​in a database.

[0161] Fig. 15 shows a second alternative medical system 56". Like reference numerals denote corresponding objects, whereby the reference numerals in Fig. 15 are each provided with two apostrophes for differentiation. The medical system 56" comprises a medical imaging device 10", which is fundamentally constructed like the imaging device 10 described above. Therefore, the identification of its individual components with reference numerals is omitted.

[0162] The medical system 56" comprises a medical device 30". This is connected and / or connectable to the imaging device 10". The medical device 30" is a suturing device, in the illustrated case a linear stapler. The medical device 30' is configured to create a suture during a procedure, to connect tissue with staples, or the like. The imaging device 10' comprises a parameter determination unit configured to determine control parameters for the medical device 30'. These can be forwarded to the medical device 30' as described above, in particular automatically. In the present case, the control parameters include, for example, a control of the closing forces of the staples depending on a determined tissue condition.Furthermore, the imaging device 10' can be used to assess whether staples were applied in the correct location and / or in the correct number, or where additional staples need to be applied to create a suture with the desired properties. Patient-specific data, such as the patient's gender or age, can be taken into account. This allows for an estimate of the expected healing behavior of the created suture.

[0163] Fig. 16 shows a third alternative medical system 56'". Like reference numerals denote corresponding objects, whereby the reference numerals in Fig. 16 are each provided with three apostrophes for differentiation. The medical system 56'" comprises a medical imaging device 10'", which is fundamentally constructed like the imaging device 10 described above. Therefore, the identification of its individual components with reference numerals is omitted.

[0164] The medical system 56'" is part of a surgical robot 86'" or is attached to such a robot. For example, an imaging device of the imaging apparatus 10'" and a medical device 30'" are each attached to a manipulator 88'", 90'". These can thus be moved and rotated, i.e. their position and / or orientation relative to the site can be changed automatically. In particular when using such a robot 86'", the position and / or orientation of the imaging apparatus 10'" or its imaging device and / or the medical device can also be determined using the sensors of the robot 86'". This can supplement image-based tracking or replace it. The robot 86'" can also be part of the medical system 56'".

[0165] As already mentioned above, the procedures described herein can also be performed partially or completely virtually, for example as part of a training and / or when using a virtual surgery planner.

[0166] List of reference symbols

[0167] 10 Medical imaging device

[0168] 11 Control unit

[0169] 12 Image acquisition unit

[0170] 14 Optics

[0171] 16 image capture sensors

[0172] 18 Object area

[0173] 20 Image analysis unit

[0174] 22 Object sub-area

[0175] 24 assessment unit

[0176] 26 Output generation unit

[0177] 28 recommendation unit

[0178] 30 Medical Device

[0179] 32 database

[0180] 34 Risk structure

[0181] 36 Risk structure

[0182] 38 Valuation unit

[0183] 40 recommendation database

[0184] 42 Parameter determination unit

[0185] 44 Device interface

[0186] 46 User interface

[0187] 48 Smoke detection unit

[0188] 50 Contamination detection unit

[0189] 52 windows

[0190] 54 Fade-in

[0191] 56 Medical System

[0192] 58 Medical imaging device

[0193] 60 lighting device

[0194] 62 Control unit

[0195] 66 Distal section

[0196] 68 Proximal section

[0197] 70 handle

[0198] 72 display device

[0199] 74 Output unit

[0200] 76 speakers

[0201] 78 Display

[0202] 80 Structure Structure Vessel Surgical Robot

Claims

Claims 1. Medical imaging device (10), in particular endoscopic imaging device, comprising: a spatially and spectrally resolving image acquisition unit (12) comprising at least one optic (14) and at least one image acquisition sensor (16) coupled to the optic (14), which are configured to generate image data of an object area (18) comprising spatial and spectral information; an image analysis unit (20) which is configured to perform an analysis of the image data based on spatial and spectral information, wherein the analysis comprises at least one evaluation comprising at least one evaluation parameter relating to an object sub-area (22) of the object area (18);an evaluation unit (24) configured to generate, according to the analysis and on the basis of information regarding a diagnostic and / or therapeutic action to be performed and / or carried out, an evaluation of an attribute of the object sub-area (22) relevant to the diagnostic and / or therapeutic action, based on the evaluation parameter; and an output generation unit (26) configured to generate an output based on the evaluation.

2. Medical imaging device (10) according to claim 1, further comprising a recommendation unit (28) configured to generate a recommendation for a user based on the assessment of the attribute of the object sub-area (22), wherein the output generation unit (26) is configured to generate the output based on the recommendation.

3. Medical imaging device (10) according to claim 2, wherein the recommendation is a recommendation for action for carrying out a diagnostic and / or therapeutic action on a patient.

4. Medical imaging device (10) according to claim 2 or 3, wherein the recommendation relates to the selection and / or the setting and / or the handling of a medical device (30) different from the imaging device (10).

5. Medical imaging device (10) according to one of claims 2 to 4, wherein the recommendation relates to whether and / or in which area surgical intervention should be performed on the object area (18), and / or wherein the recommendation relates to whether and / or in which area surgical intervention should not be performed on the object area (18).

6. Medical imaging device (10) according to one of claims 2 to 5, wherein the recommendation is based on a comparison of the assessment with a database (32) which contains data records on different diagnostic and / or therapeutic actions to be performed and / or have been performed, as well as associated recommendations.

7. Medical imaging device (10) according to any one of claims 2 to 6, wherein the assessment comprises the detection and / or classification of risk structures (34, 36), and wherein the recommendation comprises information regarding the detection and / or classification of risk structures (34, 36).

8. Medical imaging device (10) according to one of claims 2 to 7, further comprising an evaluation unit (38) which is configured to assign at least one evaluation to the recommendation depending on a behavior of the user in response to a behavior prompted by the recommendation.

9. Medical imaging device (10) according to claim 8, wherein the evaluation unit (38) is configured to transmit the evaluation to a recommendation database (40).

10. Medical imaging device (10) according to any one of claims 2 to 9, wherein the image acquisition unit (12) is configured to generate continuously updated image data, and wherein the recommendation unit (28) is configured to adjust the recommendation based on continuously updated image data essentially in real time.

11. Medical imaging device (10) according to any one of the preceding claims, further comprising a parameter determination unit (42) configured to determine at least one control parameter for controlling a medical device (30) based on the assessment of the attribute of the object sub-area (22), wherein the output generation unit (26) is configured to generate the output based on the at least one control parameter. Medical imaging device (10) according to claim 11, further comprising a device interface (44) connectable to a medical device (30) and configured to transmit the at least one control parameter to the medical device (30). Medical imaging device (10) according to claim 11 or 12, wherein the output comprises a suggestion for a user to automatically transmit the at least one control parameter to a medical device (30) that can be controlled by means of the control parameter.A medical imaging device (10) according to claim 13, further comprising a user interface (46) configured to receive at least one user input by means of which the user can accept and / or reject and / or modify the suggestion. A medical imaging device (10) according to any one of claims 11 to 14, wherein the image acquisition unit (12) is configured to generate continuously updated image data, and wherein the parameter determination unit (42) is configured to adjust the at least one control parameter in substantially real time as a function of continuously updated image data. A medical imaging device (10) according to any one of claims 11 to 15, wherein the at least one control parameter is configured to define an operating mode of the medical device (30).Medical imaging device according to one of claims 11 to 16, wherein the at least one control parameter is configured to effect a safety-relevant shutdown of the medical device (30). Medical imaging device (10) according to any one of the preceding claims, wherein the assessment comprises an evaluation of the quality of a diagnostic examination and / or a treatment outcome, and wherein the output comprises information regarding the quality of the diagnostic examination and / or the treatment outcome. Medical imaging device (10) according to any one of the preceding claims, wherein the assessment is based on at least two different sets of image data and comprises a comparison of the attribute of the object sub-area at two different time points. Medical imaging device (10) according to any one of the preceding claims, wherein the analysis of the image data comprises image segmentation. Medical imaging device (10) according to any one of the preceding claims, wherein the analysis of the image data comprises tissue recognition.A medical imaging device according to any one of the preceding claims, wherein the analysis of the image data comprises the detection of at least one medical device (30). A medical imaging device (10) according to claim 22, wherein the detection of the at least one medical device (30) comprises information about a type and / or orientation and / or position and / or state of the medical device (30). A medical imaging device (10) according to any one of the preceding claims, wherein the attribute comprises information about anatomical and / or physiological features of the object sub-area (22). A medical imaging device (10) according to any of the preceding claims, further comprising a smoke detection unit (48) and / or a contamination detection unit (50) configured to detect the presence and / or absence of smoke and / or contamination in the object area (18) and / or on a window (52). A medical imaging device (10) according to claim 25, wherein the evaluation unit is configured to take into account the presence and / or absence of smoke and / or contamination detected by the smoke detection unit (48) and / or the contamination detection unit (50) in the object area (18) and / or on the window (52) when evaluating the attribute.A medical imaging device (10) according to claim 25 or 26, wherein the image acquisition unit (12) is configured to generate new image data of the object area (18) at a time after the image data of the object area (18) has been generated, once the smoke detection unit (48) has detected the presence of smoke in the object area (18) based on the image data of the object area (18). A medical imaging device (10) according to any one of the preceding claims, wherein the output comprises at least one overlay (54) directed at a user, which is superimposed on a pictorial representation of the object area (18). A medical imaging device (10) according to claim 28, wherein the overlay (54) comprises highlighting and / or coloring at least areas of a pictorial representation of the object area (18).Medical imaging device (10) according to one of the preceding claims, wherein the output comprises at least one audio output directed to a user.

31. Medical imaging device (10) according to any of the preceding claims, wherein the output relates to a classification of tissue.

32. Medical imaging device (10) according to one of the preceding claims, wherein the output comprises a suitability and / or unsuitability of at least one object sub-area (22) of the object area (18) for the diagnostic and / or therapeutic action to be performed.

33. Medical imaging device (10) according to one of the preceding claims, wherein the output relates to the quality of a result of the diagnostic and / or therapeutic action performed.

34. Medical imaging device (10) according to any of the preceding claims, wherein the output comprises a warning directed to a user.

35. Medical imaging device (10) according to one of the preceding claims, wherein the analysis comprises a comparison of normalized and / or standardized spectral data.

36. Medical imaging device (10) according to one of the preceding claims, wherein the analysis comprises a comparison of spectral data for different spectral ranges.

37. Medical imaging device (10) according to one of the preceding claims, wherein the optics (14) and the image acquisition sensor (16) of the image acquisition unit (12) are configured to acquire hyperspectral image data.

38. Medical imaging device (10) according to one of the preceding claims, wherein the spectral information relates to the absorption of light. Medical imaging device according to any one of the preceding claims, wherein the analysis comprises assigning the image data to three-dimensional data relating to the object area (18). Medical imaging device (10) according to any one of the preceding claims, wherein the diagnostic and / or therapeutic action to be performed and / or performed comprises the input of energy into tissue, in particular thermal and / or electrosurgical and / or radiation-based energy input. Medical imaging device (10) according to any one of the preceding claims, wherein the evaluation parameter contains information regarding tissue coagulation, in particular regarding vessel sealing, tube sterilization, and / or the welding and / or joining of tissue structures such as, for example, anastomosis formation.A medical imaging device (10) according to any of the preceding claims, wherein the image acquisition unit (12) is multimodal and operable in at least a first mode in which spatially and spectrally resolved image data can be generated, and in at least a second mode in which only spatially resolved image data can be generated. A medical imaging device (10) according to any of the preceding claims, wherein the diagnostic and / or therapeutic action to be performed and / or performed is real and / or virtual. A medical imaging device (10) according to any of the preceding claims, wherein the assessment is additionally based on patient-specific data that differs from the image data.

45. Medical imaging device (10) according to one of the preceding claims, wherein the image acquisition unit (12) is stereoscopic.

46. ​​Medical system (56) comprising: a medical imaging device (10) according to any of the preceding claims; and a medical device (30).

47. Medical system (56) according to claim 46, wherein the medical imaging device (10) comprises a device interface (44), and wherein the medical device (30) is connectable and / or connected to the device interface (44) of the imaging device (10).

48. Medical system (56) according to claim 46 or 47, wherein the medical device (30) is a tissue-coagulating, in particular vessel-sealing or tissue-welding, instrument.

49. Method for operating a medical imaging device (10), in particular a medical imaging device (10) according to any one of claims 1 to 45, wherein the medical imaging device (10) comprises a spatially and spectrally resolving image acquisition unit (12) comprising at least one optic (14) and at least one image acquisition sensor (16) coupled to the optic (14), which are configured to generate image data of an object area (18) comprising spatial and spectral information, comprising the steps: - at least semi-automated creation of an analysis of the image data based on spatial and spectral information, wherein the analysis includes at least one evaluation which includes at least one evaluation parameter relating to an object sub-area (22) of the object area (18); - at least semi-automated generation, according to the analysis and based on information regarding a diagnostic and / or therapeutic action to be carried out and / or carried out, an assessment of an attribute of the object sub-area relevant to the diagnostic and / or therapeutic action (22), which is based on the evaluation parameter; and - at least semi-automated output of a result based on the assessment. Method for medical imaging, in particular carried out with a medical imaging device (10) according to any one of claims 1 to 45 and / or with a medical system (56) according to any one of claims 46 to 48, comprising the steps: - at least semi-automated generation of image data of an object area (18) that includes spatial and spectral information; - at least semi-automated creation of an analysis of the image data based on spatial and spectral information, wherein the analysis includes at least one evaluation which includes at least one evaluation parameter relating to an object sub-area (22) of the object area (18); - at least semi-automated generation, according to the analysis and based on information regarding a diagnostic and / or therapeutic action to be carried out and / or carried out, an assessment of an attribute of the object sub-area relevant to the diagnostic and / or therapeutic action (22), which is based on the Evaluation parameters are based on; and - at least semi-automated output based on the assessment. Control device comprising a processor and a memory, wherein program code is stored in the memory which, when executed by the processor, causes the execution of a method according to claim 49 or 50. Program code configured to cause the execution of a method according to claim 49 or 50 when executed by a processor and / or a control device and / or a computer. Computer program product comprising a computer-readable medium and program code stored on the computer-readable medium according to claim 52.