Endoscope device, endoscope and medical imaging system

Abstract

The invention relates to an endoscope device (10) with a shaft (12) having a proximal and a distal section (14, 16). The endoscope device (10) further comprises a hyperspectral image acquisition assembly (18) configured to acquire images of an object area (20) and to generate hyperspectral image data that includes spatial and spectral information. The image acquisition assembly (18) is arranged in the distal section (16) of the shaft (12). Furthermore, the invention relates to an endoscope (70) and a medical system (72).

Description

[0001] The invention relates to an endoscope device, an endoscope and a medical system for medical imaging.

[0002] Endoscopic devices that generate multispectral or hyperspectral images are known from the prior art. Multispectral or hyperspectral images have, in addition to two spatial dimensions, such as a conventional camera image, a spectral dimension. The spectral dimension encompasses several spectral bands (wavelength bands). Multispectral and hyperspectral images differ primarily in the number and / or width of their spectral bands.

[0003] Several imaging devices for generating such multispectral or hyperspectral images are known, particularly in the context of medical applications. For example, DE 20 2014 010 558 U1 describes a device for capturing a hyperspectral image of an examination region of an object. The device includes an input lens for generating an image in an image plane and a slit-shaped aperture in the image plane for selecting a slit-shaped region of the image. The light passing through the aperture is spread out by a dispersive element and recorded by an image sensor. As a result, the image sensor can record a plurality of spectra, each with an associated spatial coordinate, along the longitudinal direction of the slit-shaped aperture.The described device is further configured to record further spectra along the longitudinal direction of the slit-shaped aperture in a direction different from the longitudinal direction of the slit-shaped aperture. Either by successively displacing the object to be examined or by a step-by-step movement of the recording device over the body to be examined, a translational relative movement of the examination area of ​​a body with respect to the camera sensor is generated. In this way, successive slit-shaped sections of the examination area can be recorded with spectra and spatial coordinates and combined into an overall image by means of a data processing device. The method underlying this disclosure for generating multispectral or hyperspectral images is also known as the so-called pushbroom method.

[0004] In addition to the pushbroom method, other technologies for generating multispectral and / or hyperspectral image data are known. For example, in the so-called whiskbroom approach, the object to be imaged is scanned point by point, obtaining a spectrum for each point. It is also possible to acquire multiple images sequentially through different spectral filters using a so-called staring approach. Each individual image captured contains complete spatial information for a single spectral range. Another technology for acquiring multispectral and / or hyperspectral image data is the snapshot method. Using suitable optical elements such as optical slicers, lenses, and prisms, a two-dimensional multicolor image is decomposed into several individual spectral images and / or image sub-areas, which are simultaneously acquired on different detectors or detector areas.This method allows spectral data for an entire image field to be acquired simultaneously in a single acquisition step.

[0005] Imaging techniques are of central importance in the medical field, particularly in connection with minimally invasive surgical procedures, but also generally for diagnostics and for assessing the success or quality of a procedure. For example, a site can be observed during a minimally invasive procedure when tissue is to be welded or has been welded, for example, using an electrosurgical device such as a high-frequency coagulation instrument. Other examples of diagnostic and / or therapeutic actions before, during, or after which observation using imaging techniques is appropriate include the observation and assessment of ligatures, clips, and stapler insertions, tissue sampling, or various surgical procedures in general.For most surgical procedures, it is advantageous to generate spectral image data in real time so that it can be used, for example, for monitoring and / or assessing a diagnostic and / or therapeutic intervention. This includes, for example, generating a spectrally resolved image in less than one second or even multiple times per second.

[0006] The previously mentioned methods for generating hyperspectral image data are technically very complex due to their operating principles. To generate a hyperspectral image of an object's examination area using the pushbroom and whiskbroom methods, a highly precise relative movement of the object to be imaged relative to the sensor is essential. Such relative movement is typically generated by moving the sensor relative to the object to be imaged. High-precision scanning of an examination area using either of these two methods therefore requires a complex imaging device design with an accurate actuator, which not only takes up space but also represents a significant cost factor in terms of acquisition and maintenance costs.Due to the size of conventional cameras for hyperspectral imaging, these cameras have so far been installed outside of an endoscope shaft, requiring light to be guided over long distances to the camera sensor using suitable optical elements, such as rod lenses. The inventor recognized that such optical elements cause absorption, dispersion, and / or scattering effects, for example, and thus significantly impair the quality of the image data. Furthermore, the inventor recognized that, due to the proximal arrangement of the hyperspectral camera, rigid endoscope shafts must be used to reliably guide the imaging light to the camera sensor using the aforementioned optical elements in such a way that a usable image can be generated.

[0007] Furthermore, image data generation is time-consuming with the pushbroom, whiskbroom, and staring approaches. Real-time monitoring of a diagnostic and / or therapeutic procedure using any of these three methods is therefore only possible to a limited extent. While the snapshot method allows for much faster data acquisition compared to the other serial systems, this technology, due to its design, has very limited spatial resolution, which impairs visual image quality and complicates image analysis.

[0008] DE 10 2021 134 276 A1 discloses a medical device and method for examining an organic tissue by means of characteristic distribution of multispectral intensity profiles of reference tissue sections.

[0009] DE 10 2020 105 458 A1 discloses a medical imaging device and a method for operating a medical imaging device.

[0010] US 2023 / 0 225 599 A1 discloses methods and devices for multispectral imaging.

[0011] DE 10 2020 129 739 B4 discloses an endoscope device in whose shaft an imaging channel is arranged, which has several imaging branches. The imaging branches each comprise spectrally selective beam splitters, whereby the examination area to be examined is divided into spectral partial images. The partial images are transmitted via relay optics to a camera arranged at the proximal end of an endoscope shaft, which camera has a spectrometer.

[0012] DE 10 2021 130 790 A1 discloses an endoscope device that records first and second image information and performs a calibration between the first and second image information.

[0013] From US 2022 / 0 087 515 A1 an endoscope system for OCT imaging is known, wherein the image acquisition assembly is arranged at least partially in the endoscope shaft.

[0014] The invention is based in particular on the object of expanding the possible applications of hyperspectral imaging.

[0015] This object is achieved according to the invention by the features of the independent claims. Further developments of the invention can be found in the dependent claims.

[0016] An endoscope device according to the invention comprises a shaft having a proximal and a distal section. Furthermore, the endoscope device comprises a hyperspectral image acquisition assembly configured to acquire images of an object region relevant during a therapeutic and / or diagnostic procedure and to generate hyperspectral image data comprising spatial and spectral information. The image acquisition assembly is arranged in the distal section of the shaft.

[0017] The features of the invention can expand the application possibilities of hyperspectral imaging. These features also allow the use of an endoscope device with a compact design and short light paths. In particular, the invention makes it possible to perform HSI imaging while simultaneously exploiting the advantages associated with arranging a camera in the distal shaft section of an endoscope. Furthermore, an endoscope device according to the invention can achieve a high degree of quality when performing and / or assessing diagnostic and / or therapeutic procedures.

[0018] An “endoscope device” should be understood in particular to mean a preferably functional component of an endoscope, in particular a subassembly and / or a structural and / or functional component of an endoscope. The endoscope device is designed, for example, to be inserted at least partially and preferably at least largely into an artificial or natural opening, in particular a body orifice, in order to carry out a treatment or examination there. “Designed” should be understood in particular to mean specially programmed, provided, designed, constructed and / or equipped. The fact that an object is designed for a specific function should also be understood to mean that the object fulfills or executes this specific function in at least one application or operating state.

[0019] The endoscope device is configured for medical imaging. “Medical imaging” is understood, in particular, to mean imaging that allows conclusions to be drawn about physiological properties of an object region to be imaged, such as anatomy, fat content, water content, oxygenation, the presence of a dye, or the like. The object region is, in particular, a region that includes physiological components, such as tissue, blood, or the like. The object region can furthermore be an image region that includes at least a part and / or section of an imaged object. The object region can relate to tissue and / or organs and / or a part of a patient’s body. For example, the object can be a site. The object region can relate to a site. The object region lies, for example, within a natural or artificially created cavity.Such cavities include the abdominal cavity, the intestine, the bladder, the kidney, or the like. However, open tissue could also serve as the object area. In some embodiments, the endoscope device is configured to be insertable, at least in sections, into a cavity for inspection and / or observation, for example, into an artificial and / or natural cavity, such as the interior of a body, a body organ, tissue, or the like. The endoscope device can also be configured to be insertable into a housing, casing, shaft, pipe, or other, particularly artificial, structure for inspection and / or observation.

[0020] The endoscope device has, in particular, a shaft within which the hyperspectral image acquisition module is arranged. The shaft is, in particular, designed to be inserted at least partially and preferably at least largely into a particularly artificial or natural opening, in particular a body orifice. "Distal" is to be understood, in particular during operation, as facing a patient or an object area and facing away from an operator. "Proximal" is to be understood, in particular during operation, as facing away from a patient or an object area and facing away from an operator. The shaft is designed as an elongated component.An "elongated component" is understood in particular to mean a component whose main extension is at least a factor of five, preferably a factor of ten, and particularly preferably a factor of twenty greater than an extension of the component at least substantially perpendicular to the main extension, i.e., in particular, a diameter of the component. A "main extension direction" of a component is understood in particular to mean a direction that runs parallel to a longest edge of a smallest imaginary cuboid that just completely encloses the component. "At least" means "at least".By “at least substantially parallel” is meant in particular an orientation of a direction relative to a reference direction, in particular an orientation of a direction relative to a reference direction, in particular in a plane, wherein the direction and the reference direction enclose an angle of 0°, in particular taking into account a maximum deviation of less than 8°, advantageously less than 5° and particularly advantageously less than 2°. By “at least substantially perpendicular” is meant in particular an orientation of a direction relative to a reference direction, in particular in a plane, wherein the direction and the reference direction enclose an angle of 90°, in particular taking into account a maximum deviation of less than 8°, advantageously less than 5° and particularly advantageously less than 2°.A "main extension" of an object is understood to mean, in particular, an extension of the object that runs along the longest edge of the smallest imaginary cuboid that just completely encloses the object. Furthermore, the proximal section of the shaft and the distal section of the shaft can be designed as separate components that can be securely coupled together. "Secure" refers to a design of the shaft sections such that the sections cannot be accidentally lost or separated from each other.

[0021] The distal section comprises, for example, a distal end section that extends, for example, over a maximum of 20%, a maximum of 10%, or even a maximum of 5% of the total length of the shaft. The proximal section can form the shaft together with the distal section and / or directly adjoin the distal section. In this respect, the proximal section can form a large part of the shaft and, in particular, can be proximal in that it is located proximal to the distal section.

[0022] The hyperspectral image acquisition assembly can be configured for hyperspectral imaging. A "hyperspectral image acquisition assembly" is to be understood in particular as a preferably functional unit or subsystem of an endoscope device, in particular a subassembly and / or a structural and / or functional component of an endoscope device, which consists of several individual components. The hyperspectral image acquisition assembly can form an endoscope device and preferably a distal shaft section, at least partially. A "hyperspectral image acquisition assembly" is to be understood in particular as having structural features enabling it to capture hyperspectral images and generate hyperspectral image data. Hyperspectral imaging orHyperspectral image data can refer in particular to imaging in which at least 20, at least 50 or even at least 100 spectral bands can be and / or are recorded independently of one another.

[0023] Hyperspectral images are, in particular, images or image data that contain spatial and spectral information about an object region to be imaged. The spectral information refers in particular to data about the light intensity of the object to be imaged across different wavelengths. The spatial information refers to the spatial properties and position of the object(s) within the object region to be imaged.

[0024] "Images" herein are understood to mean, in particular, static and / or dynamic visual images of an object area to be imaged. Dynamic visual images can, in particular, be moving images.

[0025] The image capture module is particularly configured to generate at least two-dimensional spatial image data. The image capture module can be spatially resolving in such a way that it delivers a resolution of at least 100 pixels, preferably of at least 200 pixels, preferably of at least 300 pixels, and advantageously of at least 400 pixels in at least two different spatial directions. The image data is preferably at least three-dimensional, with at least two dimensions being spatial dimensions and / or with at least one dimension being a spectral dimension. A plurality of spatially resolved images of the object region can be obtained from the image data, each of which is assigned to different spectral bands. The spatial and spectral information of the image data can be such that an associated spectrum can be obtained for a plurality of spatial pixels.

[0026] In some embodiments, the image capture assembly is configured to continuously generate updated image data. For example, the image capture assembly may be configured to generate the image data substantially in real time, which may include, for example, generating updated image data at least every 30 seconds, in some cases at least every 20 seconds, and in some cases even at least every 10 seconds or at least every 5 seconds.

[0027] According to a further development, the distal shaft section can comprise a distal end piece in which the hyperspectral imaging assembly is arranged. In other words, the hyperspectral imaging assembly can be arranged in a shaft section directly adjacent to and / or at least encompassing an endoscope tip. The endoscope tip is understood to be, for example, the most distal section of the distal shaft section. In this way, the imaging assembly is arranged in a shaft section that is spatially closest to an object area to be observed during a therapeutic and / or surgical procedure. The arrangement in the distal end piece also allows extensive design freedom with regard to the configuration of the proximal section and the distal section.In particular, for example, the shaft can be designed to be flexible at least in sections, in particular in an area proximal to the distal end piece.

[0028] According to the invention, the endoscope device comprises a camera assembly. This allows further images, such as white light images or fluorescence images, to be recorded in addition to hyperspectral images. A user can thus additionally observe and / or easily identify and / or assess an object region to be hyperspectrally imaged. The camera assembly can represent a unit or subsystem of an endoscope device, in particular a subassembly and / or a structural and / or functional component. In one embodiment, the camera assembly can be configured in addition to the hyperspectral image acquisition assembly. The camera assembly, with all of its functional components, can be arranged in the distal shaft section and in particular in the distal end piece of the shaft. For example, the camera assembly can be a camera cube.The camera assembly further comprises at least one, in particular first, image capture sensor system with at least one image sensor configured to capture images and generate image data. The at least one image sensor can be configured, in particular, as a color image sensor or a sensor system for white-light image capture.

[0029] The endoscope device can further comprise and / or be connectable to an illumination device comprising at least one illuminant configured to illuminate and / or illuminate the object region in at least one operating state. The illuminant can comprise a white light source, a particularly tunable monochrome light source, a laser, a white light laser, at least one light-emitting diode and / or a light-emitting diode array, at least one laser diode and / or a laser diode array, or the like. The illumination device can be formed integrally with the image capture sensor system. In particular, the illumination device can utilize individual or all components of the optics of the endoscope device and / or have separate illumination optics. An illumination light beam can be guided and / or guideable, at least in sections, coaxial with a measuring light beam.

[0030] According to the invention, the endoscope device comprises at least one input optic through which imaging light can be coupled. The input optic can define a viewing direction. This can be defined, for example, by an optical axis of the input optic and, in particular, can be parallel and / or coaxial with it. The viewing direction can be parallel to a longitudinal axis of the shaft and / or the distal section and / or the distal end piece, or can be oriented obliquely or at an angle relative to it. For example, it can enclose an angle of approximately 15 degrees, 30 degrees, 45 degrees, or 60 degrees with the respective longitudinal axis. The input optic can, for example, comprise an objective lens. In some embodiments, the input optic can be attached to the distal end piece on the distal side. In some embodiments, the input optic can have a diameter that at least corresponds to the radius of the distal end section and, in particular, is greater than this.This allows a high degree of efficiency in light coupling to be achieved.

[0031] In some embodiments, the endoscope device can further be configured to vary the viewing direction or angle, particularly during a procedure on a patient. For this purpose, the endoscope device can, for example, have pivoting optics and / or a pivotable shaft section that enable variable viewing directions, and / or a digital pivoting optic that enables different viewing directions through appropriate image processing. Viewing the object area from different viewing angles offers the advantage of obtaining a comprehensive and three-dimensional perspective, i.e., a panoramic view. This allows detailed information about, for example, the shape, structure, or properties of the object to be viewed and its surroundings to be obtained.

[0032] A further development also relates to an endoscope device comprising at least one input optic and at least one beam splitter. This allows for structural simplicity. Furthermore, imaging light can be coupled effectively and with high efficiency while still being able to be fed to different imaging paths or image sensors. The at least one input optic can be configured to guide object light or imaging light coming from the object area to the image capture sensor system, for example, to focus and / or project it. The object light can, in particular, originate from illumination of the object area. The at least one input optic can be used as a common optic for both the hyperspectral image capture assembly and the camera assembly. In some embodiments, the hyperspectral image capture assembly and the camera assembly can also have separate input optics or lenses.

[0033] The beam splitter can be arranged behind the input optics and configured to supply a first portion of incident light to the hyperspectral imaging assembly and a second portion of incident light to the camera assembly. Thus, a single input beam path can be used even if different image sensors and / or imaging methods are used. In embodiments in which the endoscope device has more than one input optics, a beam splitter may be obsolete. The beam splitter can, in particular, be configured to be wavelength-selective. In particular, the beam splitter can be a dichroic mirror, in some embodiments a wavelength-selective dichroic mirror.Furthermore, the beam splitter can be configured to supply visible light to the camera assembly, in particular at least in a wavelength range from 450 nm to 650 nm and preferably at least in a wavelength range from 420 nm to 700 nm, and to supply infrared light and / or UV light to the hyperspectral image capture assembly, in particular at least light with a wavelength of over 1500 nm and under 450 nm, preferably light with a wavelength of over 1000 nm and under 420 nm, and particularly preferably light with a wavelength of over 800 nm and under 400 nm. Different beam splitters with different optical properties can expediently be used. In particular, different beam splitters can differ with regard to their refractive index, reflection, transmission, absorption, dispersion, and scattering. In some embodiments, the beam splitter can be replaceable.This makes it possible, for example, to adapt to a relevant spectral range by selecting a beam splitter that appropriately reflects or transmits light in the relevant spectral range.

[0034] According to the invention, the hyperspectral imaging assembly comprises an aperture configured to filter and transmit the imaging light according to an aperture pattern. Additionally, the imaging assembly comprises a splitting optic configured to spectrally split the imaging light transmitted through the aperture. In some embodiments, the splitting optic can be designed, for example, as a prism or an optical grating. The type and number of splitting optics can vary, in particular, depending on different aperture patterns.

[0035] Furthermore, the hyperspectral imaging assembly comprises at least one, in particular second, image acquisition sensor system, which defines a light-sensitive region and is arranged with respect to the splitting optics such that spectrally split light falls on the light-sensitive region. The second image acquisition sensor system of the hyperspectral imaging assembly can be designed additionally and separately from the first image acquisition sensor system of the camera assembly. According to the invention, the input optics, the aperture, the splitting optics, and the image acquisition sensor system are arranged in the distal shaft section. It is understood that in some embodiments, only the image acquisition sensor system, referred to here as the "second" image acquisition sensor system, can be present, for example, when no camera assembly is present.

[0036] In an alternative embodiment of the endoscope device, the camera assembly and the hyperspectral imaging assembly can share a common image acquisition sensor. Such a configuration enables a very compact and efficient design, which not only has a positive impact on manufacturing costs and image acquisition efficiency, but can also open up new areas of application.

[0037] In order to specifically deflect or align the imaging light, further embodiments can provide additional optical elements, such as deflecting mirrors or collimator lenses, in addition to the input optics and the splitting optics.

[0038] The endoscope device, and in particular the first and / or second image acquisition sensor system, is configured for multispectral imaging, in particular white-light imaging and / or hyperspectral imaging, at least in image sections, specifically for capturing and / or generating multispectral and / or hyperspectral image data, at least in image sections. The white-light imaging and the hyperspectral imaging can be performed simultaneously and / or alternately. Furthermore, they can be performed simultaneously and / or successively at times. Multispectral imaging or multispectral image data can refer in particular to imaging in which at least three spectral bands are and / or can be captured independently of one another.“Image-wise” can mean that for at least a partial area of ​​a captured image, spectral information can be captured in addition to spatial information, for example for one or more pixels and / or image strips.

[0039] The image capture sensor system can comprise at least one image sensor. The at least one image sensor can define the light-sensitive region. The image sensor can be a silicon sensor, for example a CCD sensor or a CMOS sensor. The image sensor can comprise a two-dimensional pixel pattern. Generally speaking, the image capture sensor system can define image sections that are arranged in the light-sensitive region and cover it, in particular, according to a pixel pattern. The light-sensitive region can be an image capture sensor region defined by one or more image sensors. In some embodiments, the light-sensitive region is rectangular, wherein a first side of the light-sensitive region can be oriented at least substantially parallel to the spatial axis along which the spectral splitting occurs, and wherein a second side can be oriented at least substantially perpendicular to the first side.Image strips extending parallel to the second side can thus correspond to monochromatic or narrowband spatial image strips. In particular, the image strips or image points are defined by the aperture or the aperture pattern. Image strips extending parallel to the first side can correspond to spectra of a specific image point. The first side can be longer than the second side. The term "image sensor" can refer to a complete electronic component. An image sensor within the meaning of this disclosure can accordingly comprise, in addition to a semiconductor chip, associated contacts, conductor tracks, frames, and / or structural elements.

[0040] In particular, the image capture sensors, but also any other heat source within the endoscope device, such as lighting, can be equipped with a cooling device. By regulating and, in particular, lowering the operating temperatures of the image capture sensors, image noise can be reduced and image quality improved. Furthermore, sensitive components are subjected to less thermal stress, thus extending the service life of the endoscope device and reducing maintenance costs. A cooling device suitable for this purpose is described, for example, in DE 10 2019 129 815 A1.

[0041] The splitting optics can comprise a dispersive element that directs incident light, depending on the wavelength, to different positions and thus to different pixels of the image sensor used in the image acquisition sensor system. The wavelength of the light can then be determined from the position of a pixel. In some embodiments, the image sensor in question can be arranged such that a direction of dispersive splitting runs parallel to one of the two image axes of the image sensor. An additional imaging lens, which can be arranged between the image sensor and the splitting optics, can direct the light specifically onto the image sensor.

[0042] In a further embodiment, a beam path running from the beam splitter to the image sensor of the camera assembly and a beam path running from the beam splitter to the image acquisition sensor of the hyperspectral image acquisition assembly can run at least substantially parallel to each other, at least in sections. This can mean that light rays run offset from each other in a straight line and do not intersect or converge. In other words, these beam paths can run adjacent to each other in sections.

[0043] In some embodiments of an endoscope device, the camera assembly and the hyperspectral imaging assembly can be arranged adjacent to one another, at least in sections, with respect to a longitudinal axis of the distal portion of the shaft. Splitting them into adjacent beam paths is then structurally simple to implement. The adjacent arrangement allows two input optics to be used adjacent to one another. Beam paths can thus be constructed independently of one another. It is also conceivable to use a beam splitter if the camera assembly and the hyperspectral imaging assembly are arranged adjacent to one another. A single input optic can then be used to couple in the light, but the different beam paths can be constructed separately from one another. In general, a compact arrangement of modules can be provided.

[0044] In alternative embodiments, the camera assembly and the hyperspectral image acquisition assembly can be arranged at least partially one behind the other with respect to the longitudinal axis of the distal section of the shaft. This allows the shaft cross-section to be effectively utilized for each of the image sensors used. Available installation space along the longitudinal axis can thus be utilized.

[0045] Embodiments in which the camera assembly and the image capture assembly are arranged side by side with respect to a longitudinal axis of the distal section of the shaft also have the advantage that the light path of the imaging light to the image sensors is very short. Light losses, particularly losses in the IR range, can thus be minimized with such an arrangement of the assemblies.

[0046] In the embodiment in which the assemblies are arranged one behind the other, the shaft of the endoscope device can be optimized or reduced due to the space-saving arrangement. Thus, the diameter of the shaft of the endoscope device can preferably be at most 15 mm, preferably at most 10 mm, and particularly preferably at most 5 mm.

[0047] In a further development of the endoscope device, the image acquisition sensor system can be stationary relative to the distal section of the shaft. Likewise, the entrance optics and / or the splitting optics can be stationary relative to the distal section of the shaft. Furthermore, the image acquisition sensor system can be immobile relative to the entrance optics. In other words, such an embodiment of an endoscope device does not have a displacement unit configured to displace, move, and / or shift the camera assembly and / or the hyperspectral image acquisition assembly relative to the object area to be imaged. By eliminating such a displacement unit, which is indispensable in conventional imaging devices that operate according to the pushbroom or whiskbroom principle, the imaging device can be designed to be significantly more compact, and access to the object area is made easier for a treating physician.In addition, an endoscope device can be provided that is less susceptible to vibrations and movements, thus avoiding the corresponding artifacts that can occur in the case of pushbroom and whiskbroom arrangements due to the comparatively slow and complex mechanical scanning.

[0048] According to a further embodiment, the aperture of the image capture assembly can be designed as a pinhole aperture or a slit aperture comprising a plurality of preferably parallel rows of aperture holes and / or slits. The following statements regarding aperture holes or rows of holes are also analogously applicable to slits or apertures with slits.

[0049] The aperture holes are configured to define image sections or subsections of the object area to be observed and to selectively control the light flux of the imaging light. The image sections can, for example, make up at least 1%, at least 2%, at least 5%, or at least 20% of a total image area. The rows of aperture holes can run obliquely with respect to a transverse direction of the splitting optics and / or the light-sensitive area. Furthermore, the rows of aperture holes can be spaced apart from one another by a hole row spacing. The hyperspectral image capture assembly can further define a resolvable wavelength range. The splitting optics can further be configured to split the resolvable wavelength range along a spatial axis over a splitting distance that is at most as large as the hole row spacing.Such a configuration prevents multiple spectra of different pixels from overlapping.

[0050] It goes without saying that, in principle, a variety of different aperture patterns with different properties can be used. The aperture pattern used can therefore vary depending on the application and requirements and can be replaced accordingly.

[0051] In a further development, the aperture can be mounted so as to be movable relative to the image capture sensor system. Such a configuration makes it possible to increase the spatial resolution and thus the quality of the image. In particular, in combination with an aperture having multiple rows of holes and / or slits, this allows multiple image lines to be recorded while at the same time achieving sufficient spectral resolution for each image line. Individual spectra of individual points then appear next to one another depending on the arrangement of the rows of holes or slits. A complete image can be recorded by moving the aperture relative to the image capture sensor system between individual image recordings. In turn, multiple image lines can then be hyperspectrally imaged simultaneously. In some embodiments, an aperture with a single row of holes and / or a single slit can also be used.In this case, the aperture can then be displaceable over a distance that is sufficiently large to allow the light strip guided through the single row of holes and / or the single slit onto the image capture sensor to be moved over the entire light-sensitive area of ​​the image capture sensor or at least over a large part of it. A hyperspectral data set can then be obtained from the individual images recorded in this way. The image capture assembly can comprise an aperture displacement unit with an actuator and an aperture carrier. In a further development, the aperture and the aperture carrier can be formed as a single piece. The aperture carrier can be connected to the actuator so that the aperture can be moved relative to the image capture sensor by means of the actuator.

[0052] Furthermore, the panel support can comprise a flexure joint. This can be reliable, resilient, and / or cost-effective. The flexure joint preferably defines a single pivot axis that defines the movement of the panel.

[0053] According to a further development, the actuator can be configured to generate a linear movement. This linear movement can be transformed into a pivoting movement of the aperture by means of the flexure joint. The aperture displacement unit or the aperture is preferably configured such that the direction of the pivoting movement is different from the direction of extension of the axes extending along the rows of holes or columns. Particularly preferably, the pivoting movement occurs perpendicularly or obliquely to the axes extending along the rows of holes or columns. The actuator can be designed, in particular, as a piezo actuator.

[0054] Existing installation space can be utilized efficiently, especially if the actuator is positioned behind the image acquisition sensor system with respect to a longitudinal axis of the distal section, as viewed from the input optics, so that the aperture support spans the image acquisition sensor system. This allows the actuator to be positioned proximally with respect to the image acquisition sensor system, thus leaving the light path to the image acquisition sensor system unaffected. The aperture support can thus act as a lever supporting the aperture. The aperture support can be pivotable about the pivot axis defined by the flexure joint.

[0055] In one embodiment, the aperture carrier or aperture can be moved without also moving the image capture sensor. Thus, during the displacement process, the aperture holes and the image capture sensor move relative to one another. Depending on the configuration of the image sensor, this relative movement results in less and less light from the object area to be imaged falling on the image sensor of the image capture sensor as the displacement increases. To ensure that sufficient spectral information is still captured by the image capture sensor, the aperture or aperture holes can be moved.the aperture holes are movable along a movement path over a maximum distance at which the resolved wavelength range of the pixels is shifted by a maximum of 1000%, preferably a maximum of 800%, preferably a maximum of 600%, and particularly preferably a maximum of 400% of a width and / or by a maximum of 100%, preferably a maximum of 80%, preferably a maximum of 60%, and particularly preferably a maximum of 20% of the length of the pixel pattern defining a pixel. In this way, the spatial resolution can be improved quickly and with little technical effort, so that sufficient and meaningful information about the object area to be imaged can be captured. The quality of the resolution depends, among other things, on the number of aperture holes, or the number of pixels in the detector array, as well as the amount of movement of the aperture.

[0056] According to the invention, the diaphragm is rotatably mounted about a rotation axis arranged at least substantially parallel or coaxially to an optical axis of the diaphragm. The diaphragm can be designed, in particular, as a slit diaphragm, the observation slit of which is configured to select at least one and preferably a plurality of different image strips of an intermediate image of an object region to be imaged, generated by the input optics. Furthermore, the image capture assembly can comprise a beam rotator element, in particular in the form of a Dove prism, which is rotatably mounted together and / or separately with / from the diaphragm. The beam rotator element can be configured to rotate the rays of light spectrally split by the splitting optics such that they can be detected by the image capture sensor system, regardless of the position of the diaphragm slit, without influencing the light spectrum.In other words, the beam rotator element serves to always align the beams in the same direction toward the splitting optics. Preferably, the rotation axis of the aperture is coaxial with the longitudinal axis of the beam rotator element.

[0057] Furthermore, the image capture assembly can comprise at least one rotary drive configured to rotate the aperture and / or the beam rotator element about the rotation axis. Such an arrangement allows the movement required to select different image strips to be generated simply, reliably, and reproducibly. Advantageously, the rotation axis runs through the center of gravity of the beam rotator element. Furthermore, the aperture and / or the beam rotator element can be rotatable in two opposite directions about the at least one rotation axis. Furthermore, the rotational movement can be oscillatory, in particular according to a pendulum movement, or continuous. Continuous movements are significantly more stable than, for example, oscillating movements because they are less susceptible to disturbances or irregularities.Furthermore, continuous motion can avoid pendulum movements and the associated stresses caused by periodic acceleration. In particular, a uniform rotation speed can be maintained. Depending on the application, a continuous rotary motion of the optical element may therefore be preferable. Depending on the application and the associated requirements for an endoscope device, different drive types, rotation speeds, and / or angles can be used.

[0058] In a further development, the hyperspectral image acquisition assembly can comprise a control unit configured to control the drive element of any type displacing the aperture and / or the beam rotator element and the image acquisition sensor system. This control can be synchronized. The image acquisition sensor system can further be configured to perform data acquisition, in particular continuous data acquisition, synchronized with the continuous movement of the aperture and / or the beam rotator element. Thus, the data acquired by the image acquisition sensor system can always be assigned to a position of the aperture and thus to a position on the object area. The image acquisition sensor system can, for example, be configured to generate the image data essentially in real time. An image acquisition rate of the image acquisition sensor system can, for example, be at least 1 fps, 10 fps, 20 fps, 30 fps, 50 fps, or even 100 fps.

[0059] According to a further embodiment, the hyperspectral image acquisition unit can comprise an optical intermediate image displacement unit configured to optically displace the intermediate image relative to the aperture, and in particular relative to the observation slit of the aperture, such that different intermediate image sections of the intermediate image can be selected using the aperture. The optical intermediate image displacement unit can comprise a movable optical element configured to displace the position of the intermediate image relative to the observation slit. Such an optical intermediate image displacement unit enables the scanning of an extended object region that lies outside the object region whose intermediate image falls on a positionally accurate observation slit of the aperture or that is acquired by a positionally accurate image acquisition assembly.In this way, the position of the intermediate image can be shifted quickly and with little technical effort in order to capture sufficient information about the object area to be imaged.

[0060] In a further development, the optical element can be arranged between the entrance optics and the aperture. Furthermore, the optical element can comprise a prism, in particular an at least four-sided prism. This prism can have specific optical properties. Different prisms with different optical properties can be used. In particular, different prisms can differ in terms of their refractive index, reflection, transmission, absorption, dispersion, and scattering.

[0061] Furthermore, the hyperspectral image acquisition sensor can be configured to continuously acquire hyperspectral image data during a continuous movement of the optical element. The data acquisition of the hyperspectral image acquisition sensor can be synchronized with the continuous movement of the optical element. Thus, the data acquired by the image acquisition sensor can always be assigned to a position of the optical element and thus to a position on the object area. The image acquisition sensor can, for example, be configured to generate the image data essentially in real time. An image acquisition rate of the image acquisition sensor can, for example, be at least 1 fps, 10 fps, 20 fps, 30 fps, 50 fps, or even 100 fps.

[0062] In a further embodiment, the intermediate image displacement unit can comprise a drive for the optical element, which is configured to rotate the optical element in a predetermined direction of rotation, in particular by at least 90 degrees and preferably at least 360 degrees. The direction of rotation and / or the rotational speed and / or the angle of rotation can be varied. Particularly preferably, the rotational movement can be continuous. A continuous movement can avoid pendulum movements and the associated loads due to periodic acceleration. In particular, a uniform rotational speed can be maintained. Furthermore, continuous movements are significantly more stable than, for example, oscillating movements, since they are less susceptible to disturbances or irregularities. Depending on the area of ​​application, a continuous rotational movement of the optical element can therefore be preferred.

[0063] Alternatively, the drive can be designed, for example, as a pendulum drive and configured to move the optical element oscillatory, particularly according to a pendulum motion. Depending on the application and the associated requirements for an endoscope device, different drive types can be used. The drive can be initiated and controlled, in particular, via a motor unit coupled to the optical element in a torque-transmitting manner.

[0064] In some embodiments, the endoscope device may comprise a holding device configured to hold the camera assembly and / or the hyperspectral imaging assembly. The holding device may be configured such that it can be arranged in the distal end piece of the shaft. In particular, a rail system and / or a mechanical and / or magnetic locking device may be provided, with which the holding device can be reversibly coupled to the shaft. Alternatively or additionally, the distal end piece may have an access device on the outer circumferential surface of the shaft, through which the assemblies can be made accessible to a user as needed. Both the holding device and the access device can, in particular, facilitate manufacturing, maintenance work, and / or repairs.

[0065] A further aspect also relates to a medical system comprising an endoscope device according to the invention. Furthermore, the medical system comprises a display generation unit configured to generate a display for a user based on combining information from different pixels and / or image strips to form an image sub-area. Furthermore, the display may comprise a superimposition of the image sub-area and an image, in particular a white-light image, of the object area to be imaged.

[0066] The medical system can also comprise an analysis unit configured to create an analysis based on spectral information based on image data from the image acquisition sensor system. The analysis can comprise the determination of at least one physiological parameter, in particular a perfusion parameter. The analysis is based in particular on calculations performed point by point or image section by image section, each of which is based on spectral information relating to the corresponding image point or image section. For example, a perfusion level can be determined with spatial resolution based on the analysis. Furthermore, the representation generation unit can be configured to generate a representation for a user based on the analysis. The analysis unit preferably uses a spectral analysis to determine physiological properties.

[0067] Furthermore, the representation generation unit may be configured to generate a representation for a user based on the analysis.

[0068] The medical system may further comprise an output unit configured to output the representations generated by the representation generation unit to a user. The output unit may comprise a computer and / or processor and / or memory and / or RAM and / or ports and / or a data interface for receiving, processing, and outputting unprocessed, preprocessed, and / or processed representation data.

[0069] The medical system may also comprise a display unit configured to display an image, in particular a moving image, and in particular the image data captured by the image capture device, to a user. The display unit may be coupled to the output unit, in particular wirelessly. The displayed image may be based on the image data from the image capture sensor system. The display unit may comprise a screen and / or control electronics. The display unit may comprise a computer and / or processor and / or memory and / or RAM and / or connections and / or a data interface for receiving, processing, and outputting unprocessed, preprocessed, and / or processed image data and / or display data. Both the output unit and the display unit may be components of the display generation unit.

[0070] The medical system may include a control unit. The control unit may include suitable control electronics and / or a computer. In some embodiments, the control unit includes at least one processor, computer-readable memory, an operating system, and / or suitable inputs and outputs. The control unit may include at least one control program.

[0071] In particular, functions thereof can be implemented by the control program or be part of it. The endoscope device and in particular the control unit can each comprise at least one processor and / or an associated memory with program code that implements the described functions and steps, and / or an associated main memory and / or associated connections and / or data interfaces and / or an electronic circuit for implementing the functional units mentioned herein and / or for carrying out method steps. One or more processors, memories, main memories, connections, data interfaces and / or circuits can also be associated with one or more functional units and / or implement one or more method steps.

[0072] The devices and systems according to the invention are not intended to be limited to the application and embodiment described above. In particular, they may comprise a number of individual elements, components, and units that differs from the number stated herein to fulfill a functionality described herein. Furthermore, within the ranges of values ​​specified in this disclosure, values ​​within the stated limits are also to be considered disclosed and can be used arbitrarily.

[0073] The present invention is described below by way of example with reference to the accompanying figures. The drawings, the description, and the claims contain numerous features in combination. Those skilled in the art will expediently consider the features individually and use them in meaningful combination within the scope of the claims.

[0074] If there is more than one instance of a particular object, only one of them may be provided with a reference symbol in the figures and in the description. The description of this instance can be transferred accordingly to the other instances of the object. If objects are named using numerical terms, such as first, second, third object, etc., these serve to name and / or assign objects. Accordingly, for example, a first object and a third object, but not a second object, may be included. However, a number and / or sequence of objects could also be derived using numerical terms.

[0075] They show: Fig. 1 a schematic representation of a medical system with an endoscope device; Fig. 2 a perspective side view of a distal end piece of the endoscope device according to Fig. 1; Fig. 3 a schematic cross-sectional view of a distal end piece of the endoscope device according to Fig. 2; Fig. 4 a schematic plan view of a distal end piece of the endoscope device according to Fig. 2; Fig. 5 a perspective detailed view of a partial section of the hyperspectral imaging assembly according to Fig. 2; Fig. 6 a simplified representation of the hyperspectral imaging assembly according to Fig. 2; Fig. 7 is a schematic representation of a light-sensitive area; Fig. 8 is a schematic representation of another embodiment of a distal end piece of an endoscope device; Fig. 9 is a schematic representation of another embodiment of a distal end piece of an endoscope device; Fig. 10 a displacement of image points on an object area with an endoscope device according to Fig. 2; Fig. 11 an overlay representation based on a white light image and hyperspectral image data; Fig. 12 an overlay representation based on a white light image and hyperspectral image data; Fig. 13 is a schematic representation of another embodiment of a distal end piece of an endoscope device; Fig. 14 a schematic representation of a diaphragm according to Fig. 13; Fig. 15 a schematic representation of another embodiment of a diaphragm according to Fig. 13; Fig. 16 a displacement of image strips on an object area with an endoscope device according to Fig. 13; Fig. 17 an overlay representation based on a white light image and hyperspectral image data; Fig. 18 is a schematic representation of another embodiment of a distal end piece of an endoscope device; Fig. 19 a schematic detailed view of an optical intermediate image shifting unit according to Fig. 18; Fig. 20 a displacement of image areas on an object area with an endoscope device according to Fig. 18; Fig. 21 an overlay representation based on a white light image and hyperspectral image data;

[0076] Fig. 1 shows a schematic representation of a medical system 72 with an endoscope device 10. According to the embodiment shown, the endoscope device 10 is part of an endoscope 70. The endoscope device 10 comprises a shaft 12 with a proximal section 14 and a distal section 16. The proximal shaft section 14 can be equipped with a handle 94, which serves a treating physician for safe and precise operation. The distal shaft section comprises a distal end piece 22 of the shaft 12. A hyperspectral image capture device 18 is arranged in this distal end piece 22. This hyperspectral image capture device 18 is described in more detail with reference to the following figures.

[0077] The endoscope device 10 is configured to capture image information, in particular hyperspectral image information, of an object region 20 to be imaged during a therapeutic and / or diagnostic procedure using the hyperspectral image capture module 18. Furthermore, the system 72 comprises a display generation unit 74 configured to generate a display for a user based on the captured and / or analyzed image information. A display 78 is displayed for a user via a display unit 82, here in the form of a monitor, coupled to the display generation unit 74. This display 78 shows, in particular, an overlay of an image sub-region and an image of an object region 20 to be imaged. The display 78 is further based on the analysis of the image information.

[0078] As shown, the system 10 can include a control device 92 with a supply unit 84 to which the endoscope 16 can be selectively coupled. The supply unit 84 can be configured to control the endoscope 70 and / or receive image data and / or other data from the endoscope 70. The supply unit 84 can be connectable and / or connected to the display unit 82 of the system 72, on which a user can display recorded images. The supply unit 84 can also provide illumination light for the endoscope 70. The endoscope 70 can be connectable and / or coupled to an illumination device 88 via a fiber optic cable 86. The endoscope 70 can further be connectable and / or coupled to the supply unit 84 via an electrical cable 90. The electrical cable 90 can be configured to transmit electrical energy and / or data.

[0079] In other embodiments not shown here, a separate illumination device and / or illumination light source may be provided. Furthermore, the endoscope 70 may alternatively or additionally comprise integrated lighting elements for providing illumination light.

[0080] Fig. 2 shows a perspective side view of the structure of the distal end piece 22 of an endoscope device 10 according to Fig. 1. The distal end piece 22 is, for example, essentially formed by the distal section 16 of the shaft 12. In this respect, the proximal shaft section 14 can extend to the distal end piece 22. Alternatively, the distal shaft section 16 can comprise that part of the shaft 12 intended for insertion. The proximal shaft section 14 can then be only that section of the shaft 12 that remains outside the patient when the shaft 12 is inserted. In the embodiment shown, an outer housing of the distal end piece 22 is formed as a section of a hollow cylinder. The end piece 22 can also be functionally and captively coupled to a proximal section 14 of the shaft 12 via coupling regions 98a,b. For example, the end piece can be reversibly mounted to the proximal portion 14 of the shaft 12 either magnetically or via a complementary threaded device.Alternatively, the end portion 22 can also be formed integrally with the proximal shaft portion 14. Such a configuration particularly simplifies the manufacture, maintenance, and, if necessary, replacement of an endoscope device 10.

[0081] At the distal end of the end piece 22, an end surface 100 closes the opening of the hollow cylinder in a form-fitting manner. The end surface 100 can, for example, be screwed onto the opening, firmly glued to it, or formed integrally with the hollow cylinder. Fig. In the embodiment shown in Figure 2, a recess is further provided in the end surface 100, in and / or behind which an input optic 30 is arranged. In the embodiment shown, a protective glass 108 is mounted in the recess formed in the end surface 100, which protects the image capture modules from external influences without affecting the spectral properties of imaging light AL of an object 21 to be imaged. In an alternative embodiment, the input optic 30 can also be arranged in a form-fitting manner in the recess. The protective glass 108 can also be considered part of the input optic 30.

[0082] The input optics 30 extend parallel to the longitudinal axis LA of the distal section 16 into the interior of the hollow cylinder and thus away from the end surface 100. The input optics 30 is configured to couple the imaging light AL of an object 21 to be imaged into the endoscope device. In other embodiments not shown here, the input optics 30 can also be arranged, for example, in the lateral surface of the shaft or the hollow cylinder. The input optics 30 can comprise a plurality of optical elements, for example in the form of various lenses, which focus the imaging light. The input optics 30 can be an objective lens.Following the beam path of the imaging beams AL, a beam splitter 32 is provided spatially behind the entrance optics 30. This beam splitter is configured to supply a first portion of the incident imaging light HL to the hyperspectral image capture module 18 and to supply a second portion of the incident light SL to a camera module 24 of the endoscope device 10. The beam splitter 32 shown can, in particular, be a dichroic beam splitter that is wavelength-selective.The beam splitter 32 is particularly configured to supply visible light SL to the camera assembly 24, in particular at least in a wavelength range from 450 nm to 650 nm and preferably at least in a wavelength range from 420 nm to 700 nm, and to supply infrared light to the hyperspectral image capture assembly, in particular at least light with a wavelength of more than 1500 nm, preferably light with a wavelength of more than 1000 nm and particularly preferably light with a wavelength of more than 800 nm.

[0083] Located behind the beam splitter 32 is a first image acquisition sensor system 26a associated with the camera assembly 24, comprising an image sensor 28 configured to detect the light transmitted through the beam splitter. In particular, the first image acquisition sensor system 26a is configured to detect light in the visible wavelength range.

[0084] Spatially, a mirror 96 is arranged below the beam splitter 32, which directs the light HL reflected by the beam splitter 32 toward the hyperspectral imaging assembly 18. The hyperspectral imaging assembly 18 includes an aperture 34 configured as a pinhole aperture 40. This aperture 34 is configured to filter out and transmit the light HS deflected by the mirror 96 toward the hyperspectral imaging assembly 18 according to an aperture pattern. Behind the aperture 34 is a splitting optic 36 configured to spectrally split the light HS transmitted through the aperture 34 along a spatial axis. The splitting optics 36 are followed by a focusing optics 102 in the form of a collimating lens, which directs the beam path of the spectrally split light onto a light-sensitive area 38 of a second image acquisition sensor 26b belonging to the hyperspectral image acquisition assembly 18.Like the input optics 30, the focusing optics 102 can also comprise several optical elements, for example in the form of different lenses.

[0085] The Fig. The hyperspectral image acquisition assembly 18 shown in Figure 2 also includes a diaphragm displacement unit 44, which includes an actuator 46 and a diaphragm carrier 48. In this embodiment, the actuator 46 is configured as a piezo actuator 52. The diaphragm displacement unit 44 is configured to move the diaphragm 34 relative to the image acquisition sensor system 26b. The diaphragm carrier 48 is connected to both the actuator 46 and the diaphragm 34. Furthermore, the diaphragm carrier 48 includes a flexure joint 50. The actuator 46 is configured to generate a linear movement. This linear movement is converted into a pivoting movement of the diaphragm 34 by means of the flexure joint 50. The movement path of the flexure joint 50 and the resulting movement of the diaphragm 34 is indicated by the two double arrows. The movement of the solid-state element 50 and the resulting pivoting movement of the aperture 34 is indicated by two arrows.

[0086] The aperture support 48 has a length that exceeds the shaft diameter. The length can, for example, correspond to at least 1.5 times, at least 2 times, or at least 3 times the shaft diameter. As a result, even small pivoting movements in the area of ​​the flexure joint 50 are translated into lateral movements of the aperture 34 that extend over at least 10%, at least 20%, or even at least 30% of the shaft diameter. Furthermore, the movement of the aperture is approximately linear due to the extension of the aperture support 48. The flexure joint 50 is therefore only subjected to minor loads, since it only has to accommodate rotations of a few degrees, for example, less than 20 degrees, less than 10 degrees, or even less than 5 degrees.

[0087] With respect to the longitudinal axis LA of the distal section 16, the actuator 46 is arranged behind the image acquisition sensor 26b as viewed from the input optics 30, so that the aperture carrier 48 spans the image acquisition sensor 26b.

[0088] In this embodiment, the camera assembly 24 and the hyperspectral image capture assembly 18 are arranged side by side with respect to the longitudinal axis LA of the distal section 16 of the shaft 12. According to this arrangement, the beam path from the beam splitter 32 to the image capture sensor system 26a of the camera assembly and the beam path from the beam splitter 32 to the image capture sensor system 26b of the hyperspectral image capture assembly 18 run parallel to each other, at least in sections. This spanning arrangement allows the required overall length of the aperture carrier 48 to be achieved without having to guide light passing through the aperture 34 over an unnecessarily long distance. Furthermore, the actuator 46 can be arranged on a side of the image capture sensor system facing away from the input optics 30, so that it does not interact with the light path of the image capture assembly 18.

[0089] The Fig. 3 and Fig. 4 each show a different view of the distal end piece 22 of the endoscope device 10 according to Fig. 2. Fig. 3 shows a schematic cross-sectional view of the distal end piece 22 and Fig. 4 a schematic plan view of the distal end piece 22.

[0090] Fig. 5 shows a perspective detail view of a portion of the hyperspectral imaging assembly according to Fig. 2. The diaphragm connected to the diaphragm carrier 48 comprises a plurality of rows 116 of diaphragm holes 42 that run parallel with respect to a transverse direction of the splitting optics 36. Furthermore, the rows 116 of diaphragm holes 42 are spaced from one another by a hole row spacing d. At this point, it should be noted that the diaphragm pattern shown in this embodiment is merely exemplary. It is understood that the diaphragm pattern shown in this figure and / or the diaphragm shape can vary depending on the embodiment and area of ​​application. In particular, in other embodiments, the number of diaphragm holes 42, the number of rows 116 of diaphragm holes 42, the hole row spacing d, the size of the diaphragm holes 42, the shape of the diaphragm holes 42, and the arrangement of the diaphragm holes 42 on the diaphragm 34 can vary. The same also applies to all diaphragms described below.

[0091] Fig. Figure 6 shows a simplified representation of the hyperspectral imaging assembly 18 according to the Fig. 2 to 3. Light HL is filtered out and transmitted through the pinhole 40 according to a pattern defined by the aperture holes 42. For the sake of clarity, only one aperture hole 42 is provided with a reference symbol. Behind the pinhole 40, the transmitted light strikes a splitting optic 36, which is configured to split the light in a wavelength range predefined by the hyperspectral image acquisition assembly along a spatial axis over a splitting distance d. λspectrally split. The splitting distance is at most as large as the hole row spacing d. Preferably, it is somewhat smaller than a hole row spacing projected along the beam path onto the image sensor 28, for example, by at least 10% or at least 20% smaller, so that no overlaps occur on the image sensor 28. A focusing optic 102 is arranged behind the splitting optic, which directs the beam path of the split light onto a light-sensitive area 38 of the image sensor 28 of the image acquisition sensor system 26b. The aperture displacement unit 44 is not shown in this illustration, but the direction of movement of the aperture is schematically illustrated by a double arrow.

[0092] Fig. 7 shows an example of a schematic representation of a Fig. 6, onto which spectrally split light originating from a single aperture falls. The light-sensitive area comprises a matrix of image pixels configured to capture wavelength components of the previously spectrally split light and convert them into electrical signals. In the present case, the image sensor 28 is, for example, a CCD or CMOS sensor, preferably without a color filter matrix. This thus detects light of all wavelengths of the split wavelength range. For the purposes of hyperspectral image data acquisition, the light-sensitive area 38 is divided into a detection array, which may, for example, comprise individual image cells 110. Each of these image cells 110 can be assigned a specific wavelength or wavelength range. The division into image cells 110 can be different from the individual pixels of the image sensor 28.A strip of such image cells 110 can then be used to obtain discrete spectral measurement points for a specific spatial point of the acquired image, wherein said spatial point is defined by the respective aperture hole 42. With knowledge of the splitting distance d. λ the electrical signals of different image cells 110 can be assigned to different wavelength ranges λ1- n.

[0093] The Fig. 8 and Fig. 9 show further embodiments of the endoscope device 10, in which the camera assembly 24 and the hyperspectral image acquisition assembly 18 are not arranged side by side, but one behind the other, along the longitudinal axis LA of the end section 22 of the endoscope device 10. The camera assembly 24 and the hyperspectral image acquisition assembly 18 are shown in the Fig. 8 and Fig. 9 is shown as a simplified black box. The detailed structure is shown in the Fig. 2 and Fig. 3. In order to be able to arrange the camera assembly 24 and the hyperspectral image acquisition assembly 18 one behind the other, the endoscope device 10 has additional mirrors 96a,b. These mirrors 96a,b are configured to redirect the imaging light AL or portions thereof HL in a space-saving manner.

[0094] In the Fig. In the embodiment shown in Figure 9, the endoscope device 10 has, in addition to the input optics 30a, an additional input optics 30b. The additional input optics 30b is arranged next to the input optics 30a. In some embodiments, the two input optics 30a,b can be identical or different. The position and / or number of additional input optics 30 can also vary in embodiments not shown here, depending on the area of ​​application. In the embodiment shown, the first input optics 30a is configured to couple in imaging light and guide it to the image capture sensor system 26a of the camera assembly 24. The additional input optics 30b serves to couple in imaging light and guide it to the image capture sensor system 26b of the hyperspectral image capture assembly. Thus, according to this embodiment, the two assemblies 24, 18 do not have a common input optics 30, but rather separate input optics 30a,b.

[0095] In the Fig. 10 to 12 are partial steps of image generation with a Fig. 2. An object region 20 with a plurality of tissue structures 104 is shown. The pinhole 40 or its apertures 42 are configured in an initial position to select pixels 76 of the object region to be imaged according to the aperture pattern and to guide their light to an image acquisition sensor 26b, which acquires the spectral information of the pixels 76. The spectral information obtained for the pixels 76 can then be evaluated and, for example, superimposed on a white light image. The double arrow above the pinhole 40 indicates the degree of freedom or the two possible directions of movement of the pinhole 40. By moving the pinhole 40, spectra can be acquired successively for different spatial points of the image.

[0096] Fig. 11 shows an exemplary overlay display 78. Image information based on spectrally resolved image data is superimposed on a white light image 114 in a central image area 106. By way of example, in the case shown, different tissue types are highlighted differently, with the tissue types being automatically identified based on the spectral information obtainable from the different pixels 76 described above. For the sake of clarity, only one pixel 76 is provided with a reference symbol. Since spectral data is only available for the image area 106 selected by the aperture 34, only a portion of the overlay display is provided with such information. In the present case, this information is obtained for a central area, which can be useful since a user tends to want to display the relevant object areas centrally in the image.Although spectral information is only available in sections or points, additional information can thus be obtained for the relevant image areas 106.

[0097] It is understood that alternatively, other information may be displayed based on spectral and spatial information available for image area 106. For example, a perfusion state or another spatially resolved parameter may be displayed based on a comparison of at least two spectral intensity values ​​associated with different spectral ranges and / or different wavelengths.

[0098] Likewise, it may be provided to superimpose an intensity distribution for a specific wavelength and / or a specific wavelength range on the white light image 114 in the image area 106. This can, for example, make visible the fluorescence of a dye that is introduced into the corresponding object area and is visible in the spectrally resolved pixels 76.

[0099] Furthermore, it is understood that instead of a white light image 114, another image may also form the basis for the overlay display 78, such as a fluorescence image, a monochromatic image, a false color image, or the like.

[0100] In addition, an operating mode can be provided in which only the image area 106 is displayed and / or in which only image data based on the spectral image capture, i.e. specifically on the pixels 76, are displayed in the area of ​​the image area 106.

[0101] By displacing the aperture 42 relative to the image acquisition sensor 26b by means of the aperture displacement unit 44, the spatial resolution of the spectrally resolved image data can be increased. Fig. 12, the aperture 42 was shifted to the right by a distance which is smaller than the hole row spacing d, so that points hidden by the aperture 42 can be viewed in a first image recording, which points can now be viewed through the aperture 42 in a second image. In the shifted position, which deviates from the initial position, other image points 76 are selected by the aperture holes 42 than in the Fig. 10 and Fig. 11 shown starting position. In the Fig. In the scenario illustrated by way of example in Figure 12, the aperture 34 in the displaced position captures, at least to a large extent, the image area 106 which, in the initial position, lay between the rows 116 of the aperture holes 42 and therefore could not be measured in the initial position.

[0102] Fig. 13 shows a schematic representation of another embodiment of a distal end piece 22 of an endoscope device 10. This embodiment differs from those described above essentially in the structure of the hyperspectral image acquisition assembly 18. In this embodiment, the aperture 34 is mounted for rotation about a rotation axis. The rotation axis runs at least substantially parallel to the optical axis of the aperture 34. Furthermore, the aperture 34 according to this embodiment is designed as a slit aperture 54.

[0103] However, it should be noted at this point that this embodiment is by no means limited to the use of a slit diaphragm 54. As can be seen from the explanations regarding Fig. 15, other aperture patterns that are also suitable for rotation can also be provided. A Dove prism 56 is also arranged behind the slit diaphragm 54. The Dove prism 56 is designed to redirect or rotate the beam path of the light of the image area 106, which is selected by the slit diaphragm 54, so that the light emerging from the Dove prism 56 or its beam path always has the same orientation with respect to the splitting optics 36 after and / or during a rotation of the slit diaphragm 54 by a defined angular amount. In addition, the slit diaphragm 54 is rotatably mounted together with the Dove prism 56 by means of a rotary drive 58. The rotary drive 58 is shown purely schematically and only in relation to its function. For the sake of clarity, it is shown at the bottom edge of the Fig. 13. However, it is understood that the rotary drive 58 is also structurally arranged within a housing of the distal end piece 22.

[0104] In Fig. 13, but not limited to use in the embodiment shown therein, are cooling devices 68 arranged on the endoscope device 10, in particular on the image acquisition sensor system 26a,b. These cooling devices are configured to cool the image acquisition sensor system 26a,b during and / or after operation. These can be, for example, Peltier cooling elements, heat pipes, coolant-fed coolers, and the like.

[0105] In the Fig. 14 and Fig. 15 shows two exemplary apertures 34 which are intended for use in the Fig. 13 are suitable for the embodiment of a hyperspectral imaging assembly 18. These two apertures 34 differ in the type of aperture pattern. Fig. 14 shows a slit diaphragm and Fig. 15 shows a diaphragm in which, instead of the slot, a row 116 with several diaphragm holes 42 is provided. Due to the large number of usable diaphragms 34, the illustration of further diaphragm patterns is omitted for the sake of clarity.

[0106] In the Fig. 16 and Fig. 17 are partial steps of image generation with a Fig. 13 is illustrated by way of example. An object region 20 with a plurality of tissue structures 104 is shown. The slit diaphragm 54 or its observation slit 64 is shown in an initial position AP as well as in several displacement positions VP rotated by an angular amount.

[0107] Relatively shifted observation columns 64 are in Fig. 16 shown with solid line, dotted line and dashed line.

[0108] The aperture 34 is configured to select image strips 76 of the object area to be imaged according to the observation slit 64 and to guide their light to an image acquisition sensor 26b, which captures the spectral information of the image strips 76. The spectral information obtained for the image strips 76 can then be evaluated and, for example, superimposed on a white light image.

[0109] Fig. 17 shows an exemplary overlay representation 78. Image information based on spectrally resolved image data is superimposed on a white light image 114 in a central image area 106. By way of example, in the case shown, different tissue types are highlighted differently, with the tissue types being automatically identified based on the spectral information that can be obtained from the different image strips 76 described above. Since spectral data is only available for image area 106, only a portion of the overlay representation 78 is provided with such information.

[0110] By means of the rotary drive 58, the displacement of the slit diaphragm 54 relative to the image acquisition sensor 26b can increase the spatial resolution of the spectrally resolved image data. Fig. 16, the slit diaphragm 54 was displaced by defined angular amounts, for example. In the displaced position, deviating from the initial position, different image strips 76 are selected by the observation slit 64 than in the initial position, which are only identical at one point in the immediate vicinity around the rotation axis. Advantageously, rotating the slit diaphragm 54 or rotating the observation slit 64 increases the spatial image resolution in a central region, which can be useful since a user of an endoscope prefers to view and display the relevant object area 20 centrally in the image during a therapeutic and / or diagnostic procedure.

[0111] Fig. 18 shows a schematic representation of another embodiment of a distal end piece 22 of an endoscope device 10. The endoscope device 10 comprises an entrance optics 30 that generates an intermediate image ZB of an object region 20 to be imaged. Furthermore, the endoscope device 10 comprises a slit diaphragm 54 with an observation slit 64 that selects an image strip 76 of the intermediate image ZB and onto which the intermediate image ZB falls.

[0112] The Fig. The endoscope device 10 shown in Figure 18 also comprises an optical intermediate image displacement unit 60, which is configured to optically displace the intermediate image ZB relative to the observation slit 64 and thereby generate different image strips 76. Intermediate images ZB1, ZB2, ZB3 displaced relative to one another are shown in Fig. 18 and Fig. 19 shown with solid line, dotted line and dashed line.

[0113] As can be seen, a different image strip 76 is selected from each of these intermediate images through the observation slit 64. In order to optically displace the intermediate image ZB, the intermediate image displacement unit 60 comprises an optical element 62, which in the embodiment shown here is a four-sided prism ZBVP. The prism ZBVP is coupled in a torque-transmitting manner by means of a rotary drive 58. The prism ZBVP can be pivoted by a defined angular amount by controlling the rotary drive 58. By controlling the rotary drive 58, the intermediate image displacement unit 60, as shown in Fig. 19, can be pivoted into different positions. The rotary drive 58 is designed in particular to rotate the optical element 62 in a predetermined direction of rotation by in particular at least 90 degrees and preferably at least 360 degrees and particularly preferably continuously or oscillatory. With rotation of the optical element, the position of the intermediate image ZB also changes. The observation slit 64, the dispersive element 36, the image capture sensor system 26b, the optical intermediate image displacement unit 60 and the rotary drive 58 are components of the endoscope device 10 or the hyperspectral image capture assembly 18. Arranged behind the observation slit 64 is a collimator lens 103 which is designed to parallelize the beam path of the incident light of the image strip 76 selected with the observation slit 64.Behind the collimator lens 103, a splitting optic 36 is arranged, which is configured to spectrally split the light of the image strip along a spatial axis. Following the splitting optic 36 is a focusing optic, which directs the beam path of the image strip 76 onto a light-sensitive region 38 of the image acquisition sensor system 26b. Unlike in the embodiments described above, according to the embodiment shown in FIG. Fig. 18, the slit diaphragm 54 is not displaced relative to the image capture sensor 26b.

[0114] Fig. 19 shows a schematic detailed view of an optical intermediate image shifting unit 60 according to Fig. 18.

[0115] Fig. Figure 20 shows schematically the change in the image strip 76 as a result of a displacement of the intermediate image ZB1 to ZB3 as a result of a Fig. 19, brought about by the optical intermediate displacement unit 60. Shown is an object region 20 with a plurality of tissue structures 104. The intermediate image ZB1 represents the image strip 76 selected by the observation slit 64 of the slit diaphragm 54 when the intermediate image displacement unit 60 is in the starting position. By rotating the intermediate displacement unit 60, the intermediate images ZB2, ZB3 are shifted due to the deviating beam path in the image. The two intermediate images ZB2 and ZB3 each provide a different image strip 76, shown with dotted or dashed lines.

[0116] The spectral information obtained for these image strips 76 can then be evaluated and superimposed on a white light image 114. Fig.21 shows an exemplary overlay representation 78. Image information based on spectrally resolved image data is superimposed on a white light image 114 in a central image region 106. By way of example, in the case shown, different tissue types 104 are highlighted differently, with the tissue types being automatically identified based on the spectral information that can be obtained from the different image strips 76 described above. Since spectral data is only available for image region 106, only a portion of the overlay representation 78 is provided with such information.

[0117] In further embodiments, the intermediate image can also be shifted over its entire height so that spectral information can be obtained for the entire image. List of reference symbols 10 Endoscope device 12 shaft 14 proximal shaft section 16 distal shaft section 18 hyperspectral imaging assembly 20 Object area 21 object to be imaged 22 distal end piece 24 Camera assembly 26a first image capture sensor technology 26b second image capture sensor 28 image sensor 30a,b entrance optics 32 beam splitters 34 aperture 36 Splitting optics 38 light-sensitive area 40 pinhole 42 aperture hole 44 Aperture displacement unit 46 Actuator 48 panel supports 50 flexure joint 52 Piezo actuator 54 slit aperture 56 Dove Prism 58 rotary drive 60 optical inter-image shift unit 62 optical element 64 Observation slit 68 Cooling device 70 Endoscope 72 Medical System 74 Display generation unit 76 pixels / image strip 78 Representation 80 analysis unit 82 display unit 84 supply unit 86 fiber optic cables 88 Lighting device 90 electrical cable 92 Control device 94 Handle 96a,b mirror 98a,b coupling area 100 end face 102 focusing optics 103 Collimator lens 104 Tissue structure 106 image area 108 protective glass 110 image cells 114 White light image 116 rows of holes HL first part of incident light, SL second part of incident light, visible light d Hole row spacing d λ Splitting distance E.g. intermediate image ZBA interframe section ZBVP Prisma LA longitudinal axis AL imaging light AP starting position VP relocation position

Claims

[1] Endoscope device (10) comprising: a shaft (12) having a proximal portion (14) and a distal portion (16); and a hyperspectral image acquisition assembly (18) configured to acquire images of an object region (20) of an object (21) to be imaged and to generate hyperspectral image data comprising spatial and spectral information, wherein the hyperspectral image acquisition assembly (18) is arranged in the distal portion (16), further comprising a camera assembly (24) comprising at least one first image capture sensor system (26a) with at least one image sensor (28) of the first image capture sensor system (26a) which is configured to capture images and generate image data, further comprising at least one input optics (30) through which imaging light (AL) can be coupled, wherein the hyperspectral image acquisition assembly (18) comprises: a diaphragm (34) configured to block and transmit the light (HL) according to a diaphragm pattern; a splitting optic (36) which is designed to spectrally split light (HL) transmitted through the aperture (34); and at least a second image capture sensor system (26b) with at least one image sensor (28), wherein the second image capture sensor system (26b) defines a light-sensitive region (38) and is arranged with respect to the splitting optics (36) such that spectrally split light falls on the light-sensitive region (38); wherein the at least one input optics (30), the aperture (34), the splitting optics (36) and the second image acquisition sensor system (26b) are arranged in the distal section (16), and wherein the aperture (34) is mounted rotatably about a rotation axis which is arranged at least substantially parallel to an optical axis of the aperture (34). [2] The endoscope device (10) of claim 1, wherein the distal portion (16) comprises a distal end piece (22) and wherein the hyperspectral imaging assembly (18) is disposed in the distal end piece (22). [3] Endoscope device (10) according to claim 1 or 2, wherein the image sensor (28) of the first image acquisition sensor system (26a) is a color image sensor. [4] Endoscope device (10) according to one of the preceding claims, further comprising a beam splitter (32) arranged behind the at least one input optic (30); wherein the beam splitter (32) is configured to supply a first portion of incident light (HL) to the hyperspectral image capture assembly (18) and to supply a second portion of incident light (SL) to the camera assembly (24). [5] Endoscope device (10) according to claim 4, wherein the beam splitter (32) is wavelength-selective, and wherein the beam splitter (32) is configured to supply visible light, in particular at least in a wavelength range from 450 nm to 650 nm and preferably at least in a wavelength range from 420 nm to 700 nm, to the camera assembly (24) and to supply infrared light, in particular at least light with a wavelength of more than 1500 nm, preferably light with a wavelength of more than 1000 nm and particularly preferably light with a wavelength of more than 800 nm, to the hyperspectral image capture assembly. [6] Endoscope device (10) according to one of claims 4 or 5, wherein a beam path from the beam splitter (32) to the first image capture sensor system (26a) of the camera assembly (24) and a beam path from the beam splitter (32) to the second image capture sensor system (26b) of the hyperspectral image capture assembly (18) run parallel to each other at least in sections. [7] Endoscope device (10) according to one of the preceding claims, wherein the camera assembly (24) and the hyperspectral image acquisition assembly (18) are arranged side by side with respect to a longitudinal axis of the distal portion (16) of the shaft (12). [8] Endoscope device (10) according to one of claims 1 to 6, wherein the camera assembly (24) and the hyperspectral image acquisition assembly (18) are arranged one behind the other with respect to a longitudinal axis of the distal portion (16) of the shaft (12). [9] Endoscope device (10) according to one of the preceding claims, wherein the first and / or second image acquisition sensor system (26a,b) is stationary relative to the distal portion (16) of the shaft (12). [10] Endoscope device (10) according to one of the preceding claims, wherein the at least one input optic (30) and the splitting optic (36) are stationary relative to the distal portion (16) of the shaft (12). [11] Endoscope device (10) according to one of the preceding claims, wherein the diaphragm (34) is designed as a slit diaphragm (54). [12] Endoscope device (10) according to one of the preceding claims, wherein the hyperspectral image acquisition assembly (18) comprises a Dove prism (56) which is rotatably mounted together with the aperture (34). [13] Endoscope device (10) according to one of the preceding claims, wherein the hyperspectral image acquisition assembly (18) comprises a rotary drive (58) configured to rotate the aperture (34) and / or the Dove prism (56) about the rotation axis. [14] Endoscope device (10) according to one of the preceding claims, further comprising a cooling device (68) which is configured to cool the first and / or second image acquisition sensor system (26a,b). [15] Endoscope device (10) comprising: a shaft (12) having a proximal portion (14) and a distal portion (16); and a hyperspectral image acquisition assembly (18) configured to acquire images of an object region (20) of an object (21) to be imaged and to generate hyperspectral image data comprising spatial and spectral information, wherein the hyperspectral image acquisition assembly (18) is arranged in the distal portion (16), further comprising a camera assembly (24) comprising at least one first image capture sensor system (26a) with at least one image sensor (28) of the first image capture sensor system, which is configured to capture images and generate image data, further comprising at least one input optic (30) through which imaging light (AL) can be coupled, wherein the hyperspectral imaging assembly (18) comprises: a diaphragm (34) configured to block and transmit the light (HL) according to a diaphragm pattern; a splitting optic (36) which is designed to spectrally split light (HL, SL) transmitted through the aperture (34); and at least a second image capture sensor system (26b) with at least one image sensor (28), wherein the second image capture sensor system (26b) defines a light-sensitive region (38) and is arranged with respect to the splitting optics (36) such that spectrally split light falls on the light-sensitive region (38); wherein the at least one input optics (30), the aperture (34), the splitting optics (36) and the second image acquisition sensor (26b) are arranged in the distal section (16), and wherein the aperture (34) is designed as a pinhole aperture (40) comprising a plurality of rows of aperture holes (42). [16] Endoscope device (10) according to claim 15, wherein the rows of aperture holes (42) extend obliquely with respect to a transverse direction of the splitting optics (36) and / or the light-sensitive region (38). [17] Endoscope device (10) according to claim 15 or 16, wherein rows of aperture holes (42) are spaced apart from one another by a hole row spacing (d), wherein the hyperspectral image acquisition assembly (18) defines a resolvable wavelength range, and wherein the splitting optics (36) are configured to split the resolvable wavelength range along a spatial axis over a splitting distance (dλ) which is at most as large as the hole row spacing (d). [18] Endoscope (70) with an endoscope device (10) according to one of the preceding claims. [19] Medical system (72), comprising: an endoscope (70) according to claim 18; and a representation generation unit (74) which is configured to generate a representation (78) for a user which is based on combining information from different pixels (76a) and / or image strips (76b) to form an image sub-area. [20] The medical system (72) of claim 19, wherein the representation (78) comprises an overlay of the image sub-area and an image of the object area (20) to be imaged. [21] Medical system (72) according to claim 19 or 20, further comprising an analysis unit (80) which is configured to create an analysis based on spectral information of the image strips (76b) in accordance with image data of the first and / or second image acquisition sensor system (26a,b). [22] Medical system (72) according to claim 21, wherein the analysis comprises determining at least one physiological parameter, in particular a perfusion parameter. [23] The medical system (72) of claim 21 or 22, wherein the representation generation unit (74) is configured to generate a representation for a user based on the analysis.

Images