Imaging system and methods for observation
The stereo imaging system addresses the challenge of maintaining stereoscopic perception and image orientation during rotations by using a position sensor and control device to switch between stereoscopic and monoscopic modes, ensuring effective image orientation for instruments with small shaft diameters.
Patent Information
- Application Number
- DE102019009282
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2019-06-03
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2039-06-03
AI Technical Summary
Existing stereo imaging systems face challenges in maintaining stereoscopic perception and image orientation when the observation instrument is rotated, particularly for instruments with a small shaft diameter used in minimally invasive surgeries, due to the misalignment of the stereo base with the reference horizon during rolling movements.
A stereo imaging system with an observation instrument that includes a position sensor to detect orientation, a control device operating in first and second playback modes, allowing for stereoscopic or monoscopic reproduction based on the instrument's orientation, and implementing electronic image erection to maintain image orientation during rotations.
Enables stereoscopic perception even during significant rotations by switching between stereoscopic and monoscopic modes, ensuring intuitive operation and maintaining image orientation, suitable for instruments with small shaft diameters used in surgeries like neurosurgery.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The present disclosure relates to a stereo imaging system comprising an observation instrument having an image acquisition unit for acquiring first image data and second image data which can be combined for stereo observation.
[0002] In particular, the present disclosure relates to a medical imaging system. Such imaging systems are regularly used to observe the human and / or animal body.
[0003] Imaging systems suitable for stereoscopic observation typically comprise observation instruments with stereoscopic observation optics and stereo image sensors, for example, two image sensors offset from each other. This provides two observation channels. Image sensors typically include CCD sensors, CMOS sensors, and similar devices. When using two offset image sensors, one of the two image sensors is assigned to the observer's right eye (first observation channel) and the other of the two image sensors is assigned to the observer's left eye (second observation channel).
[0004] Imaging systems suitable for stereoscopic observation are sometimes referred to as 3D imaging systems. It should be understood that these systems do not necessarily capture 3D data. However, the 3D image generated in this way allows the observer to perceive depth.
[0005] Medical imaging systems include, for example, systems for endoscopic observation. For the purposes of the present disclosure, systems for endoscopic image processing include designs in which a probe-like instrument can be inserted into natural or artificially created body openings to capture images of the interior of the body. This also includes so-called video endoscopy systems.
[0006] DE 10 2017 219 621 A1 discloses a medical visualization system comprising an endoscope and a microscope that are aligned in different orientations and whose viewing axes can be inclined to each other. The endoscope is equipped with a motion sensor, wherein an angle of the viewing axis of the endoscope relative to the viewing axis of the microscope is determined. A display device is provided that displays a first image based on the microscope data and a second image based on the endoscope data, wherein the orientation of the second image can be adjusted depending on the angle of the viewing axis of the endoscope.
[0007] US 2018 / 0042453 A1 discloses a stereoscopic endoscopy device that has two image sensors and thus two channels. A control device is provided that is designed to detect adhering foreign objects, so that only one of the two channels is used if the other channel is impaired due to the foreign objects.
[0008] WO 2016 / 012248 A1 discloses a stereoendoscope with distally arranged image sensors. Embodiments with a straight viewing direction and embodiments with an oblique viewing direction are described.
[0009] US Pat. No. 7,108,657 B2 discloses an endoscopic visualization device with two image sensors offset from each other. It is proposed that each of the two image sensors be rotatable about an axis perpendicular to its sensor plane. This allows the captured image to be rotated. This provides an image erection function.
[0010] US Pat. No. 7,037,258 B2 discloses a video endoscope with image erection capabilities. The endoscope is designed as a monoscopic instrument. It is therefore an instrument with a single viewing channel. The image is electronically rotated or erected.
[0011] US Pat. No. 7,134,992 B2 discloses another video endoscope with means for image erection. It proposes erecting the captured image depending on a determined orientation of the instrument. The endoscope has an image sensor located at the distal end. It is a monoscopic observation instrument with a single observation channel. Acceleration sensors are used to detect the orientation.
[0012] Furthermore, the present disclosure relates to medical imaging systems designed to observe the body from outside the body. In other words, these systems do not provide for the instrument to be at least partially inserted into a body opening. Such systems, in particular their observation instruments, are referred to, for example, as so-called exoscopes. Such an exoscope with stereoscopic optics is known from US 10,122,897 B2. It is proposed to design the optical unit rotatable in order to erect the image of the instrument as needed. This allows the orientation of the stereo base of the stereoscopic optics to be adapted, so that stereoscopic observation is possible even if the orientation or position of the instrument changes.
[0013] The present disclosure relates, at least in exemplary embodiments, to imaging systems and instruments for white-light observation. This includes the detection of visible light, in particular parts of the electromagnetic spectrum that are visible to the human eye. This concerns, for example, a range from 380 nm to 750 nm (nanometers). However, this is not to be understood as limiting.
[0014] In particular, this does not preclude the possibility that an imaging system according to the disclosure may also capture image information for observation in the infrared or near-infrared range, or in the ultraviolet range. This is conceivable, for example, for instruments for PDD (photodynamic diagnostics) and / or PDT (photodynamic therapy), as well as for instruments for fluorescence observation.
[0015] However, at least in essential embodiments, this is not intended to include radiological imaging methods (e.g., radiography, computed tomography, magnetic resonance imaging, ultrasound observation). Accordingly, the focus of the present disclosure is not on imaging systems for cross-sectional image observation. Instead, essential aspects of the present disclosure relate to the observation of surfaces or near-surface objects. It is understood that combined imaging systems are nevertheless conceivable.
[0016] When using observation instruments of the type described above, it is often necessary to rotate them or move them in some other way, for example, around their longitudinal axis (axis of the instrument shaft or the observation optics), in order to change the observed image section. This is particularly the case with handheld or hand-guided instruments. However, even with rigidly mounted instruments, degrees of freedom of movement can be provided to position the instrument and its observation optics as desired.
[0017] In observation instruments with only one image sensor, the orientation of the image sensor can be adjusted with respect to a reference horizon (defined, for example, by a display device such as a monitor) by rotating the image mechanically or electronically. In this way, image erection can be performed. For the purposes of the present disclosure, the feature of image erection is to be understood as a function in which the orientation (such as the directions top, bottom, right, left) of the displayed image is largely or completely retained, even though the image sensor or the observation optics is rotated (in the sense of a rolling movement). This definition can also be applied analogously to observation instruments with an inclined viewing direction, for example, to instruments with lenses whose "normal" is not parallel to the axis of rotation. In the case of a lens, the "normal" is, for example, the optical axis.For example, for an image sensor that is aligned concentrically with the lens, the "normal" is roughly perpendicular to the sensor plane. The optical axis of the lens is usually crucial for defining the viewing direction.
[0018] However, it's also conceivable to set the image to non-upright, so that the displayed image rotates with the instrument's rotation or roll. Corresponding playback modes can be selected optionally.
[0019] In an instrument designed for stereo observation, image erection presents various challenges. The main cause is the so-called stereo base, i.e., an offset (distance or angle) between the two observation channels or image sensors combined for stereo observation and stereo playback. The stereo base is ideally aligned with the reference horizon of the playback device, ideally parallel or nearly parallel to it. The reference horizon of the playback device is usually horizontal. An observer with a straight head aligns their eyes (imaginary line through both eyes) parallel to this.
[0020] A general design goal for imaging systems in medical or industrial environments is to make the respective probe component small. This can also include miniaturizing the image acquisition unit. This applies, for example, to the diameter of the probe or the lens assembly. This design goal is even more important for endoscopic instruments used in neurosurgery (brain surgery, spine surgery, etc.). However, this complicates mechanical solutions for image erection.
[0021] In general, it is desirable to design shaft assemblies of endoscopic instruments with the smallest possible shaft diameter. This can minimize trauma to the patient during access. Therefore, this design goal generally applies to instruments used in minimally invasive surgery.
[0022] Nevertheless, even with instruments that are as small as possible (e.g., in terms of shaft diameter), there is a need for stereoscopic image capture or observation. One approach for a stereoscopic instrument, such as an endoscope, could be to arrange two image sensors at or near the distal end of the instrument. The two image sensors are regularly arranged next to each other and therefore affect the overall diameter of the instrument in this area. It would be very complex to additionally provide provisions for mechanical image erection in this distal area.
[0023] Nevertheless, such instruments are often rotated around the longitudinal axis of the shaft during use. This is especially true for instruments with an oblique viewing direction. As already indicated above, this poses challenges for stereoscopic perception, at least if the image is to remain upright even during a rolling movement. Conversely, if the upright position is omitted, orientation "in the image" becomes more difficult from the observer's perspective.
[0024] Instruments with only one image sensor, i.e., with only one observation channel (monoscopic instruments), are less problematic in this regard. The captured image can be erected both mechanically (actually by hand or via an actuator) and electronically (digital erection). Mechanical erection involves rotating the sensor relative to the shaft or a bearing piece of the instrument. Electronic image erection involves image processing measures, which can ensure that the image is displayed upright if necessary.
[0025] Against this background, the present disclosure is based on the object of specifying a stereo imaging system with an observation instrument and a method for stereo observation that allow a practically perceptible reproduction even when the instrument or the image acquisition unit of the instrument is rotated, wherein the reproduced image can be erected as needed. At the same time, a stereoscopic representation (or 3D representation) should be enabled over the largest possible rotation range. Intuitive operation is preferably possible. Preferably, the imaging system or the method still allows basic functionality in a state in which stereoscopic acquisition and reproduction is not practical, so that at least one observation channel can be used.
[0026] Furthermore, within the scope of the present disclosure, a corresponding computer program for controlling a stereo imaging system is to be specified.
[0027] This object is achieved according to a first aspect by a stereo imaging system, in particular a medical imaging system, which comprises the following: - an observation instrument with an image acquisition unit for acquiring first image data and second image data, which can be combined for stereo observation, - at least one position sensor for detecting an orientation, in particular a rotational orientation, of the instrument with respect to a position reference, - a control device operable in a first playback mode and a second playback mode depending on the orientation of the instrument, wherein the control device is designed to output an image signal which, in the first playback mode, comprises a stereo signal based on the first image data and the second image data, and, in the second playback mode, comprises a mono signal based on the first image data or the second image data, wherein the control device is designed to erect images output with the image signal in the second playback mode depending on the orientation, and wherein the control device is further operable to enable an adapted transition when changing between the first playback mode and the second playback mode.
[0028] The object of the invention is thus completely solved.
[0029] According to the invention, the capability for stereoscopic observation is utilized. This applies in particular to positions (especially rotational positions) of the image acquisition unit in which the orientation of the stereo base does not deviate from the position reference (e.g., the horizon or artificial horizon) or deviates only within acceptable limits.
[0030] However, if the orientation signals (rotational or roll orientation) provided by the position sensor indicate that the stereo functionality is no longer usable due to the rotation of the image acquisition unit, the control unit can output a mono signal (using only one viewing channel). This can be the case if the operator can no longer orient themselves clearly in the image, for example, when controlling surgical instruments visible there.
[0031] The above measure has the advantage that an image erection, in particular an electronic image erection, is enabled, whereby, depending on the current angle of rotation, a stereoscopic reproduction based on two observation channels or a monoscopic reproduction based on one observation channel is possible.
[0032] While this means that in certain orientations of the observation instrument or its image acquisition unit, only a two-dimensional representation without a sense of depth is used, this type of representation can also be used for observation.
[0033] However, if the position sensor determines that the current rotational orientation (roll position) of the image acquisition unit enables stereoscopic playback, i.e., playback with a sense of depth (3D playback), then stereoscopic playback can be switched over. Regarding image erection, it is conceivable to swap the two observation channels for playback when the image acquisition unit is rotated by approximately 180°, rotating their image information by 180°. In this way, image erection can also be achieved with stereoscopic playback, at least for a small rotation angle range (approximately 180°).
[0034] According to this approach, at least in exemplary embodiments, an instrument can be realized that enables both stereoscopic observation and electronic image erection. This refers to exemplary embodiments and is not to be understood as limiting.
[0035] Nevertheless, this approach is particularly suitable for instruments with a small shaft diameter, i.e., for instruments used in minimally invasive surgery, especially neurosurgery (brain surgery, spine surgery). In exemplary embodiments, the observation instrument is a medical observation instrument. The smaller the shaft diameter, the more difficult it becomes to implement mechanical solutions for image erection.
[0036] The observation instrument is usually designed as a video endoscope or video exoscope. A video endoscope is designed for observing the interior of the body and can be inserted through a natural or artificial opening. A video exoscope is designed for observing the body from outside the body. The instruments can be designed for human medicine and, if necessary, for veterinary medicine. Industrial applications for video endoscopes and video exoscopes are also conceivable.
[0037] The position reference to which the position signal determined by the position sensor refers can, for example, be a horizon (possibly an artificial horizon). The position reference is determined, for example, by the eye distance (or the orientation of the corresponding vector) of the observer when viewing the displayed image.
[0038] For example, the image acquisition unit comprises two image sensors spaced apart from each other. These can be parallel to each other or slightly tilted relative to each other. Each of the two image sensors (right and left) is assigned one of two observation channels. The two image sensors can be combined for stereoscopic observation of an object in an object plane. The two image sensors are spaced apart and thus adapted to the disparity of the observer's eyes.
[0039] This design is not intended to be limiting. In general, the stereo base is determined by the distal apertures of the lens unit and their distance from each other. This also determines the position of the image sensors and their distance from each other.
[0040] In the case of mono playback or 2D observation, only one of the two channels, i.e., only one of the two image sensors, is used. This doesn't necessarily mean that only one of the two image sensors actually delivers a signal. However, at least when displaying the image, only the signal from one of the two channels is used.
[0041] The orientation of the instrument is recorded via the position sensor relative to a reference, such as a horizon. This reference can be a general (global) reference or a reference defined on a case-by-case basis. Typically, a horizon is used, which reflects the position and orientation of both eyes of an observer.
[0042] In the first playback mode, the first image data and the second image data are combined to enable stereo playback (3D playback). In the second playback mode, either the first image data or the second image data is used to enable mono playback (2D playback).
[0043] At least according to exemplary embodiments, the observation instrument is a handheld or hand-guided instrument. However, this is not to be understood as limiting. According to an alternative embodiment, the observation instrument is mounted on a tripod or similar device. The orientation of the instrument, in particular its roll orientation, can be changed directly by the operator and / or via a motor-driven mechanism. The method according to the disclosure can be configured accordingly.
[0044] According to a further aspect of the present disclosure, the object of the invention is achieved by a method for stereo observation, the method comprising the following steps: - Provision of an observation instrument with an image acquisition unit for acquiring first image data and second image data, which can be combined for stereo observation, - Detecting an orientation, in particular a rotational orientation, of the instrument with respect to a position reference, and - Operating the imaging system in a first playback mode or a second playback mode depending on the orientation of the instrument, comprising: in the first playback mode, outputting an image signal comprising a stereo signal based on the first image data and the second image data, in the second playback mode, outputting an image signal comprising a mono signal, erecting the images to be output as needed depending on the detected orientation, at least in the second playback mode, and when switching between the first playback mode and the second playback mode, generating an adapted transition.
[0045] In this way, the object of the invention is also completely achieved.
[0046] The method can also be referred to as an imaging method for stereo observation. It is understood that the method can be configured according to the exemplary embodiments of the imaging system described in this disclosure, and vice versa.
[0047] In an exemplary embodiment, the method comprises providing a control device for controlling the imaging system.
[0048] In a further exemplary embodiment of the method, the method comprises providing a stereo imaging system with an observation instrument designed for stereo observation, with at least one position sensor for detecting an orientation, in particular a rotational orientation, of the instrument, and with a control device which can be operated at least in a first playback mode or in a second playback mode depending on the orientation of the instrument, wherein the control device is designed to output an image signal which, in the first playback mode, comprises a stereo signal based on the first image data and the second image data, and in the second playback mode comprises a mono signal based on the first image data or the second image data, and wherein the control device is designed to erect images output with the image signal in the second playback mode depending on the orientation.
[0049] In the first playback mode, the output images, at least in exemplary embodiments, are not erected, i.e., unerected. Accordingly, the displayed images would rotate along with the rotation angle (roll angle) of the instrument relative to the attitude reference.
[0050] According to an exemplary embodiment of the imaging system or method, the control device is configured to align output images in the second display mode such that the orientation of the displayed image relative to a display horizon does not change or changes only within defined limits. The result of the alignment is then the display of upright images.
[0051] In this way, image erection can be achieved. In the context of the present disclosure, the term image erection describes a function in which the reproduced image retains its orientation (top - bottom - right - left), even when the instrument is rotated. In particular, this refers to a so-called rolling movement, i.e. a rotation of the image acquisition unit or a lens assembly coupled to it. The rotation defines an axis for the rolling movement. For example, the rotation in an endoscope can occur around the longitudinal axis of an instrument shaft. However, flexible or partially deflectable observation instruments are also known in which no rigid longitudinal axis is installed. Furthermore, observation instruments with a shaft and an observation head mounted thereon are also known, in which the rotation does not occur around the longitudinal axis of the shaft but around an axis through the observation head.This axis, for example, is perpendicular to the longitudinal axis of the shaft.
[0052] According to a further exemplary embodiment of the imaging system or the method, the control device is designed to output images to be output in the first playback mode in an unaligned manner, so that changes in the orientation of the instrument are accompanied by changes in the orientation of the images to be output.
[0053] The term "unerected" is understood in the context of the present disclosure to mean that the images to be output are output without erection, i.e., not erected. Accordingly, if the instrument rotates 5° around its longitudinal axis, an object in the image to be output would also be rotated by 5° during playback.
[0054] It has been shown that, under certain circumstances, even slight rotation angles are acceptable in the first display mode, where the stereoscopic effect (3D effect with a sense of depth) is perceptible to the observer and orientation in the image is easy. Therefore, within the scope of this design, it is proposed to maintain stereoscopic reproduction in the first display mode within limited rotation angles (also referred to as roll angles). This makes the imaging system more tolerant of slight rotational movements.
[0055] According to a further exemplary embodiment of the imaging system or method, the control device is operable in a first rotation angle range of the instrument in the first playback mode, wherein the control device is operable in a second rotation angle range of the instrument in the second playback mode. For example, the first rotation angle range comprises at least one neutral position of the instrument with respect to the position reference. A rotation angle range is at least a defined subsection of the available rotation angle.
[0056] The neutral position includes, for example, a parallel alignment of a connecting line between the two image sensors or the apertures of the two channels with respect to the horizontal (or an artificial horizon). It goes without saying that the neutral position or initial position can also be defined on a case-by-case basis using suitable control elements.
[0057] For example, the neutral position includes an upright position (perpendicular to the stereo base) with respect to the position reference (e.g., the horizon). Starting from the neutral position, the rotation angle can be measured. A roll angle is typically an angle that describes a rotation around a longitudinal axis (of a shaft and / or the image acquisition unit).
[0058] The horizon defining the neutral position can be adjusted during operation, at least according to one exemplary embodiment. This allows, for example, global movements that involve the movement of the observation instrument. This applies, for example, to a repositioning of the patient that involves the movement of the observation instrument. Adjusting the artificial horizon can be used to largely maintain the position and orientation of other instruments in the displayed image if the patient and the observation instrument are rotated, but these other instruments are not.
[0059] Conceivable rotation angle ranges can be defined either in degrees (relative to a 360° full circle) or in hours and minutes relative to a 12 o'clock clock face. A neutral position corresponds to an orientation of 0° or 0 o'clock. A position rotated by 180° corresponds to an orientation of 180° or 6 o'clock. Intermediate positions for the rotation angle result accordingly. In an exemplary embodiment, the first rotation angle range and the second rotation angle range complement each other to form 360°.
[0060] For example, the first rotation angle range covers a range between 11 o'clock and 1 o'clock, or between 330° and 30°. Accordingly, the second rotation angle range can cover a range between 1 o'clock and 11 o'clock, or between 30° and 330°. Accordingly, the instrument would be fully rotatable around its longitudinal axis.
[0061] However, it is also conceivable to design the instrument so that it cannot rotate completely around its longitudinal axis (or to configure the image erection only for such a partial area). For example, a partial area between 9 o'clock and 3 o'clock, or 270° and 90°, is considered. Accordingly, the first rotation angle range can furthermore comprise, for example, a area between 11 o'clock and 1 o'clock, or 330° and 30°. However, the second rotation angle range then has (partial) sections between 9 o'clock and 11 o'clock and between 1 o'clock and 3 o'clock (corresponding to a first section between 270° and 330° and a second section between 30° and 90°).
[0062] According to a further exemplary embodiment of the imaging system or method, the first rotation angle range has two sections offset by 180° from each other. When the instrument is rotated by 180° from the neutral position, the image acquisition unit or the two image sensors for the first and second image data are again aligned parallel to the position reference (horizon) and rotated. The image sensors are swapped sides. Such a rolling movement of approximately 180° is generally referred to as a flip. Stereo playback is also possible in the section offset by 180°. This can also include a (static) image erection. For an image erection, the image information (first and second image data) for the first observation channel and the second observation channel would have to be swapped (swap right with left, and vice versa), and the partial images (right and left) would each have to be rotated by 180°.However, in addition to this “static” image erection, there is no continuous erection for small rotations starting from 180°.
[0063] Accordingly, the first rotation angle range comprises a range with two sections. A first section is given, for example, between 11 o'clock and 1 o'clock, or 330° and 30°. A second section is given, for example, between 5 o'clock and 7 o'clock, or 150° and 210°. Accordingly, the second rotation angle range comprises a first section and a second section. The first section comprises a range between 1 o'clock and 5 o'clock, or between 30° and 150°. The second section comprises a range between 7 o'clock and 11 o'clock, or between 210° and 330°. Stereoscopic observation and image reproduction are then possible in the range between 11 o'clock and 1 o'clock (or 330° and 30°) and between 5 o'clock and 7 o'clock (or 150° and 210°). It is understood that the above information for the first rotation angle range and the second rotation angle range is of an exemplary nature and is not to be understood as limiting.The first rotation angle range can be symmetrical to the vertical (0° position). However, non-symmetrical angle ranges with respect to the vertical are also conceivable.
[0064] According to a further exemplary embodiment of the imaging system or the method, the control device is designed to exchange the first image signal and the second image signal and to rotate the first image signal and the second image signal by approximately 180°.
[0065] This preferably occurs in the section of the first rotation angle range that is offset by 180° from the neutral position. This allows for a static image erection with 3D functionality even during a flip.
[0066] Accordingly, the first playback mode according to this embodiment comprises two operating modes. The first operating mode relates to an initial orientation in accordance with the position reference (which, for example, defines the neutral position). The second operating mode relates to the flip mode (rotation by 180° therefrom). Accordingly, the first section can also be referred to as the reference section and the second section as the flip section.
[0067] According to a further exemplary embodiment of the imaging system or method, the second rotation angle range comprises at least a position of the instrument rotated by 90° relative to the position reference. This also applies, for example, to a position rotated by 270°. In any case, within such a range (roll angle of 90° or 270°), the stereo functionality cannot be readily utilized if an upright image is desired. Therefore, it is acceptable to use only one observation channel in this range to output a 2D image. This image can then, at least in exemplary embodiments, be electronically rotated to upright the image as needed.
[0068] According to a further exemplary embodiment of the imaging system or the method, the first rotation angle range, based on an angle scale in which 0° (degrees) describes an ideal orientation of the instrument with respect to the position reference, comprises a first section which comprises a range with a first limit between 310° and 350° and a second limit between 10° and 50°, and wherein the first rotation angle range preferably has a second section which is offset by 180° relative to the first section.
[0069] For example, the first rotation angle range is arranged symmetrically with respect to a vertical (at 0°). Accordingly, the first rotation angle range, for example, covers a range between + / - 10° (plus / minus 10°) and + / - 50° with respect to the vertical, i.e., for a small range, it covers 0° + / - 10° and for a large range, 0° + / - 50°. For example, the first rotation angle range covers a range of 0° + / - 45°. In a further example, the first rotation angle range covers a range of 0° + / - 30°. Accordingly, the second section can cover a range from 180° + / - 10° up to 180° + / - 50°, for example 180° + / - 30° or 180° + / - 45°.
[0070] In absolute terms, in one embodiment, the first section of the first rotation angle range extends between 330° and 30°, with the second section of the first rotation angle range extending between 150° and 210°. The second rotation angle range, in which the instrument can be operated in the second playback mode, can extend complementarily to the first rotation angle range, for example, with a first section between 30° and 150° and a second section between 210° and 330°.
[0071] In a further embodiment, the first section of the first rotation angle range extends between 315° and 45°, with the second section of the first rotation angle range extending between 135° and 225°. Accordingly, the second rotation angle range in this embodiment comprises a first section between 45° and 135° and a second section between 225° and 315°.
[0072] In a third embodiment, the first portion of the first rotation angle range extends between 345° and 15°, while the second portion of the first rotation angle range extends between 165° and 195°. Accordingly, the second rotation angle range in this embodiment comprises a first portion between 15° and 165° and a second portion between 195° and 345°.
[0073] It is understood that other embodiments for the first rotation angle range, i.e. for the first playback mode, and the second rotation angle range, i.e. the second playback mode, are also conceivable.
[0074] The first rotation angle range, i.e., the range with stereo reproduction but without complete image erection, is selected such that the operator can still easily orient themselves in the image, which is often the case with a slight tilt. This results in the advantage of stereo reproduction. The first rotation angle range or the second rotation angle range can be defined depending on the specific application of the imaging system or the objective and subjective preferences of different operators.
[0075] It is conceivable to specify a fixed first angle range. It is also conceivable to specify a small number of variants for the first angle range (e.g., + / - 30° or + / - 45°, each relative to the vertical—at 0° and, if necessary, at 180°) from which the operator can choose. It is also conceivable to give the operator the option of freely defining the first angle range, and thus also the second angle range, within wide limits.
[0076] In general, the sections can comprise ranges of at least 355° to 5° (i.e. at least + / -5°) and of at least 175° to 185°. This includes embodiments in which the first angular range comprises values between + / - 30° and + / - 45°, in each case based on a 0° position and optionally a 180° position. However, this is not to be understood as restrictive. Other angular ranges are conceivable. If one of the two rotation angle ranges is defined, the corresponding other rotation angle range can be derived. In this embodiment, the two rotation angle ranges add up to 360°. On the full circle, each of the two rotation angle ranges can be provided twice, offset by 180°.
[0077] According to a further exemplary embodiment, the first rotation angle range comprises sections between 315° and 45° and between 135° and 225°. Thus, the second rotation angle range comprises sections between 45° and 135° and between 215° and 315°. According to a further exemplary embodiment, the first rotation angle range comprises sections between 330° and 30° and between 150° and 210°. Thus, the second rotation angle range comprises sections between 30° and 150° and between 210° and 330°. However, this is not to be understood as limiting.
[0078] The control device is operable to enable an adapted transition when changing between the first playback mode and the second playback mode. According to a further exemplary embodiment of the imaging system or the method, the transition comprises an adaptation between the orientation of the mono signal in the second playback mode and the orientation of the stereo signal in the first playback mode at a switching angle between the first playback mode and the second playback mode.
[0079] The transition (also known as transition mode) makes switching between stereo and 2D display more seamless for the operator. Image signals output in the first display mode (stereo) are usually prepared separately for the right and left eye. This can, for example, include half-images showing the object being observed from two slightly different viewing angles. When switching between the first display mode and the second display mode (and vice versa), there is a change between a display with two different half-images and a display with uniform images for the right and left eye. For example, during the transition, one of the two half-images is deliberately hidden, after which the other half-image becomes relevant for both eyes (and vice versa for the other display mode).
[0080] According to a further exemplary embodiment of the imaging system or the method, the control device is designed to align images output in the second playback mode depending on the angle of rotation, so that preferably a transition with few jumps or without jumps takes place between the first playback mode and the second playback mode.
[0081] In this way, a smooth or nearly smooth transition is enabled without a "jumping" image. Thus, there are preferably no abrupt changes in the rotational orientation of the image when the instrument is rotated and a switch occurs between the first and second display modes. In other words, at least in exemplary embodiments, a transition with little or no rotational angle jumps between the first display mode and the second display mode is enabled. It is understood that even smaller jumps may be entirely acceptable and possible. However, these jumps should not have any adverse effects on the observer's visual orientation in the displayed image.
[0082] This design takes into account the fact that, at least in exemplary embodiments, the first playback mode encompasses tolerant angular ranges (approximately 330° to 30° or even larger) in which the image in the object plane is stereoscopically captured and reproduced, whereby a strict image orientation is omitted within this (small) range. Stereo playback is generally only useful if the displayed image rotates, as it were, with the rotation of the instrument.
[0083] However, if one then leaves the first angular range in which the first playback mode is possible, an abrupt transition to the second playback mode would possibly result in an immediate alignment of the image with respect to the position reference. However, this would result in the image "jumping forward" or "jumping back" from the observer's perspective. Therefore, at least in exemplary embodiments, it is envisaged to provide a transition mode to make this transition less abrupt.
[0084] According to a further exemplary embodiment of the imaging system or the method, in the second playback mode the control device interpolates the images to be output between the unerected state and the erected state.
[0085] In this way, the transition can be made smooth and, ideally, smooth or with minimal jerking through appropriate interpolation steps. This improves perception and avoids excessive visual strain on the user during perception.
[0086] Depending on the angle of rotation of the instrument, the orientation of the output image changes, although it is still erected. Interpolation of the images (or interpolation of the orientation of the images) helps to avoid or at least reduce jumps during the transition between the first playback mode and the second playback mode. For example, interpolation occurs between an actual rotation during the transition between the first playback mode and the second playback mode and a target orientation that corresponds to the (ideally or nearly ideally) erected image.
[0087] This means that the image does not "jump" or abruptly rotate during the transition between the first playback mode and the second playback mode. Instead, the image is continuously moved or rotated. Thus, the observer perceives the change between the first playback mode and the second playback mode (which also includes a change between 3D and 2D). Furthermore, the first playback mode can be designed with sufficient tolerance for stereoscopic playback, so that the stereo effect is not only possible with absolutely precise alignment of the image acquisition unit with respect to the position reference.
[0088] According to a further exemplary embodiment of the imaging system or the method, the control device is designed to erect images to be output in the second playback mode between a switching angle which is associated with the transition between the first playback mode and the second playback mode, and in which the image in particular has an orientation corresponding to the switching angle, and a limit angle or limit angle range of the instrument in the second playback mode.
[0089] For example, the critical angle describes a range in which the reproduced image is completely erected, i.e. where the orientation of the reproduced image matches the target orientation defined by the position reference or a selected artificial horizon. For example, the critical angle is assigned to a rotational orientation of the instrument of 90° and 270°. There can therefore be two critical angles. It is also conceivable to offer complete erection of the image in a critical angle range, i.e. not just at a specific angular position. Such a critical angle range can, for example, cover a range of 90° + / - 15°. A complementary, second critical angle range can, for example, cover a range of 270° + / - 15°. This means that the reproduced image does not change its erection or only changes it insignificantly when the instrument is rotated within this range.
[0090] According to a further exemplary embodiment of the imaging system or method, the switching angle, based on an angle scale where 0° describes an ideal alignment of the instrument with respect to the position reference, is between 25° and 50°, preferably between 30° and 45°. With a symmetrical design of the first angular range, there is accordingly a further switching angle symmetrical to the vertical, for example in the range between 310° and 335°, preferably between 315° and 330°. In an exemplary embodiment, the switching angles correspond to the limits of the first rotation angle range in the first playback mode.
[0091] According to a further exemplary embodiment of the imaging system or the method, the control device is designed to rotate images to be output in the second playback mode between the critical angle or critical angle range and a further switching angle such that the orientation of the displayed image is adapted to the further switching angle when the instrument is rotated towards the further switching angle.
[0092] The further switching angle describes the transition between the second playback mode and the first playback mode with the instrument in an overhead orientation. Given the instrument's position, which is rotated at least approximately 180° for stereoscopic playback, the first image signal and the second image signal are swapped and each rotated by 180°, at least in one exemplary embodiment. The image is therefore fundamentally upright, although certain rotation angles are tolerated for robust stereo playback.
[0093] In an exemplary embodiment, the displayed image is therefore not shown in a state rotated beyond the (first) switching angle. In a first exemplary embodiment, this means that the displayed image is shown in a state rotated within + / - 45° in both the first display mode and the second display mode. In a further exemplary embodiment, the displayed image is shown in a state rotated within + / - 30° in both the first display mode and the second display mode.
[0094] According to a further exemplary embodiment of the imaging system or method, the further switching angle, based on an angle scale where 0° describes an ideal alignment of the instrument with respect to the position reference, is between 130° and 155°, preferably between 135° and 150°, with the critical angle being approximately 90°. With a symmetrical alignment of the first angular range, there is accordingly a further switching angle symmetrical to the vertical, for example in the range between 205° and 230°, preferably between 210° and 225°.
[0095] According to another exemplary embodiment of the instrument, the control device allows a change in the position reference, in particular a change from an ideal position reference, which is predetermined by the orientation of the image acquisition unit. The position reference is typically determined by the arrangement of the two apertures of the image acquisition unit. For example, the instrument is in an ideal orientation in which an observer can stereoscopically perceive an upright image if the two apertures are offset from each other in a horizontal plane (horizontal alignment of the stereo base).
[0096] Accordingly, this orientation would also be the reference for image erection in the second display mode. However, there are also known applications where an adjustment of the position reference is desired. This can be the case, for example, when the observation object (e.g., the patient) is repositioned, but at the same time, the relationship between the observation object and the (erected) image output by the observation instrument should be maintained. In this case, an artificial horizon can be defined, which results from a selected offset (offset angle) from the original horizon (e.g., in the ideal orientation).
[0097] In such an operating mode with an adjusted artificial horizon, the newly adjusted artificial horizon can now represent the new position reference for aligning the instrument. In the second playback mode, the displayed images can be erected taking the new position reference into account. It goes without saying that in the first playback mode, even if the artificial horizon is changed, the physical orientation of the image acquisition unit is not changed. When switching from the second playback mode to the first playback mode, there is a transition from the erected 2D image to the non-erected stereoscopic image. The image would therefore jump "forward" or "backward", whereby the transition can be designed to be smooth as described above. Whether the image jumps "forward" or "backward" depends on the direction of rotation.A similar situation can arise if the first playback mode includes, in addition to the first section (neutral position), a second section (flip position) offset by 180°.
[0098] For example, at least one position sensor is provided to detect the rotational position / orientation of the image acquisition unit. The position sensor detects, for example, a rolling position (rotation around the longitudinal axis) of the observation instrument and / or the image acquisition unit. In a stereo endoscope, the position sensor detects, for example, a rolling position of the instrument shaft.
[0099] Various configurations of the position sensor are conceivable. It can comprise one or more acceleration sensors. Acceleration sensors can detect relative positions. Gyroscopes or similar sensors are suitable for detecting an absolute position / orientation. It is understood that the position sensor can comprise at least one discrete sensor. In exemplary embodiments, the position sensor is arranged in the instrument. The position sensor is arranged, for example, in the shaft of the instrument, in the handle, or in the observation head.
[0100] The position sensor can be located at the proximal or distal end of the instrument. If multiple sensors are installed, they can be spatially distributed. Spatial distribution allows the position or orientation to be recorded, taking into account the relative orientation between the position sensors.
[0101] However, it is also conceivable to design the position sensor at least partially using software. The position sensor does not necessarily have to be a discrete sensor. The position sensor can also be a tracker located on or in the instrument, which is detected by an external navigation system optically, via magnetic field changes, or direct electromagnetic signal transmissions and enables the calculation of the instrument's position. The position of the recorded and displayed image can also be detected using image processing operations, particularly when movement is tracked. This can include pattern recognition, for example, so that a position is determined in the image and this position is continuously maintained during the movement of the instrument or image acquisition unit, at least in the second playback mode.
[0102] According to a further exemplary embodiment of the imaging system or method, the observation instrument is designed as an instrument with an inclined viewing direction. For the purposes of the present disclosure, an inclined viewing direction is understood to mean, for example, an inclination of an optical axis of the objective relative to a longitudinal axis (rotational axis) of the shaft.
[0103] Instruments with a viewing direction inclined by, for example, 10°, 15°, 20°, 30°, or 45° are known. Such angles of inclination are also conceivable within the scope of the present disclosure. However, this is not to be understood as limiting. At least in exemplary embodiments, the term "inclined viewing direction" does not encompass a completely lateral viewing direction (tilted 90° relative to the longitudinal axis). Such instruments are referred to as side-view instruments.
[0104] Instruments with a variable viewing direction are also known, in which the angle of inclination relative to the longitudinal axis can be adjusted. An instrument with an inclined viewing direction can also be referred to as an oblique-view instrument. The angle of inclination of the viewing direction covers, for example, ranges between 10° and 60° relative to the longitudinal axis.
[0105] Instruments with a tilted viewing direction pose particular challenges when it comes to stereoscopic observation during the rolling movement around the shaft axis. Instruments with a tilted viewing direction are often deliberately rotated around the shaft axis to vary the current field of view or to change the representation of objects in the field of view. In this way, different areas or objects of observation can be observed depending on the respective rolling angle. However, at least in exemplary embodiments, this means that certain sections of the rotational ranges can be captured stereoscopically, whereas other sections can only be captured and displayed monoscopically, at least if image erection is desired.
[0106] If, during stereoscopic observation, the individual images are individually erected, then when the instrument rotates (rolls), the orientation of the stereo base would also have to be adjusted depending on the angle of rotation to enable stereoscopic observation with both the right and left eyes. Accordingly, an additional combined internal pivoting movement of the image sensors or the lenses of the image acquisition unit would be required to adjust the stereo base to the desired image orientation during playback and the given rotational orientation of the image acquisition unit. However, this is mechanically difficult to implement. Furthermore, the tilted viewing direction complicates the composition of an erected stereo image regardless of the rotational orientation of the image acquisition unit.
[0107] According to a further exemplary embodiment of the imaging system or the method, the observation instrument carries the image acquisition unit, wherein the image acquisition unit has a stereo image sensor or two individual sensors offset from one another.
[0108] The sensors of the image acquisition unit are arranged, for example, at or near the distal end of the instrument or its shaft. According to an alternative embodiment, the sensors of the image acquisition unit are arranged at or near the proximal end of the instrument or its shaft.
[0109] According to another exemplary embodiment, the imaging system further comprises a playback unit with at least one display, in particular a 3D screen and / or 3D glasses. 3D glasses can be used, for example, in combination with a 3D screen. However, 3D glasses with their own displays are also known. Such playback units are referred to, for example, as HMDs (Head Mounted Displays). The method according to the disclosure can utilize such devices.
[0110] Preferably, the display is suitable for playback in stereo mode (3D playback) using the image data of both observation channels and also for playback in mono mode (2D playback) using the image data of only one observation channel.
[0111] The object of the present disclosure is further achieved by a computer program comprising program code configured to cause an imaging system according to one of the embodiments described herein to execute the steps of the method according to one of the embodiments described herein when the computer program is executed on the control device of the imaging system.
[0112] It is understood that the features of the invention mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.
[0113] Further features and advantages of the invention will become apparent from the following description of several preferred embodiments with reference to the drawings. They show: Fig. 1: a broken perspective rear view of an observation instrument in the form of an endoscope; Fig. 2: a broken side view of the instrument according to Fig. 1; Fig. 3: perspective front view of an observation instrument in the form of an exoscope; Fig. 4: a broken side view of the instrument according to Fig. 3; Fig. 5: a simplified schematic view of an imaging system with an observation instrument in the form of an endoscope; Fig. 6: a simplified schematic view of an imaging system with stereo functionality; Fig. 7: a frontal view of an image acquisition unit for stereo observation; Fig. 8: another view of the image acquisition unit according to Fig. 7 in a opposite Fig. 7 rotated display; Fig. 9 to Fig. 14: various simplified schematic representations to illustrate the relationship between a display mode and a current rotational orientation of an observation instrument; Fig. 15: a simplified schematic diagram illustrating the image composition in stereo playback; Fig. 16: a further illustration of the arrangement according to Fig. 15 to illustrate the image composition when rotated by 180°; Fig. 17: a further simplified schematic diagram illustrating the change between a first playback mode and a second playback mode; Fig. 18: a schematically simplified block diagram illustrating an embodiment of a method for stereo observation with an observation instrument; and Fig. 19: another schematically simplified block diagram illustrating another embodiment of a method for stereo observation with an observation instrument.
[0114] Fig. 1 illustrates, by way of a perspective view, an exemplary embodiment of an observation instrument 10 in the form of an endoscope 12. Fig. Figure 2 shows a corresponding partial side view. The observation instrument 10 is designed, for example, as a stereo endoscope 12. The endoscope 12 is designed for observing the interior of the body.
[0115] Generally, medical and non-medical (technical) applications for endoscopes 12 and similar observation instruments 10 are known. The endoscope 12 has a shaft 14 that defines a longitudinal axis 16. For example, the shaft 14 has a distal end and a proximal end. For the purposes of the present disclosure, a distal end is an end facing away from the observer (operator of the instrument) and toward the object being observed. Furthermore, for the purposes of the present disclosure, a proximal end is an end facing away from the object being observed and toward the observer (operator of the instrument).
[0116] At the proximal end of the shaft 14, the endoscope 12 has a housing 20. The operator can grasp and guide the endoscope 12 in the area of the housing 20. When used as a medical instrument 10 for observing the interior of the body, the shaft 14 of the instrument 10 can be at least partially inserted into a natural or artificial body opening.
[0117] The housing 20 includes, for example, control elements 22 and connections 24, 26. The connections 24, 26 include, for example, an electrical supply line, a signal line, a lighting connection, and connections for liquids or gases required during the medical procedure.
[0118] Fig. 2 illustrates that the instrument 10 is designed, for example, as an oblique-view endoscope 12. An image acquisition unit 30 with a viewing cone / field of view 32 is indicated at the distal end of the shaft 14. The field of view 32, or its center (axis 34), is inclined relative to the longitudinal axis 16. Angles of inclination can be, for example, 30°, 45°, 60°, and the like. Larger or smaller angles of inclination are also conceivable. Oblique-view endoscopes 12 with an adjustable viewing direction (adjustable angle of inclination) are also known. The axis 34 is designed, for example, as a normal to the sensor surface of the image acquisition unit 30.
[0119] Furthermore, Fig. In FIG. 2, a double arrow labeled 36 indicates a rotational movement of the instrument 10 about the longitudinal axis 16 of the shaft 14. Such a movement can be referred to as a rolling movement. Accordingly, the longitudinal axis 16 is a rolling axis in this exemplary embodiment. During the movement according to the arrow 36, the image acquisition unit 30, which is inclined relative to the longitudinal axis 16, is also rotated. Accordingly, such a rolling movement allows a significantly larger area of an object field to be observed overall, even if the instrument 10 is otherwise not moved.
[0120] The Fig. 3 and Fig. 4 illustrate another observation instrument 50, which in this case is designed as a so-called exoscope 52. By way of example, the instrument 50 is a stereo exoscope 52. The exoscope 52 has a shaft 54 with a longitudinal axis 56. At the proximal end of the shaft 54, the exoscope 52 has a housing 60 with operating elements 62. Furthermore, connections 64, 66 are formed on the housing 60, in particular at its proximal end. The exoscope 52 can also be held by the operator in the region of the housing 60 and guided and positioned in this way. In principle, it is also conceivable to mount the exoscope 52 on a tripod or the like. This can be a passive tripod (without the possibility of motorized adjustment) or an active manipulator (comparable to a robot, for example). However, use as a handheld / hand-guided instrument 50 is also conceivable.
[0121] At the distal end of the shaft 54, the instrument 50 has an observation head 68 with an image acquisition unit 70. In the exemplary embodiment, the image acquisition unit 70 has a field of view or a viewing cone 72, the axis of which is designated by 74. In the exemplary embodiment according to the Fig. 3 and Fig. 4, the axis 74 is aligned approximately perpendicular to the longitudinal axis 56 through the shaft 54. The axis 74 is, for example, a normal to a sensor surface of the image capture unit 70. However, this is not to be understood as limiting. The axis 74 can also be an optical axis of the optical arrangement of the image capture unit 70, regardless of how the image sensor(s) are aligned therewith.
[0122] Furthermore, in Fig. 4, a working distance is indicated by 78. It is understood that the working distance 78 does not necessarily have to be decisive in relation to the other dimensions of the instrument 50. In general, the exoscope 52 is designed according to the Fig. 3 and Fig. 4 in contrast to the endoscope 12 according to the Fig. 1 and Fig. 2 designed to observe an object (patient, technical object) from the outside (outside the body). In this respect, exoscopes are similar to other observation instruments such as microscopes.
[0123] In Fig. 4, a rotational / rolling movement of the image acquisition unit 70 is further indicated by 76. It is conceivable to design the exoscope 52 such that the image acquisition unit 70 is rotatable about the axis 74. In an oblique-view instrument, a different rotation axis may be provided. In the design according to the Fig. 3 and Fig. 4, the image acquisition unit 70 can be rotated in the observation head 68. Accordingly, the exoscope 52 is not rotated in its entirety. Rotation 76 can be provided either manually or by motor.
[0124] The working distance 78 of exoscopes is usually significantly greater than that of endoscopes. The working distance 78 can, for example, range from 100 mm (millimeters) to 500 mm. However, this is not to be understood as a limitation. Exoscopes typically also have a sufficiently large depth of field, which is significantly greater than the depth of field of endoscopes. For example, the depth of field can cover ranges of at least 5 mm, preferably at least 10 mm, more preferably at least 20 mm. However, this is also not to be understood as a limitation.
[0125] Both when observing with the endoscope 12 and the exoscope 52, it is advantageous to have a stereoscopic view. A stereoscopic view allows a sense of depth (spatial impression) and facilitates, for example, navigation with other instruments in the observed area.
[0126] Accordingly, both endoscopes 12 and exoscopes 52 with 3D functionality (in the sense of stereoscopic observation) are known. Such instruments typically have two offset image sensors or a stereo sensor with corresponding offset sensor areas. Stereo observation is enabled by two observation channels with two spaced-apart apertures, which are adapted for stereoscopic vision with the right eye and the left eye.
[0127] For stereo playback, for example, so-called 3D monitors are known, which utilize a stereoscopic effect with special aids (3D glasses). Also known are so-called HMDs (Head Mounted Displays), i.e., video glasses with a stereo effect worn by the viewer themselves.
[0128] However, stereoscopic observation and reproduction reach their limits when the instrument rolls, causing the stereo base (the imaginary line by which the two sensors / sensor surfaces are offset from each other) to shift relative to a reference horizon. This is especially the case when image erection is desired. The reference horizon is typically defined by the arrangement of a reproduction unit and ultimately by the arrangement of the human eyes (interpupillary distance, disparity).
[0129] If the stereo base of the image acquisition unit no longer corresponds at least approximately to the reference horizon, then stereoscopic reproduction – while maintaining image erection – can no longer be guaranteed with sufficiently high quality. This is especially true if attempts are made to "artificially" maintain the orientation of the output image. Stereo observation without image erection is possible, but orientation in the non-erected image becomes more difficult with increasing angle of rotation. It is therefore not simply possible to implement measures for (digital) image erection in stereo observation instruments. The 3D image is usually formed based on two observation channels. Image erection (digital / electronic or via a corresponding actuator for rotating the respective sensor) could rotate the respective individual image in the first channel and the second channel to erect it.However, in large angular ranges, there would then no longer be sufficient correspondence between the reference horizon and the position of the stereo base. The present disclosure addresses this problem.
[0130] With reference to Fig. 5 and Fig. 6, an exemplary embodiment of an imaging system 100 with an observation instrument 110 in the form of a stereo endoscope 112 is illustrated using two schematically simplified block diagrams. The endoscope 112 has a shaft 114 that defines a longitudinal axis 116. An image acquisition unit 130 is located at the distal end of the shaft 114. In the exemplary embodiment according to Fig. 5, the image acquisition unit 130 is oriented obliquely with respect to the shaft 114 such that the axis 134 is inclined relative to the longitudinal axis 116. The endoscope 112 further comprises a housing 120 at the proximal end of the shaft 114. The image acquisition unit 130 is aligned with an observation object 140. The curved double arrow 136 in Fig. 5 illustrates a rolling movement of the instrument 112 around the longitudinal axis 116.
[0131] In the exemplary embodiment, a sensor unit 142 is further accommodated in the housing 120, which, for example, has a first sensor 144. The first sensor 144 is, for example, one or more acceleration sensors. In this way, changes in position / position of the instrument 110 can be detected. Furthermore, a second sensor 146, for example, a gyroscope, is provided in the sensor unit 142. The sensors 144, 146 can be referred to as position sensors. Accordingly, the sensor unit 142 can have absolute measuring position sensors 146 and relative measuring position sensors 144. It is understood that the sensor unit 142 can also be provided at a different position on the instrument 112, for example, close to the image acquisition unit 130. Other configurations of the sensor unit 142 with the at least one position sensor 144, 146 are readily conceivable.
[0132] It is also conceivable to implement the sensor unit 142 or at least one of the position sensors 144, 146 outside the observation instrument 110. For example, monitoring the position / orientation of the instrument 110 can also be realized via external sensors (optical or electromagnetic tracking of markers, or the like). In principle, a digital implementation of the at least one position sensor is also conceivable. This can, for example, include image processing operations (pattern recognition, motion tracking, etc.). Combined designs for detecting the orientation, in particular the current roll attitude of the instrument or the image acquisition unit, are readily conceivable.
[0133] The instrument 110 is coupled via a signal line (wired or wireless) to a control device 150 having at least one control unit 152. The control device 150 can also be at least partially integrated into the instrument 110. However, the control device 150 can also be designed as a separate control device. The control device 150 can generally be designed as a central or decentralized / distributed control device. The control device 150 is configured to receive data from the instrument 110. This data is based on image information / image signals acquired with the image acquisition unit 130. The control device 150 is configured for data processing, in particular for processing and preparing image data.
[0134] An exemplary embodiment of a control device 150 is marketed by the applicant under the designation "IMAGE1 S" as a so-called camera platform. Typically, a stereo observation system comprises such a camera platform and at least one display unit (such as a 3D monitor) in addition to the instrument (endoscope or exoscope).
[0135] In the design according to Fig. 5, the control device 150 is again coupled (wirelessly or wired) via a signal line to a playback unit 160 for stereo playback. The playback unit 160 is designed, for example, as a monitor, in particular as a 3D monitor. It is understood that, under certain circumstances, additional equipment may be required to view a 3D image (stereo image), such as suitable glasses. The playback unit 160 allows playback of the observed object 140; see reference numeral 164 for the designation of the displayed image. Fig. 5, a position reference is further indicated at 166. The position reference 166 comprises at least a horizon or a coordinate system to which the displayed image 162 is oriented. With a suitable alignment between the image acquisition unit 130, the observed object 140, and the displayed image 164 on the display 162, a stereoscopic display is enabled.
[0136] It is understood that the Fig. The arrangement shown in Figure 5 can also be supplemented by a light source or other units suitable for use with the instrument 110.
[0137] Fig. 6 shows in addition to Fig. 5 shows further design features of the exemplary embodiment of the imaging system 100. The image acquisition unit 130 is formed at the distal end of the shaft 114. The image acquisition unit 130 comprises, for example, a first image sensor 180 and a second image sensor 182. In the illustrated embodiment, these are also arranged at the distal end of the shaft 114. However, this is not to be understood as limiting. Designs with proximally arranged image sensors are also conceivable.
[0138] A lens assembly 184 is arranged in front of the image sensors 180, 182. The lens assembly 184 includes, for example, a cover glass 186 and optical units 188, 190 with apertures assigned to the image sensors 180, 182. The optical units 188, 190 define the respective field of view of the image sensors 180, 182. Each of the two image sensors 180, 182 is assigned to an observation channel 194, 196. One of the two observation channels 194, 196 can be referred to as the right channel and the other as the left channel. Accordingly, one of the two image sensors 180, 182 can be referred to as the right sensor and the other as the left sensor.
[0139] In the embodiment according to Fig. 6, the term "observation channel" does not necessarily refer to an optical observation channel. Instead, channels 194, 196 are typically designed as channels for transmitting image data to control device 150, which is captured by the right and left image sensors 180, 182. Control device 150 serves to process and prepare the image data. Two output channels 200, 202 are also available at the output of control device 150, which in turn can be referred to as right and left output channels 200, 202. If image information (e.g., half-images for the right and left eye) is provided via both output channels 200, 202, a suitable display unit 160 can be controlled for stereoscopic display (3D display). Fig. 6 illustrates such a 3D rendering of the observation object 164. The observation object 164 is further aligned with respect to the position reference 166.
[0140] Fig. 7 and Fig. 8 illustrates a frontal view of an exemplary embodiment of an image acquisition unit 130 with two image sensors 180, 182. In the illustrated embodiment, the image acquisition unit 130 is accommodated in the shaft 114 of an instrument 110. However, this is not to be understood as limiting. The two image sensors 180, 182 are spaced apart from each other by a distance 216 that is adapted to the stereo basis (e.g., the distance between the apertures of the optical units 188, 190). The relative position of the two image sensors 180, 182 further defines a sensor horizon 218. If the sensor horizon 218 of the two image sensors 180, 182 coincides with the position reference 166 (in particular with the artificial horizon), an erected 3D representation is possible.
[0141] Fig. 8 illustrates a different orientation compared to the Fig. 7 rotated state of the image acquisition unit 130. Accordingly, the (new) sensor horizon 220 is rotated by a certain angle 222 relative to the old sensor horizon 218. However, if the previous sensor horizon 218 is oriented parallel to the artificial horizon of the position reference 166, this is no longer the case with the new sensor horizon 220. In addition to the rotated base 216, it is also clear that a height offset, designated 226, exists.
[0142] The twisting / skewing in Fig. 8 makes 3D representation more difficult, especially when image erection is desired, i.e. an "upright" reproduction image 164 even with a rotating image acquisition unit 130. This applies even more to instruments 110 with an oblique viewing direction (with respect to the longitudinal axis of the shaft 114, which is also, for example, the roll axis).
[0143] The present disclosure addresses these disadvantages with various display modes. Where possible, emphasis is placed on 3D display. However, if a 3D display does not seem appropriate due to the given rotational position of the image acquisition unit, a 2D display is offered to the observer. In 2D mode, (digital) image erection can be ensured. The 2D mode is suitable for applications where the orientation of the view from the observer's perspective is more important than stereoscopic observation in 3D mode.
[0144] In this context, reference is made to the Fig. 9-14. The Fig. 9-14 illustrate the currently selected rotational orientation of the observation instrument 110 by means of an angle scale 230 and a position arrow 232 drawn therein. Furthermore, the Fig. 9-14 show the resulting representation and orientation for the reproduced image 164 of the observation object at the reproduction unit 160 with respect to a position reference 166 (such as an artificial horizon), provided that the image processing is carried out with the control device 150 according to an exemplary embodiment of the present disclosure. It is understood that the representations of the tilt angles in the Fig. Figures 9-14 are merely examples. Larger, but also smaller, areas with 3D representation are conceivable.
[0145] The rotation angle scale 230 comprises various ranges 240, 242, 244, 246, of which ranges 240 and 244 are assigned, by way of example, to the first playback mode with 3D display. Ranges 242 and 246, in contrast, are assigned to the second playback mode with 2D display. The size of the respective ranges 240, 242, 244, 246 is exemplary in nature. The transition between the various playback modes occurs between ranges 240, 242, 244, and 246 at a switching angle.
[0146] The 0° position corresponds, for example, to the Fig. 7. In the 0° position, the horizon 218, which is defined by the two sensors 180, 182 or the observation optics connected upstream of them, corresponds to the artificial horizon (horizontal) of the position reference 166. This allows a 3D image of the observation object 164 to be output. Image erection is not required insofar as the observation object 164 is displayed in the desired orientation even without additional measures.
[0147] If the instrument 110 or the image acquisition unit 130 is rotated, it is possible at small rotation angles (compare Fig. 10) it is still conceivable to reproduce the observation object 164 stereoscopically (3D reproduction). There is no longer a perfect alignment between the horizon 218 and the position reference 166. The resulting deviation (compare for example Fig. 8) is, however, in the Fig. 10 is still acceptable. Accordingly, the observation object 164 can still be displayed stereoscopically.
[0148] In Fig. 10, the observation object 164 is displayed with a tilt angle in the range 240, which corresponds to the angle of the pointer 232 indicated on the scale 230. This tilt, i.e., the omission of image erection, is still acceptable within limits. The advantage lies in the preservation of the 3D functionality. However, it is also conceivable to erect the observation object 164 displayed in the image in the first playback mode (ranges 240, 244), analogous to the orientation in Fig. 9). On the one hand, this would have the advantage of image erection. On the other hand, however, it could result in further impairments in 3D display. However, within acceptable limits, this is conceivable.
[0149] In contrast, in essential embodiments, the observation object 164 is not erected, at least in the area 240. This is also conceivable in the opposite area 244 (without prejudice to the flip function that may be required).
[0150] Between the representations in Fig. 10 and Fig. 11 there is a transition from area 240 to area 242. In other words, Fig. 10 the display in the first playback mode (3D). Fig. Figure 11 shows the display in the second playback mode (2D). As already explained above, electronic image erection is easier to implement in the second playback mode.
[0151] If one were to take the image of the observation object 164 at the Fig. 11 indicated angle of rotation abruptly at the transition between the areas 240, 242, a noticeable “jump” would result, comparable to the direct consequence of the representation of the observation object 164 in Fig. 10 and the representation in Fig. 12. Instead, within the scope of the present disclosure, at least in exemplary embodiments, it is proposed to gently and not abruptly erect the image of the observation object 164, starting from the "orientation" transferred when switching from the first playback mode to the second playback mode. Such a transitional movement makes the transition between the two playback modes more pleasant for the observer. In other words, an interpolation of the displayed image between two orientations (partially rotated and upright) takes place.
[0152] Similarly, when switching from the second playback mode to the first playback mode, it is conceivable to slightly correct the image orientation in the 2D display in order to anticipate a smooth transition to the expected tilt of the displayed observation object 164 upon entering the first playback mode. Here, the image is interpolated between the upright orientation and the partially rotated orientation. See also the example below in Fig. 13 and Fig. 14 described transition.
[0153] Fig. Figure 12 illustrates that image erection is then possible within the area 242 in which the second playback mode is used. Image erection can be performed in a generally known manner for the 2D image of the observation object 164. Digital image erection is known. Furthermore, image erection by moving the image sensor is known.
[0154] The pointer 232 is in Fig. 12 at a critical angle in the range 242. In the exemplary embodiment, the critical angle is approximately 90°. In the opposite range 246, which is also assigned to the second playback mode, a further critical angle is approximately 270°. At least when the instrument has been rotated towards the critical angle, i.e., approximately by 90° or by 270°, a complete image erection is provided in the exemplary embodiment. The observation object 164 is perfectly or at least approximately aligned with respect to the position reference at the critical angle. Between the orientation at the switching angle (transition between range 240 and range 242, see also Fig. 11) and the orientation at the critical angle according to Fig. 12, a gradual, angle-dependent erection of the observation object 164 can take place.
[0155] Based on the orientation in Fig. 12 illustrates Fig. 13 shows a state in which the pointer 232 is still in the area 242, but is approaching the area 244. Thus, starting from the second playback mode, a transition to the first playback mode is imminent. Therefore, in Fig. 13 the orientation of the reproduced observation object 164 is already adapted to the expected orientation when entering the first playback mode.
[0156] Fig. Figure 14 shows such a state. The instrument 110 or the image acquisition unit 130 is rotated by almost 180°. Nevertheless, a 3D representation is now possible again. Furthermore, the displayed observation object 164 is at least vertically correct with respect to the original arrangement ( Fig. 9). Based on the given orientation of the instrument or the image acquisition unit 130, Fig. 14 a so-called image flip takes place, comprising on the one hand a rotation of the individual images by 180° and on the other hand an exchange of the two channels.
[0157] Overall, the observer is presented with an easily comprehensible image with a clear orientation, even when the instrument or image acquisition unit rolls. The 3D display can be used whenever it appears possible under the given conditions. Automatic switching between 3D and 2D is possible. Manual switching is also conceivable, at least as an additional option.
[0158] It is understood that the sequence of representations in Fig. 9 to Fig. 14 concerns an instrument with a straight line of sight. However, the above statements are also applicable to instruments with an oblique / tilted line of sight. It is understood that with instruments with an oblique line of sight, the field of view shifts in such a way that, as the instrument rotates, various observation objects appear in the field of view. Regarding alignment, however, the above statements are also applicable. The benefit becomes apparent at the latest when additional instruments (forceps, tweezers, and the like) are used in addition to the observation instrument and appear at least partially in the field of view.
[0159] Based on the schematically simplified representation of the Fig. 15 and Fig. 16 describes the image flip mentioned above when the instrument 110 is rotated by 180°. In Fig. 15 and Fig. 16, the instrument 110 is in one of the areas 240, 244, in which a 3D rendering is useful and desired. Fig. 15 there is a condition which is approximately the same as in Fig. 9. Each of the two sensors 180, 182 captures an image for the two channels 194, 196. The control device 150 comprises a block that schematically assigns the two channels 194, 196 to the output channels 200, 202. In the configuration according to Fig. 15, the image captured by the two sensors 180, 182 already has the desired orientation. Accordingly, only the signal from the two sensors 180, 182 needs to be combined to enable a 3D representation of the reproduced observation object 164.
[0160] In contrast, the two sensors 180, 182 capture the image according to the configuration in Fig. 16 is upside down. However, if the image data of both channels 194, 196 were simply rotated (compare blocks 250, 252), the first channel and the second channel would still be swapped. Therefore, in the embodiment according to Fig. 16 proposes, on the one hand, rotating the two (partial) images of the two sensors 180, 182 in order to erect the respective individual image. Furthermore, in this exemplary embodiment, the signal from the two channels 194, 196 is swapped (crossed) during processing and forwarding to the output channels 200, 202, as exemplified by block 254. In this way, the image flip (also referred to as a 180° flip) is realized. In area 244, an approximately erected representation in 3D mode is possible, even if the instrument 110 or the image acquisition unit 130 is rotated by 180°. A 3D representation is also desired near the 180° position.
[0161] It is understood that blocks 250, 252, 254 may be functional blocks of the control device 150. Blocks 250, 252, 254 may be implemented in software and / or hardware. Blocks 250, 252, 254 may be designed as discrete blocks with specific individual functions or as universal functional blocks.
[0162] The Fig. The functionality of the instrument 110 illustrated in Figures 9-16 can be controlled by the control device 150 in an associated operating mode. In an exemplary embodiment, the instrument 110 is also capable of other operating modes with the control device 150.
[0163] A first operating mode, for example, comprises operation of the instrument 110 in a pure 2D mode, i.e., without stereoscopic reproduction, wherein image erection is also dispensed with. The reproduced image will therefore rotate analogously to the rotation of the instrument 110 or the image acquisition unit. A second operating mode, for example, comprises operation of the instrument 110 in a stereo mode with stereoscopic reproduction. This entails that image erection is dispensed with.
[0164] A third operating mode involves operating the instrument 110 in 2D mode, continuously leveling with respect to a reference horizon. Ideally, therefore, the rotational orientation of the displayed image does not change when the instrument is rotated. For the sake of completeness, it should be noted that with oblique-view instruments, the image content (field of view) changes when the instrument is rotated. However, the displayed image retains its rotational orientation. The fourth operating mode is combined operation with stereo playback (first playback mode) and 2D playback (second playback mode), wherein at least in the second playback mode, the image is at least partially leveled.
[0165] With reference to Fig. 17 is based on an angle scale 330, which in principle corresponds to the angle scale 230 according to the Fig. 9-16, illustrates the transition between the first playback mode and the second playback mode, as well as a conceivable image transformation (interpolation) in the second playback mode as a component of the image erection. The angle scale 330 represents a full circle, which corresponds to a complete rotation around the roll axis. Partial areas 340, 344 are provided, which are assigned to the first playback mode. Furthermore, partial areas 342, 346 are provided, which are assigned to the second playback mode. Switching angles 350, 352, 354, 356 are assigned to the respective transitions. The switching angles 350, 352, 354, 356 are, for example, 45°, 135°, 225°, and 315°. In an alternative, in Fig. In the embodiment not shown in Figure 17, the switching angles 350, 352, 354, 356 are 30°, 150°, 210° and 330°.
[0166] The pointer 360 illustrates the rotational position of the instrument 110 or its image acquisition unit 130 during rotation in the first playback mode (areas 340, 344) and in the second playback mode (areas 342, 346). Fig. 17 also shows limit angles 362, 364 in the respective sub-areas 342, 346 of the second playback mode. For example, limit angles 362, 364 are each centered in sub-areas 342, 346. Thus, limit angle 362 is 90°, for example. Limit angle 364 is 270°, for example.
[0167] Fig. 17 further contains a corresponding diagram 370, which comprises a developed view of the angle scale 330. An axis 372 describes the current angle of rotation / roll of the instrument 110 or its image acquisition unit 130 and thus corresponds to the respective position of the pointer 360 during rotation. Another axis 374 describes a resulting tilt angle, i.e., a resulting rotational orientation of the displayed image.
[0168] In the embodiment according to Fig. 17, the sub-areas 340, 344 cover, for example, ranges of 0° + / - 45° and 180° + / - 45°. Stereo reproduction is conceivable in these ranges; the instrument can be operated in the first reproduction mode. In the first reproduction mode, no comprehensive electronic image erection is provided. Accordingly, the tilt of the displayed image relative to the position reference (artificial horizon) develops proportionally or even directly proportionally to the angle of rotation of the instrument 110 or its image acquisition unit 130. This is illustrated in diagram 370 by the line labeled 380; compare the sections between 0° and 45°, between 135° and 225°, and between 315° and 360°. Therefore, if the instrument 110 is rotated by 30° relative to the position reference, the displayed image is also rotated by 30°.
[0169] In the section between 135° and 225°, the instrument 110 is essentially turned upside down in its rotational orientation, so that the reproduced image is analogous to that in Fig. 16 can be adapted to enable a static quasi-erection. The image of the two observation channels is then also turned upside down, so that the output image has at least approximately the desired orientation.
[0170] In the remaining sub-areas 342, 346, however, a 2D representation based on an observation channel is used; compare the sections between 45° and 135° and between 225° and 315°. Electronic image erection is possible in the second display mode. This allows the orientation of the displayed image to be adjusted or decoupled from the current rotation angle of the instrument. In exemplary embodiments, however, the output image in the second display mode is not immediately and constantly erected to an ideal orientation. Such an ideal orientation corresponds, for example, to the 0° position on axis 374 in diagram 370. Such a function would result in the image abruptly rotating immediately upon passing one of the switching angles 350, 352, 354, 356—in the exemplary embodiment, by 45° with respect to the 0° deflection.This can be perceived as disadvantageous, especially when operating near the respective switching angle 350, 352, 354, 356.
[0171] In order to counteract such "jumping" of the reproduced image, it is proposed to interpolate the orientation of the image in the second reproduction mode depending on the angle, so that on the one hand a sufficiently stable erection and good orientation in the image is possible, and on the other hand jumps in the angle of rotation are minimized or avoided.
[0172] Curves 382, 384, 386 illustrate exemplary angular orientations of the displayed image as a function of the actual rotational orientation of the instrument 110 or the image acquisition unit 130 in the second display mode. These curves / lines 382, 384, 386 are arranged in the sub-areas 342, 346 of the second display mode. In the second display mode, curves 382, 384, 386 provide a transition between the spaced-apart sub-areas 340, 344 of the first display mode, in particular without a rotation angle jump.
[0173] Curve 382 behaves essentially proportionally or inversely proportionally to the current rotation angle of the instrument 110. For example, in diagram 370, a linear section extends between the 45° position at the switching angle 350 and the 135° position at the switching angle 352. As the rotation angle of the instrument 110 increases, the displayed image rotates in the opposite direction. The situation is similar in the exemplary embodiment between the switching angles 354 and 356, i.e., between 225° and 315°. A boundary condition in the exemplary embodiment according to Fig. 17 is the passage through the 0° position at the limit angles 362, 364, i.e., at 90° and 270° of the instrument's rotational position. In this way, with the symmetrical, particularly mirror-symmetrical, design of the angular ranges 340, 342, 344, 346, approximately centrally in the respective sub-range 342, 346 of the second display mode, a complete erection of the displayed image is achieved.
[0174] Curve 384 is fundamentally based on the curve of curve 382. However, the curve of curve 384 pursues the goal of fully erecting the image not only directly at the critical angles 362 and 364, but also in their vicinity (in the example, approximately + / - 20°). This creates a range in which the displayed image is sufficiently stable and is not rotated or rotated only slightly. Nevertheless, curve 384 contains ramps that allow a smooth transition to the first playback mode.
[0175] Curve 386 is designed as a spline (polynomial curve), with the basic curve following curve 384. This way, "kinks" can be avoided when passing through the curve.
[0176] It is understood that the curves 382, 384, 386 for the second playback mode can be combined with the curve 380 for the first playback mode to implement the desired behavior in the first and second playback modes.
[0177] With reference to Fig. 18 illustrates an exemplary embodiment of a method for stereo observation, in particular a method for stereo observation with image erection, at least with partial image erection, using a schematic block diagram.
[0178] The method comprises a first step S10, which concerns the provision of an observation instrument with stereo functionality. The instrument can be a stereo endoscope or a stereo exoscope, for example. The instrument is typically provided with an image acquisition unit that can acquire first image data and second image data. For this purpose, the image acquisition unit can have a first sensor and a second sensor offset therefrom. In this way, two observation channels (right and left) are formed. In this way, the first image data and the second image data can be combined for stereo observation. However, this presents challenges with regard to the desired image erection when the instrument is rotated, so that the stereo base of the instrument changes relative to an (ideal) reference horizon.
[0179] This is followed by step S12, which involves monitoring the position or detecting the angular position (roll position) of the instrument or its image acquisition unit. The detected position or rotation allows us to determine whether the given orientation of the image acquisition unit makes a 3D display without image erection appropriate, or whether switching to a 2D display using only one of the two observation channels is necessary to utilize electronic image erection.
[0180] Depending on the detected angle of rotation, a step S14 may follow, in which a first playback mode is activated. The first playback mode includes 3D playback. Alternatively, a step S16 may follow, in which a second playback mode is activated. The second playback mode includes 2D playback.
[0181] Step S18 follows a step S18, which includes a playback using both observation channels (right and left) for 3D playback with a sense of depth. At least in exemplary embodiments of the method, the displayed image of the observation object is not continuously erected in the first playback mode. This causes the displayed image to rotate along with the rotation of the instrument. However, since the first playback mode can only be used effectively within limited rotation angle ranges, the rough image alignment remains correct, allowing orientation within the image. Nevertheless, the first playback mode offers the advantage of stereoscopic observation.
[0182] Typically, the first playback mode also includes a state in which the instrument or its image acquisition unit is rotated by approximately 180°. This is because the stereo base (defined, for example, by the position or orientation of the apertures of the observation optics) is then aligned parallel or nearly parallel to the reference horizon. However, to ensure that the displayed image is not upside down, a so-called 180° flip is performed. This includes, for example, a 180° rotation of the two channels and a swap (right with left, and vice versa). In this way, a quasi-erection in the 180° rotated state can be realized together with the 3D display. However, there is no continuous, tightly tracked image erection.
[0183] However, if the method according to step S16 is executed in the second playback mode, only one of the two monitoring channels is provided or used for playback in the subsequent step S20. This has the advantage that the stereo basis no longer needs to be taken into account, since, in a sense, only a mono signal is processed for output.
[0184] Accordingly, the image to be output can be digitally / electronically erected. Image erection occurs in step S22. Depending on the continuously detected rotation angle position, tracking or continuous image erection can now be ensured. For the observer, the general orientation of the image does not change, or only changes within defined limits, when the instrument is rotated.
[0185] The method concludes in step S24 with the reproduction of the image, in the exemplary embodiment either a 3D reproduction without immediate image erection, except in the case of an upside-down orientation of the instrument, or a 2D reproduction with image erection.
[0186] With reference to Fig. Figure 19 describes an exemplary embodiment of a method for stereoscopic observation using a further block diagram. The method can also be designed and described as a method for controlling an imaging system.
[0187] The method begins with step S50, which, for example, involves activating the imaging system. This is followed by step S52, which involves detecting a current rotational (rolling) position of an instrument or its image acquisition unit. At least one corresponding sensor can be provided for this purpose. Detection allows monitoring of the detected angle in step S54. Step S56 determines which of three (global) angular ranges the instrument or its image acquisition unit is currently located in. For example, a position relative to a reference horizon is used, whereby the stereo base can be used as an instrument-internal reference for this purpose.
[0188] Depending on which angular range is present, the instrument can subsequently be operated in a first playback mode (step S58), a second playback mode (step S60), or a third playback mode (step S62), also referred to as transition mode.
[0189] Step S58 aims at 3D playback using both image channels. Step S60 aims at 2D playback using only one image channel and preferably also at continuous image erection. Step S62 aims at enabling a transition between the first playback mode and the second playback mode.
[0190] Step S58 is followed by a step S64, which includes image manipulations adapted to the first playback mode for 3D observation. Step S64 includes, for example, a so-called 180° flip, which takes into account a 180° rotation of the instrument. If the instrument is rotated by approximately 180° with a stereo base, 3D observation is generally possible again. In order for the image to appear upright to the observer, image manipulations are required, which are performed in step S64. At least in one exemplary embodiment of the method, apart from the possible 180° flip, there is no provision for digitally erecting the image in the first playback mode.
[0191] Step S60 is followed by a step S66, which includes image manipulations adapted to the second playback mode. The image is prepared for 2D playback. At least in some embodiments, this includes continuous image erection, so that the displayed image appears upright to the observer, even if the instrument is rotated. The image erection can have a static target, i.e., precisely one target orientation. However, the image erection can also be dependent on the angle of rotation, particularly to avoid large jumps when switching between the first playback mode and the second playback mode.
[0192] Step S62 is followed by step S68, which includes image manipulations adapted to the transition mode. The transition mode primarily serves to enable a smooth transition between the first playback mode and the second playback mode. For example, the transition occurs between the display of two slightly offset fields (stereo) in the first playback mode and the display of just one image (2D), i.e. the data from just one observation channel, in the second playback mode. It is conceivable to fade in or fade out one of the two fields in stereo mode for a specific period of time (e.g., a defined number of individual images), so that the other of the two fields becomes the dominant image. In this way, the transition between 2D and 3D is smoothed. This avoids sudden changes between successive individual images.
[0193] In step S68, the orientation is adjusted or modified so that the image changes smoothly between the different modes for the observer, whereby instead of a "jump", a smooth transformation / rotation based on interpolations is visible, which is perceived as more pleasant when viewed.
[0194] In the subsequent step S70, playback takes place taking the respective mode into account. Monitoring (step S54) occurs continuously during instrument operation if a corresponding operating mode is selected. Transition mode S62 allows a visually pleasing transition between the first playback mode according to step S58 and the second playback mode according to step S60.
[0195] Step S72 concludes the method and includes, for example, deactivation of the imaging system.
Claims
[1] Stereo imaging system, in particular medical imaging system, comprising: - an observation instrument (10, 50, 110) with an image acquisition unit (30, 70, 130) for acquiring first image data and second image data, which can be combined for stereo observation, - at least one position sensor (144, 146) for detecting an orientation, in particular a rotational orientation, of the instrument (10, 50, 110) with respect to a position reference (166), and - a control device (150), characterized by , that the control device (150) is operable in a first playback mode and a second playback mode depending on the orientation of the instrument (10, 50, 110), the control device (150) is designed to output an image signal which, in the first playback mode, comprises a stereo signal based on the first image data and the second image data, and, in the second playback mode, comprises a mono signal based on the first image data or the second image data, the control device (150) is designed to erect images output with the image signal in the second playback mode depending on the orientation, and the control device (150) is further operable to enable an adapted transition when changing between the first playback mode and the second playback mode. [2] Imaging system according to claim 1, wherein the control device (150) is adapted to align output images in the second display mode so that the orientation of the displayed image with respect to a display horizon does not change or changes within defined limits. [3] Imaging system according to claim 1 or 2, wherein the control device (150) is designed to output images to be output in the first playback mode in an unerected manner, so that changes in the orientation of the instrument (10, 50, 110) are accompanied by changes in the orientation of the images to be output. [4] Imaging system according to one of claims 1 to 3, wherein the control device (150) is operable in a first rotation angle range of the instrument (10, 50, 110) in the first playback mode, and wherein the control device (150) is operable in a second rotation angle range of the instrument (10, 50, 110) in the second playback mode. [5] The imaging system of claim 4, wherein the first rotation angle range comprises two sections offset by 180° from each other. [6] The imaging system according to claim 5, wherein the control device (150) is configured to swap the first image signal and the second image signal and to rotate the first image signal and the second image signal by approximately 180°. [7] Imaging system according to one of claims 4 to 6, wherein the second rotation angle range comprises at least one position of the instrument (10, 50, 110) rotated by 90° relative to the position reference (166). [8] Imaging system according to one of claims 4 to 7, wherein the first rotation angle range, based on an angle scale in which 0° describes an ideal orientation of the instrument (10, 50, 110) with respect to the position reference (166), comprises a first section which comprises a range with a first limit between 310° and 350° and a second limit between 10° and 50°, and wherein the first rotation angle range preferably has a second section which is offset by 180° from the first section. [9] The imaging system of any one of claims 1 to 8, wherein the transition comprises an adjustment between the orientation of the mono signal in the second playback mode and the orientation of the stereo signal in the first playback mode at a switching angle between the first playback mode and the second playback mode. [10] Imaging system according to one of claims 1 to 9, wherein the control device (150) is designed to align images output in the second playback mode as a function of the angle of rotation, so that preferably a transition with little or no jumps takes place between the first playback mode and the second playback mode. [11] An imaging system according to claim 10, wherein in the second reproduction mode, the control device (150) interpolates the images to be output between the unerected state and the erected state depending on the angle of rotation. [12] Imaging system according to claim 10 or 11, wherein the control device (150) is designed to erect images to be output in the second playback mode between a switching angle which is associated with the transition between the first playback mode and the second playback mode, and in which the image in particular has an orientation corresponding to the switching angle, and a limit angle or limit angle range of the instrument in the second playback mode. [13] Imaging system according to claim 12, wherein the switching angle, based on an angle scale where 0° describes an ideal orientation of the instrument (10, 50, 110) with respect to the position reference (166), is 25° to 50°, preferably 30° to 45°. [14] Imaging system according to claim 12 or 13, wherein the control device (150) is designed to rotate images to be output in the second display mode between the limit angle or limit angle range and a further switching angle such that the orientation of the displayed image is adapted to the further switching angle when the instrument is rotated towards the further switching angle. [15] Imaging system according to claim 14, wherein the further switching angle, based on an angle scale where 0° describes an ideal orientation of the instrument (10, 50, 110) with respect to the position reference (166), is 130° to 155°, preferably 135° to 150°, and wherein the limit angle is 90°. [16] Imaging system according to one of claims 1 to 15, wherein the observation instrument (10, 50, 110) is designed as an instrument with an inclined viewing direction. [17] Imaging system according to one of claims 1 to 16, wherein the observation instrument (10, 50, 110) carries the image acquisition unit (30, 70, 130), and wherein the image acquisition unit (30, 70, 130) comprises a stereo image sensor or two mutually offset individual sensors (180, 182). [18] Method for stereo observation, comprising the following steps: - Providing an observation instrument (10, 50, 110) with an image acquisition unit (30, 70, 130) for acquiring first image data and second image data, which can be combined for stereo observation, - detecting an orientation, in particular a rotational orientation, of the instrument (10, 50, 110) with respect to a position reference (166), and - Operating the imaging system (100) in a first playback mode or a second playback mode depending on the orientation of the instrument (10, 50, 110), comprising: in the first playback mode, outputting an image signal comprising a stereo signal based on the first image data and the second image data, In the second playback mode, output of an image signal that includes a mono signal, Erecting the images to be output depending on the detected orientation, at least in the second playback mode, and when switching between the first playback mode and the second playback mode, generating an adjusted transition. [19] A computer program comprising program code configured to cause an imaging system according to any one of claims 1 to 17 to perform the steps of the method according to claim 18 when the computer program is executed on the control device (150) of the imaging system (100).
Citation Information
Patent Citations
Device for recording an image of an object field on a human or animal body
US10122897B2
Image orientation for endoscopic video displays
US7037258B2
Endoscopic visualization apparatus with different imaging systems
US7108657B2
Gravity referenced endoscopic image orientation
US7134992B2
Stereo video endoscope having a plurality of optical modules
WO2016012248A1
