Imaging device with extended zoom function and focus tracking

Abstract

The invention relates to an imaging device (100) for visually scalable display of a region to be observed (101), in particular a surgical region, comprising: an optical image detection device (102) for detecting a recorded image of the region to be observed (101); an optical and / or electronic zoom mechanism (108, 109) associated with the image detection device (102) for zooming the detected recorded image, and an adjustable focusing device (112) associated with the image detection device; a robotic holding arm (103) for moving the image detection device (102) relative to the area to be observed (101) wherein the holding arm (103) provides a mechanical zoom function (110) by adjusting an axial distance (d) between the image detection device (102) and the area to be observed (101); an input unit (106) for detecting a user-side input command for scaling the recorded image on the display unit (105); a control device (115) for adjusting the zoom mechanism (108, 109) of the image detection device (102) and the mechanical zoom function (110) of the holding arm, and for adjusting the focusing device (112), the control device (115) performing the adjustment of the mechanical zoom function (110) and the focusing device (112) in such a way that the control device (115) performs the adjustment of the mechanical zoom function (110) and the focusing device (112). The holding arm (103) and the focusing device (112) are controlled only on the basis of an input command for scaling the recorded image detected by the input unit (106).

Description

Technical Field

[0001] The invention relates to an imaging device and a method for visually scalable display of a region to be observed, in particular an operating region, with an extended zoom function and an optimized focus adjustment. Background Art

[0002] Endoscopes and exoscopes for diagnosing or performing surgical operations on patients are well known in the prior art. An endoscope or exoscope is often arranged on a motorized holding arm, which is placed by the user so that the camera unit of the endoscope or exoscope appropriately displays the area to be detected, usually the surgical site, to the treating surgeon. The detected image is then output as a real-time image on a suitable imaging unit, such as a screen. The distance between the holding arm and the patient is usually selected in such a way that, on the one hand, the portion of the image shown is suitable for the surgeon while ensuring good access to the patient. Once a suitable position and orientation of the holding arm and of the endoscope or exoscope arranged thereon has been found, the focus of the camera unit is usually set manually.

[0003] Furthermore, in the prior art, the corresponding camera unit of an endoscope or exoscope is equipped with a zoom mechanism, in particular an optical zoom or an electronic zoom (or digital zoom), which the user can use before and / or during treatment or surgery to zoom in or out the image of the inspected area as required. In order to extend the zoom range, in addition to the zoom mechanism of the camera unit, it is also known to provide a mechanical zoom function by changing the position of a holding arm along the optical axis of the endoscope and thus by changing the distance between the camera unit and the inspected area.

[0004] However, readjustment or tracking of the focus setting of the camera unit is a challenge, since, in particular for microscope lenses, the depth of field is very small, so that even small changes in the position of the holding arm relative to the inspected area or the surgical site can lead to blurring. Thus, for example, providing continuous autofocus carries the risk of undesired fluctuations in image sharpness or of focusing on undesired sub-areas within the entire inspected area.

[0005] Manual or user-triggered focus adjustments are also disadvantageous because during surgery or treatment the user typically has both hands on the instrument in the field and does not desire additional manual manipulation of the device.

[0006] For example, US 11,033,338 B2 discloses an imaging system comprising an endoscope having a camera for image detection and a display connected to the camera for displaying the detected image area in an enlarged manner during endoscopic surgery, wherein the endoscope is arranged on a movable holding arm. The system has a control device for controlling the camera on the endoscope and a control device for controlling the holding arm, wherein the camera is configured to provide optical and electronic zoom. In addition to the optical and electronic zoom functions of the camera, a mechanical zoom function is provided by changing the position of the holding arm along the optical axis of the endoscope. An autofocus device is used to adjust the focus setting of the camera accordingly, which brings the above-mentioned disadvantages.

[0007] US 8,715,167 B1 discloses a remote surgery system for minimally invasive surgery, which includes an input device and a robot system connected to the input device, the robot system having: a first manipulator with an endoscope fixed thereon for optical image detection, the endoscope being connected to an input console of the remote surgery system for image display; and a second manipulator, the second manipulator being configured to hold and guide surgical instruments, and the second manipulator being connected to an input instrument on the input console. The remote surgery system is configured to change the focus and zoom factor in response to the movement of the robot system, for example, so as to maintain the focus of the image detection device when the distance to the object to be detected changes. Here, the robot system includes a plurality of sensors that monitor the movement of the robot system elements, determine the distance change relative to the initial focus in the coordinate system of the robot system, and adjust the focus setting of the image detection device based on this. On the one hand, although this can achieve accurate position detection and adjust the focus setting based on this, on the other hand, this requires a large number of sensors on the robot system and relevant information monitoring of each of these sensors, which not only requires increased hardware costs, but also increases the possibility of errors in the focus setting process directly related thereto. Summary of the invention

[0008] The object of the present invention is therefore to at least partially overcome the above-mentioned disadvantages of the prior art. In particular, the object of the present invention is to provide an improved imaging device and a corresponding method, which imaging device or method enables scalable imaging of the surgical area with an extended zoom function while providing a simple and effective tracking of the image clarity of the image.

[0009] The above object is achieved by a device having the features of independent claim 1. Furthermore, the object is achieved by a method according to the invention having the features of independent claim 16.

[0010] Advantageous developments of the invention are given in the dependent claims. All combinations of at least two features disclosed in the description, the claims and / or the drawings fall within the scope of the invention. It is to be understood that the embodiments and implementations described with respect to the device according to claim 1 may relate in an equivalent, although not verbatim, form to the method according to claim 16 without explicit mention to this end. It is also to be understood that customary conversions of the corresponding terms within the framework of usual language practice and / or substitutions of meanings in accordance with generally recognized language literature, in particular the use of synonyms supported by generally recognized language literature, are also included in the present disclosure without explicit mention in their corresponding expressions.

[0011] According to a first aspect of the present invention, an imaging device for visually zooming and displaying an area to be observed, in particular an operating area, is proposed. The device comprises: an optical image detection device for detecting a recorded image of the area to be observed; an optical and / or electronic zoom mechanism assigned to the image detection device for zooming the detected recorded image; and an adjustable focusing device assigned to the image detection device; a robot holding arm for moving the image detection device relative to the area to be observed, wherein the holding arm is configured to provide a mechanical zoom function by adjusting the axial distance between the image detection device and the area to be observed; an input unit for detecting an input command from the user side for zooming the recorded image on a display unit; a control device for adjusting the zoom mechanism of the image detection device and the mechanical zoom function of the holding arm; and for adjusting the focusing device. According to the present invention, the control device performs the adjustment of the mechanical zoom function and the focusing device in the following manner, that is, the holding arm and the focusing device are controlled or manipulated only based on the input command for zooming the recorded image detected by the input unit, in particular the detected target value for zooming.

[0012] According to a second aspect of the present invention, a method for visually zooming and displaying an area to be observed, in particular an operating area, is proposed. The method comprises at least the following steps: using an optical image detection device to detect a recorded image of the area to be observed for display on a display unit; manually and / or automatically adjusting a focusing device assigned to the image detection device to initially set the focus of the detected recorded image; using an input unit to detect an input command for zooming the recorded image; zooming the recorded image on the display unit by adjusting an optical and / or electronic zoom mechanism assigned to the image detection device, and adjusting the mechanical zoom function by moving the image detection device along the optical axis of the image detection device with the aid of a robot holding arm; focusing the zoomed recorded image by adjusting the focusing device.

[0013] In contrast to the prior art, according to the invention, the mechanical zoom function is regulated by moving the holding arm and the focusing device in order to track the focus or image sharpness of the recorded image by controlling the focusing device solely on the basis of the detected input command for zooming the recorded image. This means that, in contrast to the regulation of the holding arm movement and the corresponding focus tracking known in the prior art, no further input variables, in particular no actual values ​​of position data or movement data for the holding arm, are detected as feedback values ​​for regulating the mechanical zoom function and the focusing device on this basis. In other words, the present control or manipulation of the mechanical zoom function and the focusing device is carried out without or independently of the detection of feedback values, in particular without the need to detect further input variables or input values, in particular without the need to detect actual values ​​of position data or movement data for the holding arm, which can be detected, for example, by a corresponding sensor system.

[0014] According to the invention, an improved imaging device and a corresponding method are thus provided, which enable scalable imaging of the surgical field with an extended zoom function or zoom function and at the same time allow simple and effective or efficient control of the system components and in particular regulation of the image sharpness of the holding arm and tracking imaging. A significantly faster and sufficiently accurate tracking of the focus point setting can be performed based on measured input variables, without having to provide regulation based on the determination of actual values ​​of position data or movement data, in particular of the holding arm.

[0015] This is independent of any internal control of the robot holding arm or focusing unit, which can also measure and compare their own actual position and target position during the movement for safety reasons. However, this information is not used to control other components.

[0016] In a preferred embodiment, the control device is designed to control or actuate the zoom mechanism of the image capture device, the mechanical zoom function of the holding arm, and the focusing device solely on the basis of input commands detected by the input unit for zooming the recorded image, and in particular without reading out or evaluating position data or movement data of the robot holding arm for providing control variables. The zoom mechanism of the robot holding arm and the focusing device are both actuated solely on the basis of the detected input commands and therefore not on the basis of other control values, in particular without detecting position data or movement data of the robot holding arm.

[0017] Preferably, the control device is further configured to detect a target value for zooming of the recorded image or a desired change in zooming of the recorded image, and to control the zoom mechanism of the image capture device, the mechanical zoom function of the holding arm, and the focusing device based on the target value detected by the input unit. Here, the corresponding target values ​​for adjusting the zoom mechanism, the mechanical zoom function, and the focusing device are preferably calculated based on the detected target value for the desired zooming of the recorded image, wherein the control device then performs corresponding adjustments to the individual components based on the calculated target values.

[0018] In a preferred embodiment, the mechanical zoom function of the holding arm is preferably performed only by changing the position of the image detection device along its optical axis. Here, the axial distance between the image detection device and the area to be observed, ie the distance along the optical axis of the image detection device, is changed.

[0019] The image acquisition device is advantageously arranged on a holding element or manipulator of the robot holding arm, which is preferably arranged at the end. The holding element or manipulator of the robot holding arm, which is arranged at the end, is preferably movable based on a user input via an assigned input device. The assigned input device can be an input unit on the device side, which, in addition to controlling the mechanical zoom function, can also be optionally configured for movable control of the robot holding arm. It should be understood that a separate input unit can also be provided for manipulating the holding arm.

[0020] In a preferred embodiment, the holding arm and / or the control device are configured so that the axial distance between the image detection device and the area to be observed can be changed only when a mechanical zoom function is provided. In other words, the holding arm and / or the control device are configured so that the change of the axial distance between the image detection device and the area to be observed is prevented or prohibited by an input unit on the device side or a separate input unit, which is used to manipulate the holding arm in addition to providing the manipulation of the mechanical zoom function. The holding arm and / or the control device are preferably configured so that a holding element or manipulator arranged on the end side of a robot holding arm on which the image detection device is arranged can be moved and / or rotated in one or more dimensions around the area to be observed by the input unit, wherein the axial distance between the image detection device and the area to be observed remains constant. The robot holding arm and / or the control device are preferably configured so that the image detection device arranged on the holding arm can be moved, in particular, on a spherical surface, for example, around the area to be observed, in particular around the surgical site or the trocar, while maintaining a constant axial distance between the image detection device and the area to be observed.

[0021] The control device of the device can be a central control device, which is connected at least to the input unit, the image acquisition device and the robot holding arm. In addition, the control device can include at least one separate control unit, in particular for controlling the robot holding arm, or can be selectively connected to the control unit. In this case, the separate control unit is designed for preferably bidirectional data communication with the control device of the device.

[0022] In a preferred embodiment, the zoom mechanism is equipped with a preferably internal storage unit and / or control unit to provide the minimum and maximum zoom factors and the current actual zoom factor to the control device of the device. In the present case, the zoom factor is understood to be the magnification factor or reduction factor or zoom factor of the detected recorded image.

[0023] The corresponding minimum and maximum zoom factors can be stored for the corresponding image capture device in the storage unit and / or the control unit. The corresponding minimum and maximum zoom factors can depend in particular on the corresponding optics of the image capture device and / or the corresponding image sensor of the image capture device. Thus, for example, the image capture device can have replaceable optics, wherein for different optics, different sets of minimum and maximum zoom factors can be stored or can be stored in the storage unit and / or the control unit, respectively. The minimum and / or maximum limits can in particular correspond to the corresponding lower and / or upper limits of the reduction or enlargement of the zoom lens, which depends on the lenses used and their possible travel paths in the lens.

[0024] Furthermore, different electronic or digital zoom factors can be stored or can be stored depending on the corresponding image sensor, in particular depending on the image quality of the image sensor. In a further preferred embodiment, the values ​​of the minimum and maximum zoom factors of the optical and / or electronic zoom mechanism can also be configurable, in particular adjustable by user input.

[0025] The optical zoom mechanism can include, in particular, a motor-actuable magnification optical element of the image capture device in order to provide a selectively adjustable optical magnification or zoom function of the image capture device.

[0026] The electronic zoom mechanism can include, in particular, a software-based enlargement and / or reduction or zoom function of the captured recorded images, the so-called digital zoom.

[0027] The focusing device of the device is preferably a focusing optical device assigned to the image detection device, which is selectively and preferably motor-driven and adjustable to adjust the image clarity of the detected recorded image. In a preferred embodiment, the focusing device is assigned a preferably internal storage unit and / or control unit to provide the control device with minimum and maximum focus values ​​or focus settings and the current actual focus value or actual focus setting. The focus setting includes in particular the position of the focusing lens in the optical device of the image detection device. Similar to the above-mentioned implementation scheme for the scaling factor of the corresponding image detection device, the corresponding values ​​can be stored in the storage unit and / or control unit. The corresponding minimum and maximum focus values ​​or focus settings can depend in particular on the corresponding optical device of the image detection device and / or the corresponding image sensor of the image detection device.

[0028] In a preferred embodiment, a preferably internal storage unit and / or a control unit assigned to the focusing device is designed to provide a focusing characteristic curve of the respective image capture device in order to assign a corresponding focus setting to a corresponding magnification by a zoom mechanism of the image capture device and / or a corresponding distance to the region to be observed (in particular in a mechanical zoom function). The focusing characteristic curve preferably comprises a curve or a characteristic curve from which a corresponding focus lens position relative to a corresponding distance to the region to be observed can be derived for focusing the optical device or the recorded image to be detected. The corresponding focusing characteristic curve can be assigned to the respective image capture device with an associated focusing lens in the storage unit and / or the control unit, wherein the corresponding setting values ​​can be predefined for the focusing lens assigned to the image capture device and / or the corresponding setting values ​​can be learned or can be learned based on empirical measurement data with the respective image capture device and a robot holding arm connected to the image capture device.

[0029] In a preferred embodiment, the control device is configured to compare the target value for zooming detected by the input unit with the minimum and maximum zoom factors and the actual zoom factor provided by the zoom mechanism (i.e., the current zoom factor of the zoom mechanism), and based on this, selectively or in combination control the optical and / or electronic zoom mechanism and the robot holding arm to provide a mechanical zoom function. Here, the control device can also be configured to perform a preferred control of the optical and / or electronic zoom mechanism and / or the mechanical zoom function based on the detected target value for zooming and the comparison with the current actual zoom factor of the zoom mechanism. In a particularly preferred embodiment, the control device is configured to first control the optical and / or electronic zoom mechanism until the minimum or maximum zoom factor of the zoom mechanism is reached, and then further adjust the zoom by controlling the robot holding arm and thus by the mechanical zoom function.

[0030] In order to provide a mechanical zoom function, the control device is preferably configured to calculate the necessary change in the position of the holding arm, in particular the change in position of the image detection device arranged on the holding arm along the optical axis of the image detection device, based on the detected input command, in particular based on the target value for zooming the recorded image. Here, the control device is preferably configured to output one or more target values ​​for causing the robot holding arm to move, in particular change in position, along the optical axis of the image detection device in order to change its position. The corresponding position change can then be performed by controlling the robot holding arm by the control device itself or a separate control unit of the robot holding arm, to which the target value is provided or transmitted. More preferably, the control device is configured so that the focusing device is controlled based on the calculated change in the position of the holding arm, in particular the change in the axial distance from the area to be observed, and in particular the position of the focusing lens of the image detection device is adjusted. As described above, the focusing device is preferably controlled by reading out the stored focusing characteristic curve of the image detection device.

[0031] The control device can be configured so that when zooming the recorded image by adjusting the optical and / or electronic zoom mechanism and / or the mechanical zoom function, the simultaneous and / or subsequent control of the focusing device to track the focus is preferably determined based on the stored focusing characteristic curve of the corresponding image detection device.

[0032] In a preferred embodiment, the input unit of the device is configured so that it can realize a variable, in particular deflection-dependent target value detection of the zoom factor. The input unit preferably comprises input mechanisms with multiple degrees of freedom, which are configured to detect deflections in corresponding directions. In particular, the input unit can include a 3D joystick, which, in addition to detecting lateral input commands in a first movement plane, can also detect vertical input commands in a direction perpendicular to the first movement plane. In addition, the 3D joystick can be configured to detect deflections around a vertical rotation axis and / or a tilt axis preferably arranged perpendicular thereto. Preferably, the target value detection of the zoom factor can be performed by detecting vertical input commands, in particular in a first direction for zooming out and in a second direction opposite to the first direction for zooming in. The input unit can be operated by the user with his hands or feet.

[0033] The input unit can also be designed to selectively control the image capture device and / or the movable holding arm. In this case, the input unit can have, for example, a selective input function, such as a manually operable key or a foot switch, by which a distinction can be made between controlling the holding arm via the input unit, in particular for changing the position and / or location of the holding arm, and detecting a desired zoom factor or detecting an adjustment to a current zoom factor.

[0034] In a preferred embodiment, the image detection device is a stereoscopic exoscope known per se. The stereoscopic exoscope preferably has at least an optical, controllable zoom mechanism (in particular an optical zoom) and a motor-driven focus controller. The image detection device may also have a mirror unit arranged at the far end for deflecting the viewing direction or the detection direction by 90°. The mirror unit may optionally be designed to be movable or rotatable. The image detection device may, for example, have two to four image sensors configured to detect (stereo) recorded images having a resolution preferably of 4K or higher.

[0035] In another preferred embodiment, the image acquisition device comprises an autofocus function, which can be activated and / or controlled by the control device. For example, the control device can be designed to activate a preferably one-time, i.e. non-continuous, autofocus function of the image acquisition device after the end of the movement of the holding arm. Since the current control of the mechanical zoom function or the holding arm is carried out without feedback of sensor data from the holding arm, the control device is designed here to carry out a time estimate, i.e. how long the robot holding arm needs to achieve the desired or calculated movement, based on the target value calculated for the movement of the holding arm, and based on this, to control the autofocus function after the expiration of the previously calculated time interval.

[0036] The autofocus function of the image detection device can be implemented using means known per se; in particular, the autofocus function can include contrast measurement, phase comparison or active distance measurement.

[0037] In a preferred embodiment of the method according to the invention, the zooming and adjusting of the focusing device or the focus tracking of the recorded image is performed by actuating the zoom mechanism, the robot holding arm and the focusing device solely on the basis of an input command for zooming the recorded image detected by the input unit and in particular without reading out or evaluating position data or movement data of the robot holding arm. The zooming of the recorded image and the focusing of the recorded image can be performed simultaneously or sequentially.

[0038] In a preferred embodiment of the method, based on the detected target value for zooming the recorded image, in particular the desired magnification of the recorded image, zooming is first performed by manipulating the zoom mechanism of the image detection device, and only when the maximum zoom factor of the image detection device is exceeded, zooming is performed by changing the axial distance between the image detection device and the area to be observed, by changing the position of the holding arm. In addition, it is advantageously possible to control in such a way that, in the case of desiring to reduce the recorded image, zooming is first performed by changing the axial distance between the image detection device and the area to be observed (by changing the position of the holding arm, in particular by increasing the axial distance), and manipulation of the zoom mechanism of the image detection device is performed only when a predefined zoom factor is exceeded.

[0039] However, the reverse sequence is also possible, so that when zooming in with a desired magnification factor, for example if the control device recognizes that the desired target zoom cannot be achieved only by the zoom mechanism of the image capture device, a mechanical zoom is first performed by the holding arm. Then further zooming is performed by the optical and / or electronic zoom mechanism. When zooming out, the sequence is corresponding.

[0040] In another preferred embodiment, when the position of the holding arm is changed, the manipulation of the focusing device includes at least the following additional steps: determining an initial focus setting before performing the movement of the holding arm; calculating the necessary change in the position of the holding arm along the optical axis of the image detection device to provide a detected target value for the scaling of the recorded image; calculating a target value for the focus setting based on the calculated change in the position of the holding arm and based on a focusing characteristic curve stored for the image detection device; and manipulating the focusing device to set the determined target value. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Further advantages and details of the invention emerge from the following description of preferred embodiments of the invention and with reference to the drawings, which are merely schematic.

[0042] in:

[0043] Figure 1 A schematic diagram showing an embodiment of the device according to the invention is shown;

[0044] Figure 2 A side view of an embodiment of an image detection device according to the present invention is shown;

[0045] Figure 3 A block diagram of the device components is shown;

[0046] Figure 4 shows a schematic flow chart outlining the steps of a preferred embodiment in zooming and focus tracking,

[0047] Figure 5 A schematic diagram showing the application range of optical zoom and mechanical zoom mechanisms; and

[0048] Figure 6 A schematic diagram of a focusing characteristic curve is shown.

[0049] In the drawings, the same elements or elements having the same function are marked with the same reference numerals. DETAILED DESCRIPTION

[0050] Figure 11 shows a schematic diagram of an embodiment of a device 100 according to the invention. The device is designed to visually display a region 101 to be observed, in particular an operating region, which is shown schematically here only as an example, in a scalable manner. The region 101 to be observed can be a surgical site.

[0051] The device 100 has an image detection device 102, which is configured to detect a recorded image, in particular a live video image. The image detection device 102 can be arranged at a distance of 20 cm to 50 cm or can be arranged above the area to be detected. The image detection device 102 is arranged at the end section or manipulator 103a of the robot holding arm 103. The robot holding arm 103 can be arranged at a support element 104, such as a selectively movable cart. Alternatively, it is also possible to be installed on the ceiling or wall in the treatment room. The image detection device 102 is also connected to a display unit 105, and the detected and selectively zoomable recorded image is output on the display unit. The device 100 also includes an input unit 106 and preferably includes a foot switch 107 assigned to the input unit. The foot switch 107 can be configured to selectively switch the control of the robot holding arm 103 or the image detection device 102. The input unit 106 is preferably designed as a 3D joystick and is preferably capable of detecting vertical movements (double arrow C) in addition to lateral input commands (double arrows A, B). In addition, the input unit 106 can be designed to detect rotational movements and / or tilting movements (not shown) of the 3D joystick. The input unit 106 is particularly designed to detect input commands for zooming or changing the zoom ratio of a recorded image displayed on the display unit 105. Here, the input unit is preferably designed to detect a deflection-dependent target value of the zoom factor, in particular a deflection-dependent change of the current zoom factor. This means that a greater deflection of the 3D joystick along the double arrow C results in a greater zooming in or out of the recorded image on the display unit 105.

[0052] The zooming, ie, reduction or enlargement, of the recorded image can be performed by an optical zoom mechanism 108 and / or an electronic zoom mechanism 109 (see Figure 3 ) and / or by a mechanical zoom function 110 provided by a holding arm (see Figure 3 ) is performed, wherein the robot holding arm moves the image detection device 102 held on the robot holding arm relative to the area to be observed 101 along the optical axis 111 of the image detection device 102 to adjust the axial distance between the image detection device and the area to be observed (double arrow D).

[0053] Figure 2A side view of a preferred embodiment of an image detection device 102 is shown. In the present case, the image detection device is a stereoscopic exoscope known per se. The stereoscopic exoscope has an optical zoom mechanism that can preferably be controlled by a motor, i.e. an optical zoom 108 that is assigned to the optics of the exoscope or is comprised by the optics, and a focusing device 112 that can preferably be controlled by a motor, i.e. a focusing lens that is assigned to the optics of the exoscope or is comprised by the optics. The image detection device 102 also includes a mirror unit 113 that is arranged at the far end and is used to deflect the viewing direction or the detection direction by 90°. In addition, the image detection device 102 can have a filter wheel 114 for accommodating different, application-specific color filters. The image detection device can also have a lighting unit that is not shown to illuminate the area to be observed.

[0054] Figure 3 Shown according to Figure 1 The device 100 comprises a control device 115 connected to an image capture device 102, a robot holding arm 103 and an input unit 106. The control device 115 may comprise a central control unit with a processor and associated memory means (not shown).

[0055] According to the invention, the control device is configured such that it adjusts the mechanical zoom function 110 and the focusing device 112 by controlling the holding arm 103 and the focusing device 112 without using sensor values ​​of the holding arm 103 for regulation, in particular without actual values ​​of position data or movement data of the holding arm 103. In particular, the control device 115 should control the holding arm 103 and the focusing device 112 only based on input commands detected by the input unit 106 to zoom the recorded image or change the zoom ratio of the recorded image. The control unit 115 is configured here to control the mechanical zoom function 110 of the holding arm 103 by changing the axial distance d between the image acquisition device 102 and the area to be observed 101.

[0056] The control device 115 is designed such that it can control the optical zoom function 108 (in particular by adjusting the lens optics) and / or the electronic zoom function 109 (in particular by displaying a sub-portion of the digital image area on the display unit 105) of the image capture device 102. In this case, the control device 115 is preferably designed to read out the corresponding actual value, in particular the current magnification factor or zoom factor, of the optical zoom mechanism 108 and / or the electronic zoom mechanism 109. For this purpose, the optical zoom mechanism 108 and / or the electronic zoom mechanism 109 can have an internal storage unit and / or control unit which stores the corresponding minimum and maximum zoom factors in addition to the current actual value of the zoom factor.

[0057] The control device 115 is also configured to selectively or in combination control the zoom mechanisms 108, 109 and the mechanical zoom function 110 in order to provide an extended zoom range or an extended overall zoom range (see also Figure 5 ). In this case, the control device 115 realizes the focus tracking of the recorded image by controlling the focusing device 112, which is preferably carried out simultaneously or sequentially. The focusing device 112 preferably also has an internal storage unit and / or a control unit, which can provide the control device with the current actual value of the focus setting of the focusing device 112. In addition, the corresponding focus characteristic curve 116 of the corresponding image acquisition device or optical device or the identification number of the image acquisition device 102 is stored in the focusing device 112 and / or the control device 115, from which the corresponding focus setting or focusing lens F can be obtained. P The relationship between the position in the optical device of the image detection device 102 and the corresponding distance d of the image detection device 102 or the optical device of the image detection device 102 from the area to be observed 101. In particular, it can be read from the stored focus characteristic curve to which position of the focusing lens the corresponding focusing device is to be set in order to achieve focusing of the recorded image at a specific distance d between the detection optical device and the area to be observed 101.

[0058] Combine the following Figure 4 An embodiment of the steps of zooming and focus tracking using the device according to the present invention is described.

[0059] Figure 4 A flow chart 200 is shown for operating the optical zoom mechanism 108 and the mechanical zoom function 110 of the image sensing device 102 to provide extended zoom and tracking focus settings, for example, for magnifying a recorded image, in accordance with the present invention.

[0060] It is assumed that before using the device, the position of the image detection device 102 relative to the area 101 to be observed is adjusted manually and / or electronically, in particular by the desired orientation of the robot holding arm 103 and the image detection device 102. Here, the user can select the desired observation angle and the desired distance d between the image detection device 102 and the area 101.

[0061] In a first step 201, an initial focus setting of the captured recorded image is first performed by manual and / or electronic adjustment of a focusing device 112 assigned to the image capture device 102. In addition, an initial zoom factor or magnification factor can also be adjusted by manual and / or electronic adjustment of, for example, an optical zoom mechanism 108.

[0062] In step 202, the zoom mechanism 108 and the focusing device 112 are read out by the control device 115 to provide the actual value of the current zoom factor of the image detection device 102 and the focus setting or the setting position of the focusing lens of the optical device of the image detection device 102. The stored focus characteristic curve can thus be used to determine the focal distance or distance d (position P0, see Figure 6 ).

[0063] Step 202 can be triggered, for example, by pressing the foot switch 107, wherein the foot switch 107 can serve as a trigger for changing the zoom ratio by an input on the input unit or the 3D joystick 106. Alternatively, step 202 can also be triggered directly on the input unit 106.

[0064] In step 203 , an input of a desired zoom factor or a desired change in the zoom factor is detected using the input unit 106 . Here, the input unit 106 can detect a target value of a desired amplification, which is sent to the control device 115 .

[0065] In step 204, a new zoom factor or magnification factor to be set by the device is calculated based on the user input or the desired magnification. In this case, the detected actual zoom factor is compared with the minimum and maximum values ​​of the zoom factor of the zoom mechanism 108. The zooming using the zoom mechanism 108 of the image acquisition device 102 can be performed until the stored maximum value of the zoom mechanism 108 is reached without the mechanical zoom function 110 being manipulated, see iterative loop 205.

[0066] If the desired zoom value exceeds the maximum value of the zoom mechanism 108 (checked in step 206), the necessary change in distance d1 to the region to be observed 101, which is necessary to achieve the desired remaining magnification, is calculated in step 207. Based on this, a target value output is then performed in step 208 in order to control the robot holding arm 103 and thus change and in particular shorten the distance d along the optical axis 111 towards the position P1 of the holding arm 103.

[0067] In step 209, the focusing device 112 is controlled by the control device 115 to track the image sharpness. In this case, the focusing lens is adjusted according to the stored focus characteristic curve based on the input command for enlarging the recorded image detected by the input unit 106. In particular, the control device 115 adjusts the distance change d1 previously determined and thus based on the new position P1 (see Figure 6 ) to determine the necessary settings of the focusing device or the position F1 of the focusing lens in the optical device of the image detection device 102.

[0068] If it is now desired to reduce the zoom factor again starting from the new position P1 or a reduction in the zoom factor is detected via the input unit 106, the control device 115 preferably controls so that the mechanical zoom function 110 is first actuated to reduce the zoom factor. In particular, the axial distance d is increased until the previously controlled distance change d1 is reached. Then the zoom mechanism 108 of the image capture device 102 is actuated.

[0069] If a further reduction in the zoom factor is desired or detected via input unit 106, zoom mechanism 108 is controlled to zoom out until a stored minimum value of the zoom mechanism is reached. After reaching the minimum value of zoom mechanism 108, mechanical zoom function 110 is then actuated by increasing axial distance d again, i.e. further away from image capture device 102 along optical axis 111.

[0070] Refer again below Figure 5 The above-mentioned control performed by the control device 115 is explained by the schematic diagram in FIG. Figure 5 The zoom region 120 of the exemplary optical zoom mechanism 108 of the image acquisition device 102 can be seen in FIG. ini Initially, when zooming in (arrow 121) or zooming out (arrow 122), the optical zoom mechanism 108 is first operated until the maximum zoom factor S is reached. max Or the minimum scaling factor S min When the maximum zoom factor is exceeded or the minimum zoom factor is fallen below, the mechanical zoom function 110 is controlled to zoom in by shortening the axial distance d between the imaging device 102 and the operating area 101 in the area 123 or to zoom out by increasing the axial distance in the area 124 .

[0071] If the zoom factor is to be called back accordingly, that is, from the minimum zoom factor S of the image detection device 102 min and the maximum scaling factor S max If the area outside the area is adjusted back toward the area 120, the mechanical zoom function is first operated to reduce or increase the zoom factor until a distance within the area 120 is reached, at which distance the optical zoom mechanism 108 can be used.

[0072] The above-described manipulation may depend on the activation of the foot switch. For example, if the foot switch is released after zooming out (arrow 122) into the area 124, then, unlike the above-described manipulation, and for example in the case of (again) increasing the zoom factor from the area 124, the optical zoom mechanism 108 may first be immediately manipulated, and only when the maximum possible zoom factor of the optical zoom mechanism 108 is reached, the mechanical zoom function 110 may be manipulated for further zooming in.

[0073] Reference numerals list

[0074] 100 devices

[0075] 101 Area to be observed

[0076] 102 Image detection device

[0077] 103 Robot holding arm

[0078] 103a End side section

[0079] 104 Cart

[0080] 105 Display

[0081] 106 Input Unit

[0082] 107 Foot switch

[0083] 108 Optical zoom mechanism / zoom

[0084] 109 Electronic zoom mechanism / zoom

[0085] 110 Mechanical zoom function

[0086] 111 Optical axis

[0087] 112 Focusing device

[0088] 113 Mirror unit

[0089] 114 Filter Wheel

[0090] 115 Control Device

[0091] 116 Focus characteristic curve

[0092] 120 Zoom range of optical / electric zoom mechanism

[0093] 121 Zoom

[0094] 122 Zoom out

[0095] 123 Mechanical zoom function magnification area

[0096] 124 Mechanical zoom function reduced area

[0097] d Axial distance

[0098] d1 distance change

[0099] d ini Initial distance

[0100] S maxMaximum zoom factor of the optical zoom mechanism

[0101] S min Minimum zoom factor of optical zoom mechanism

[0102] P 0、1 Keep arm position

[0103] F P Focus lens position

Claims

1. An imaging device (100) for visually displaying a region to be observed (101), in particular a surgical region, in a scalable manner, the imaging device comprising: an optical image detection device (102), the image detection device being used to detect a recorded image of the area to be observed (101); an optical and / or electronic zoom mechanism (108, 109) assigned to the image capture device (102), and an adjustable focusing device (112) assigned to the image capture device, the zoom mechanism being used to zoom the captured recorded image; A holding arm (103) of the robot, the holding arm being used to move the image detection device (102) relative to the area to be observed (101), wherein the holding arm (103) provides a mechanical zoom function (110) by adjusting an axial distance (d) between the image detection device (102) and the area to be observed (101); An input unit (106) for detecting an input command from a user side for scaling the recorded image on the display unit (105); a control device (115) for adjusting the zoom mechanism (108, 109) of the image detection device (102) and the mechanical zoom function (110) of the holding arm, and for adjusting the focusing device (112), It is characterized in that The control device (115) performs adjustment of the mechanical zoom function (110) and the focusing device (112) in the following manner, that is, the holding arm (103) and the focusing device (112) are controlled only based on the input command for zooming the recorded image detected by the input unit (106).

2. The device according to claim 1, characterized in that The control device (115) controls the zoom mechanism (108, 109), the mechanical zoom function (110) of the holding arm (103) and the focusing device (112) solely based on an input command for zooming the recorded image detected by the input unit (106), and in particular without having to read out or evaluate position data or movement data of the holding arm (103) of the robot.

3. The device according to claim 1 or 2, characterized in that The input unit (106) detects a target value for zooming the recorded image, and the control device (115) controls the zoom mechanism (108, 109) of the image detection device, the mechanical zoom function (110) of the holding arm (103), and the focusing device (112) based on the target value detected by the input unit (106).

4. The device according to any one of claims 1 to 3, characterized in that The mechanical zoom function (110) of the holding arm (103) is achieved by changing the position of the image detection device (102) along its optical axis (111).

5. The device according to any one of the preceding claims, characterized in that The zoom mechanism (108, 109) is equipped with a preferably internal storage unit and / or control unit in order to provide the control device (115) with a minimum and maximum zoom factor and a current actual zoom factor.

6. The device according to any one of the preceding claims, characterized in that The focusing device (112) is equipped with a preferably internal memory unit and / or a control unit to provide the control device (115) with the minimum and maximum focus value / focus setting and the current actual focus value / focus setting.

7. A device according to any one of the preceding claims, characterised in that The control device (115) and / or a preferably internal storage unit and / or a control unit associated with the focusing device (112) provide a focusing characteristic curve of the image acquisition device (102) in order to assign a corresponding focus setting to a corresponding distance (d) from the area to be observed (101).

8. The device according to any one of the preceding claims, characterized in that The control device (115) compares the target value for zooming detected by the input unit (106) with the minimum and maximum zoom factors and the actual zoom factor of the zoom mechanism (108, 109), and based on this, selectively or in combination controls the optical and / or electronic zoom mechanism (108, 109) and the holding arm (103) of the robot for providing the mechanical zoom function (110).

9. The device according to any one of the preceding claims, characterized in that The control device (115) firstly controls the optical and / or electronic zoom mechanism (108, 109) until a minimum or maximum zoom factor of the zoom mechanism (108, 109) is reached and only then performs a further adjustment of the zoom by controlling the holding arm (103) of the robot.

10. The device according to any one of the preceding claims, characterized in that The control device (115) calculates the necessary change in the position of the holding arm, in particular the change in the position of the image detection device (102) arranged at the holding arm along the optical axis (111) of the image detection device (102), based on the target value for scaling the recorded image, and controls the focusing device (112) based on the change in the position of the holding arm, in particular the change in the axial distance (d) from the area to be observed (101).

11. The device according to claim 10, characterized in that The control device (115) determines the simultaneous and / or subsequent actuation of the focusing device (112) during the zooming of the recorded image in order to track the focus based on a stored focusing characteristic curve of the corresponding image capture device (102).

12. The device according to any one of the preceding claims, characterized in that The input unit (106) enables variable, in particular deflection-dependent, target value detection of a zoom factor and optionally selective control of the image capture device (102) and / or the movable holding arm (103).

13. The device according to any one of the preceding claims, characterized in that The image capture device (102) comprises a stereoscopic exoscope, which preferably has at least an optical zoom mechanism (108) and a motor-driven focusing device (112).

14. Apparatus according to any one of the preceding claims, characterised in that The movable holding arm (103) of the robot has its own control unit, which is used for two-way data communication with the control device (115) of the device (100).

15. Apparatus according to any one of the preceding claims, characterised in that When the movement of the holding arm is completed, the control device (115) activates a preferably one-time, ie non-continuous, autofocus function of the image capture device (102).

16. A method for visually displaying a region to be observed, in particular a surgical region, in a scalable manner, the method comprising at least the following steps: - using an optical image detection device (102) to detect the recorded image of the area to be observed (101) so as to display it on a display unit (105); - initially setting the focus of the recorded image by manually and / or automatically adjusting a focusing device (112) assigned to the image capture device; - detecting an input command for scaling the recorded image using an input unit (106); - zooming the recorded image on the display unit (105) by adjusting an optical and / or electronic zoom mechanism (108, 109) assigned to the image capture device, and adjusting a mechanical zoom function (110) by moving the image capture device (102) along an optical axis (111) of the image capture device (102) by means of a holding arm (103) of a robot; - focusing the zoomed recorded image by adjusting the focusing device (112), It is characterized in that The mechanical zoom function (110) and the focusing device (112) are adjusted in such a way that the holding arm (103) and the focusing device (112) are controlled solely on the basis of an input command for zooming the recorded image detected by the input unit (106).

17. The method according to claim 16, characterized in that When zooming the recorded image and adjusting the focusing device (112), the zoom mechanism (108, 109), the holding arm (103) of the robot and the focusing device (112) are controlled only based on the input command for zooming the recorded image detected by the input unit (106), and in particular, there is no need to read out or evaluate the position data or movement data of the holding arm (103) of the robot.

18. The method according to claim 16 or 17, characterized in that The recorded image is preferably zoomed and focused simultaneously or sequentially.

19. The method according to any one of claims 16 to 18, characterized in that Based on the detected target value for scaling the recorded image, scaling is first performed by manipulating a scaling mechanism (108, 109) of the image detection device (102), and scaling is performed only when the maximum scaling factor of the image detection device is exceeded by changing the axial distance (d) between the image detection device (102) and the area to be observed (101) and by changing the position of a holding arm.

20. The method according to any one of claims 16 to 19, characterized in that Manipulating the focusing device (112) while changing the position of the holding arm comprises at least the following additional steps: - determining an initial focus setting before performing movement of the holding arm (103); - calculating the necessary change in the position of the holding arm along the optical axis (111) of the image detection device (102) in order to provide the detected target value for scaling the recorded image; - calculating a target value for the focus setting based on the calculated change in the position of the holding arm and based on a focus characteristic curve stored for the image detection device (102); - operating the focusing device (112) to set the determined target value.

Images