Imaging device having extended zoom functionality and focus tracking
Patent Information
- Application Number
- EP2023737958
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-07-04
- Filing Date
- 2023-06-29
- Publication Date
- 2025-05-14
AI Technical Summary
Current imaging devices for endoscopic and exoscopic procedures face challenges with focus tracking due to small depth of field, leading to image blurring and require manual adjustments, which are inconvenient during surgeries, and existing solutions with sensor-based systems are complex and prone to errors.
An imaging device with a robotic holding arm that adjusts mechanical scaling and focus settings based solely on user input commands without feedback from position or movement data, using optical and electronic scaling means and motor-driven focus optics to maintain image sharpness.
Enables efficient and accurate tracking of image sharpness with extended zoom functionality, reducing the need for manual adjustments and minimizing hardware complexity, allowing for faster and more reliable focus adjustments during procedures.
Smart Images
Figure 1.1
Abstract
Description
[0001] Imaging device with extended zoom functionality and focus tracking
[0002] The present invention relates to an imaging device and a method for scalable visual representation of an area to be viewed, in particular a surgical area, with an extended zoom functionality and an optimized focus adjustment.
[0003] Endoscopes and exoscopes for use in diagnostic or surgical procedures on patients are well known in the art. The endoscope or exoscope is sometimes attached to a motorized holding arm, which is positioned by the user in such a way that a camera unit of the endoscope or exoscope appropriately displays the area to be captured, usually a surgical site, for the attending surgeon. The captured image is then output as a live image on a suitable imaging unit, such as a screen. The distance between the holding arm and the patient is generally chosen such that the displayed image section is suitable for the surgeon while at the same time ensuring good access to the patient. Once a suitable placement and orientation of the holding arm and the endoscope or exoscope attached to it has been found, the focus of the camera unit is usually adjusted manually.
[0004] Furthermore, it is state of the art for the respective camera unit of the endoscope or exoscope to be equipped with scaling means, in particular an optical, electronic, or digital zoom, with which the user can enlarge or reduce an image of the captured area as needed before and / or during a treatment or procedure. To expand a zoom range, it is also already known to provide, in addition to scaling means of the camera unit, a mechanical scaling function by changing the position of the holding arm along an optical axis of an endoscope and thus by changing a distance between the camera unit and the captured area.
[0005] However, readjusting or adjusting the focus of the camera unit presents a challenge, as the depth of field is very shallow, especially with microscopic lenses. Therefore, even a small change in the position of the support arm relative to the captured area or the surgical site can result in blur. For example, providing continuous autofocus carries the risk of undesirably pulsating image focus or focusing on an undesirable portion of the captured area.
[0006] Manual or user-triggered focus adjustment is also disadvantageous, since the user usually has both hands on the instruments at the site during a procedure or treatment and additional manual operation of the device is undesirable.
[0007] US 11,033,338 B2, for example, discloses an imaging system comprising an endoscope with a camera head for image acquisition and a display connected thereto for the enlarged representation of a captured image area during an endoscopic procedure, wherein the endoscope is arranged on a movable support arm. The system has a control device for controlling the camera head on the endoscope and a control device for controlling the support arm, wherein the camera head is designed to provide an optical and electronic zoom. In addition to the optical and electronic scaling function of the camera head, a mechanical scaling function is provided by changing the support arm position along an optical axis of the endoscope. The focus setting of the camera head is adjusted by means of an autofocus device, which entails the disadvantages outlined above.US 8,715,167 B1 discloses a telesurgical system for minimally invasive interventions, comprising an input device and a robot system connected thereto. The first manipulator includes a first manipulator with an endoscope attached thereto for optical image acquisition, which is connected to an input console of the telesurgical system for image display, and a second manipulator configured to hold and guide a surgical instrument and connected to an input device on the input console. The telesurgical system is configured to change a focus and a scaling factor in response to a movement of the robot system, for example, to maintain focusing of the image acquisition device when the distance to an object to be acquired changes.The robot system comprises several sensors that monitor the movement of the robot system elements, determine a change in distance from an initial focus point in a coordinate system of the robot system, and adjust the focus setting of the image capture device based on this change. While this enables precise position detection and the resulting adjustment of the focus setting, it also requires a large number of sensors on the robot system and the associated information monitoring of each of these sensors. This not only results in increased hardware complexity but also increases the susceptibility to errors in the focus setting, which directly depends on it.
[0008] The object of the present invention is therefore to at least partially overcome the aforementioned disadvantages of the prior art. In particular, the object of the invention is to provide an improved imaging device and a corresponding method that enables a scalable image of a surgical area with extended zoom functionality while simultaneously providing simple and effective tracking of the image sharpness.
[0009] The above-mentioned object is achieved by a device having the features of independent claim 1. Furthermore, the object is achieved by a method according to the invention having the features of independent claim 16.
[0010] Advantageous developments of the invention are specified in the dependent claims. The scope of the invention includes all combinations of at least two features disclosed in the description, the claims and / or the figures. It is understood that exemplary embodiments and embodiments described with reference to the device according to claim 1 can refer in an equivalent, even if not word-identical, form to the method according to claim 16 without being explicitly mentioned for this purpose. It is also understood that customary linguistic transformations and / or a corresponding replacement of respective terms within the framework of common linguistic practice, in particular the use of synonyms supported by generally accepted linguistic literature, are also encompassed by the present disclosure content without being explicitly mentioned in their respective formulation.
[0011] According to a first aspect of the invention, an imaging device for the scalable visual representation of an area to be viewed, in particular an operating area, is proposed.The device comprises an optical image capture device for capturing a recorded image of the area to be viewed, optical and / or electronic scaling means assigned to the image capture device for scaling the captured image, and an adjustable focusing device assigned to the image capture device, a robotic holding arm for moving the image capture device relative to the area to be viewed, wherein the holding arm is designed to provide a mechanical scaling function by adapting an axial distance between the image capture device and the area to be viewed, an input unit for capturing a user-side input command for scaling the recorded image on a display unit, and a control device for adjusting the scaling means of the image capture device and the mechanical scaling function of the holding arm, as well as the focusing device.According to the invention, the control device is designed to carry out an adjustment of the mechanical scaling function and the focusing device by controlling or activating the holding arm and the focusing device solely based on the input command for scaling the recorded image detected by the input unit, in particular a detected target value for the scaling.
[0012] According to a second aspect of the invention, a method for the scalable visual representation of an area to be viewed, in particular an operating area, is proposed.The method comprises at least the following steps: capturing an image of the area to be viewed using an optical image capturing device for display on a display unit; initial focus adjustment of the captured image by manual and / or automatic adjustment of a focusing device associated with the image capturing device; capturing an input command for scaling the captured image using an input unit; scaling the captured image on the display unit by adjusting optical and / or electronic scaling means associated with the image capturing device and adjusting a mechanical scaling function by moving the image capturing device along an optical axis of the image capturing device using a robotic holding arm; focusing the scaled captured image by adjusting the focusing device.
[0013] In contrast to the prior art, according to the invention the mechanical scaling function is adjusted by moving the holding arm and the focusing device to track the focus or sharpness of the recorded image by controlling the focusing device solely on the basis of the recorded input command for scaling the recorded image. This means that, in contrast to the control of holding arm movement and corresponding focus tracking known from the prior art, no further input variables, in particular no actual values for position or movement data of the holding arm, are recorded as feedback values for a control of the mechanical scaling function and the focusing device based thereon. In other words, the present control or activation of the mechanical scaling function and the focusing device takes place without or withindependent of the recording of feedback values, in particular without recording of further input variables or values, in particular without recording of actual values for position or movement data of the holding arm that can be detected, for example, by corresponding sensors.
[0014] According to the invention, an improved imaging device and a corresponding method are provided, which enable a scalable image of a surgical area with extended scaling or zoom functionality, while simultaneously allowing simple, effective, and efficient control of the system components, in particular the adjustment of the holding arm and tracking of the image sharpness of the image. This allows for significantly faster and sufficiently accurate tracking of the focus setting based on the measured input variable, without the need for control based on the determination of actual values, in particular for position or movement data of the holding arm.
[0015] This is independent of any possible internal control of the robotic holding arm or the focusing unit, which can each measure and compare their own actual and target positions during movement, also for safety reasons. However, such information is not used to control the other component.
[0016] In a preferred embodiment, the control device is configured to control or actuate the scaling means of the image capture device, the mechanical scaling function of the holding arm, and the focusing device solely based on the input command acquired by the input unit for scaling the captured image, and in particular without reading or evaluating position or movement data of the robotic holding arm to provide a control variable. In this case, both the scaling means of the robotic holding arm and the focusing device are actuated solely based on the acquired input command and thus not based on further control values, in particular without acquiring position or movement data of the robotic holding arm.
[0017] The control device is preferably further configured to detect a target value for the scaling of the captured image or a desired change in the scaling of the captured image and, based on the target value detected by the input unit, to control the scaling means of the image capture device, the mechanical scaling function of the holding arm, and the focusing device. In this case, respective target values for the adjustment of the scaling means, the mechanical scaling function, and the focusing device are preferably calculated based on the detected target value for a desired scaling of the captured image, with the control device subsequently performing a corresponding adjustment of the individual components based on the calculated target values.
[0018] In a preferred embodiment, the mechanical scaling function of the support arm is preferably performed exclusively by changing the position of the image capture device along its optical axis. In this case, an axial distance, i.e., a distance along the optical axis of the image capture device, between the image capture device and the area to be viewed is changed.
[0019] The image capture device is advantageously arranged on a holding element or manipulator, preferably arranged at the end of the robotic holding arm. The holding element or a manipulator of the robotic holding arm, preferably arranged at the end, is preferably movable based on a user input by means of an associated input device. The associated input device can be the device-side input unit, which can be selectively configured for the movable control of the robotic holding arm, in addition to controlling the mechanical scaling function. It is understood that a separate input unit can also be provided for controlling the holding arm.
[0020] In a preferred embodiment, the support arm and / or the control device are configured such that a change in the axial distance between the image capture device and the area to be viewed is only possible when the mechanical scaling function is provided. In other words, the support arm and / or the control device are configured such that a change in the axial distance between the image capture device and the area to be viewed is prevented or blocked by the device-side input unit or a separate input unit for controlling the support arm outside of the control for providing the mechanical scaling function.The holding arm and / or the control device are preferably designed such that a holding element arranged at the end or a manipulator of the robotic holding arm, on which the image capture device is arranged, can be moved and / or rotated by the input unit in one or more dimensions around the area to be viewed, wherein the axial distance between the image capture device and the area to be viewed is kept constant. The robotic holding arm and / or the control device are preferably designed such that the image capture device arranged on the holding arm can be moved, in particular on a spherical surface, for example, around the area to be viewed, in particular a surgical site or a trocar, while maintaining a constant axial distance between the image capture device and the area to be viewed.
[0021] The control device of the device can be a central control device that is connected at least to the input unit, the image capture device, and the robotic holding arm. Furthermore, the control device can comprise at least one separate control unit, in particular for controlling the robotic holding arm, or can be selectively connected to it. In this case, the separate control unit is designed for preferably bidirectional data communication with the control device of the device.
[0022] In a preferred embodiment, the scaling means are assigned a preferably internal memory and / or control unit for providing a minimum and maximum scaling factor as well as a current actual scaling factor to the control device of the device. In this context, a scaling factor is understood to mean a magnification or reduction factor or zoom factor of the captured image.
[0023] A respective minimum and maximum scaling factor can be stored in the memory and / or control unit for a respective image capture device. In this case, a respective minimum and maximum scaling factor can be dependent, in particular, on a respective lens of the image capture device and / or a respective image sensor of the image capture device. For example, the image capture device can have interchangeable lenses, wherein different sets for a minimum and maximum scaling factor can be stored or can be stored in the memory and / or control unit for different lenses. The minimum and / or maximum limits can, in particular, correspond to a respective lower and / or upper limit for the reduction or magnification of a zoom lens, which depends on the lenses used and their possible travel paths in the lens.
[0024] Furthermore, depending on a particular image sensor and in particular depending on the image quality of the image sensor, a different electronic or digital scaling factor can be stored or can be stored. In a further preferred embodiment, the values for a minimum and maximum scaling factor for the optical and / or electronic scaling means can also be configurable, in particular adjustable by user input.
[0025] The optical scaling means may in particular comprise a motor-controlled magnification optics of the image capture device for providing a selectively adjustable optical magnification or zoom functionality of the image capture device.
[0026] The electronic scaling means may in particular comprise a software-based magnification and / or reduction or zoom functionality of the captured image, i.e. a so-called digital zoom.
[0027] The focusing device of the apparatus is preferably a focus optics unit which is assigned to the image capture device and which can be selectively and preferably motor-drivenly adjusted for adjusting the image sharpness of the captured image. In a preferred embodiment, a preferably internal storage and / or control unit is assigned to the focusing device for providing a minimum and maximum focus value or a focus setting as well as a current actual focus value or an actual focus setting to the control device. The focus setting here comprises in particular a position of the focus lens in an optics unit of the image capture device. The respective values can be stored in the storage and / or control unit analogous to the above explanations for a scaling factor for a respective image capture device. In this case, a respective minimum and maximum focus value ora focus setting may in particular be dependent on a respective optics of the image capture device and / or a respective image sensor of the image capture device.
[0028] In a preferred embodiment, the preferably internal memory and / or control unit associated with the focusing device is configured to provide a focus characteristic curve of the respective image capture device for assigning a respective focus setting to a respective magnification by the scaling means of the image capture device and / or a respective distance from the area to be observed, particularly in the case of the mechanical scaling function. The focus characteristic curve preferably comprises a curve or characteristic curve from which a respective focus lens position emerges relative to a respective distance from the area to be observed for focusing the optics or the captured image.The respective focus characteristic curve can be assigned to a respective image capture device with an associated focus lens in the storage and / or control unit, wherein the respective setting values for a focus lens assigned to the image capture device can be predefined and / or can be taught or taught based on empirical measurement data with the respective image capture device and the associated robotic holder arm.
[0029] In a preferred embodiment, the control device is designed to compare a target value for the scaling detected by the input unit with a minimum and maximum scaling factor, as well as a provided actual scaling factor of the scaling means, i.e. a current zoom factor of the scaling means, and based thereon, to control a selective or combined control of the optical and / or electronic scaling means and the robotic holding arm to provide the mechanical scaling function. In this case, the control device can further be designed to carry out a preferred control of the optical and / or electronic scaling means and / or the mechanical scaling function based on the detected target value for the scaling and a comparison with a current actual scaling factor of the scaling means.In a particularly preferred embodiment, the control device is configured to first control the optical and / or electronic scaling means until a minimum or maximum scaling factor of the scaling means is reached, and only then to perform a further adjustment of the scaling by controlling the robotic holding arm and thus by the mechanical scaling function. To provide the mechanical scaling function, the control device is preferably configured to calculate a necessary change in the holding arm position, in particular a position change of the image capture device arranged on the holding arm, along an optical axis of the image capture device based on the detected input command, in particular based on a target value for the scaling of the recorded image.In this case, the control device is preferably designed to output one or more target values for a movement, in particular a change in position of the robotic holding arm along the optical axis of the image capture device for changing its position. The respective position change can then be carried out by controlling the robotic holding arm via the control device itself or a separate control unit of the robotic holding arm, to which the target values are provided or transmitted. Further preferably, the control device is designed to control the focusing device and, in particular, adjust the position of the focus lens of the image capture device based on the calculated change in the holding arm position, in particular a change in the axial distance from the area to be viewed.The focusing device is preferably controlled by reading out a stored focus characteristic curve of the image capture device, as described above.
[0030] The control device can be designed in such a way that, when scaling the recorded image by adjusting the optical and / or electronic scaling means and / or the mechanical scaling function, a simultaneous and / or subsequent control of the focusing device for tracking the focus and preferably based on a stored focus characteristic curve of the respective image capture device is determined.
[0031] In a preferred embodiment, the input unit of the device is designed such that it enables variable, in particular deflection-dependent, setpoint detection of a scaling factor. The input unit preferably comprises input means with multiple degrees of freedom, which are designed to detect a deflection in a respective direction. In particular, the input unit can comprise a 3D joystick, which, in addition to detecting lateral input commands in a first plane of movement, enables detection of vertical input commands in a direction perpendicular to the first plane of movement. Furthermore, the 3D joystick can be designed to detect a deflection about a vertical axis of rotation and / or a tilt axis preferably arranged perpendicular thereto.The target value for the scaling factor can preferably be detected by detecting a vertical input command, in particular in a first direction for reduction and in a second direction, opposite to the first, for enlargement. The input unit can be operated by a user with the hand or with a foot.
[0032] The input unit can further be configured for selectively controlling the image capture device and / or the movable support arm. In this case, the input unit can, for example, have a selective input function, such as a manually operable button or foot switch, by means of which a distinction can be made between controlling the support arm by the input unit, in particular for changing the position and / or orientation of the support arm, and detecting a desired scaling factor or detecting an adjustment of the current scaling factor.
[0033] In a preferred embodiment, the image capture device is a conventional stereo exoscope. The stereo exoscope preferably has at least optical, controllable scaling means, in particular an optical zoom, as well as a motor-driven focus control. The image capture device can further have a distally arranged mirror unit for deflecting the viewing or capture direction by 90°. The mirror unit can optionally be designed to be movable or rotatable. The image capture device can, for example, have two to four image sensors, which are designed to capture a (stereo) image with a resolution of preferably 4K or higher.
[0034] In a further preferred embodiment, the image capture device comprises an autofocus function which can be activated and / or controlled by the control device. For example, the control device can be designed to activate a preferably one-time, i.e., non-continuous, autofocus function of the image capture device after completion of a holding arm movement. Since the present control of the mechanical scaling function or of the holding arm takes place without feedback of sensor data from a holding arm, the control device is designed, for example, such that, based on the respectively calculated target values for a movement of the holding arm, a time estimate is made of how long the robotic holding arm requires to implement the desired or calculated movement, and based on this, the autofocus function is controlled after the previously calculated time interval has elapsed.
[0035] An autofocus function of the image capture device can be implemented using means known per se; in particular, the autofocus function can comprise a contrast measurement, a phase comparison or an active distance measurement.
[0036] In a preferred embodiment of the method according to the invention, scaling and adjustment of the focusing device or focus tracking of the captured image is carried out by controlling the scaling means, the robotic holding arm, and the focusing device based solely on the input command for scaling the captured image detected by the input unit, and in particular without reading or evaluating position or movement data of the robotic holding arm. Scaling of the captured image and focusing of the captured image can occur simultaneously or sequentially.In a preferred embodiment of the method, based on a detected target value for scaling the recorded image, in particular for a desired magnification of the recorded image, scaling is first carried out by controlling the scaling means of the image capture device and only when a maximum scaling factor of the image capture device is exceeded is scaling carried out by changing the axial distance between the image capture device and the area to be viewed by changing the position of the holding arm.Furthermore, control can advantageously be carried out in such a way that, when a reduction in the recorded image is desired, scaling is first carried out by changing the axial distance between the image capture device and the area to be viewed by changing the position of the holding arm, in particular by increasing the axial distance, and only when a predefined scaling factor is exceeded is the scaling means of the image capture device activated.
[0037] However, the reverse sequence is also possible, so that when scaling with the desired magnification, mechanical scaling is first performed by the support arm, for example, if the control system detects that the desired target scaling cannot be achieved using the scaling means of the image capture device alone. Further scaling is then performed using the optical and / or electronic scaling means. The sequence is the same for reduction.
[0038] In a further preferred embodiment, controlling the focusing device when the holding arm position changes comprises at least the following additional steps: determining an initial focus setting before executing a movement of the holding arm; calculating the necessary change in the holding arm position along the optical axis of the image capture device to provide the detected target value for scaling the captured image; calculating a target value for the focus setting based on the calculated change in the holding arm position and on the basis of a focus characteristic curve stored for the image capture device; controlling the focusing device to set the determined target value. Further advantages and details of the invention will become apparent from the following description of preferred embodiments of the invention and from purely schematic drawings.
[0039] They show:
[0040] Fig. 1: a schematic view of an embodiment of a device according to the invention;
[0041] Fig. 2: a side view of an embodiment of an image capture device according to the invention;
[0042] Fig. 3: a block diagram of the device components; and
[0043] Fig. 4: a schematic flow diagram outlining the steps of a preferred embodiment in scaling and focus tracking,
[0044] Fig. 5 is a schematic representation of the application areas of optical scaling and mechanical scaling means; and
[0045] Fig. 6 is a schematic representation of a focus characteristic curve.
[0046] Identical elements or elements with the same function are provided with the same reference numbers in the figures.
[0047] Fig. 1 shows a schematic representation of an embodiment of a device 100 according to the invention. The device is designed for the scalable visual representation of an area 101 to be viewed, in particular an operating area, shown schematically here only as an example. The area 101 to be viewed can be an operation site. The device 100 has an image capture device 102 which is designed to capture a recorded image, in particular a live video image. The image capture device 102 can be arranged or can be arranged, for example, at a distance of 20 to 50 cm above an area to be viewed. The image capture device 102 is arranged on an end section or manipulator 103a of a robotic holding arm 103. The robotic holding arm 103 can be arranged on a stand element 104, such as a selectively movable carriage.Alternatively, ceiling or wall mounting in a treatment room is also possible. The image capture device 102 is also connected to a display unit 105, on which the captured and selectively scalable image is output. The device 100 further comprises an input unit 106 and preferably a foot switch 107 associated with it. The foot switch 107 can be configured to selectively switch the control of the robotic support arm 103 or the image capture device 102. The input unit 106 is preferably configured as a 3D joystick and preferably enables the detection of a vertical movement (double arrow C) in addition to the detection of lateral input commands (double arrows A, B). Furthermore, the input unit 106 can be configured to detect a rotational movement and / or a tilting movement of the 3D joystick (not shown).The input unit 106 is designed, in particular, to detect an input command for scaling or changing the scaling of the recorded image shown on the display unit 105. In this case, the input unit is preferably designed to detect a deflection-dependent setpoint value of the scaling factor, in particular a deflection-dependent change of a current scaling factor. This means that a greater deflection of the 3D joystick along the double arrow C leads to a greater enlargement or reduction of the recorded image on the display unit 105. Scaling, i.e., a reduction or enlargement, of the recorded image can be carried out by means of optical and / or electronic scaling means 108, 109 assigned to the image capture device 102 (see Fig. 3) and / or by means of a mechanical scaling function 110 provided by the holding arm (see Fig.3), in which the robotic holding arm moves the image capture device 102 held thereon relative to the area 101 to be viewed along an optical axis 111 of the image capture device 102 in order to adapt an axial distance between the image capture device and the area to be viewed (double arrow D).
[0048] Fig. 2 shows a side view of a preferred embodiment of the image capture device 102. The image capture device is in this case a stereo exoscope known per se. This preferably has motor-controlled optical scaling means, i.e. an optical zoom 108 assigned to or comprised by an optics of the exoscope, and a preferably motor-controlled focusing device 112, i.e. a focus lens assigned to or comprised by an optics of the exoscope. The image capture device 102 further comprises a distally arranged mirror unit 113 for a 90° deflection of the viewing or capture direction. Furthermore, the image capture device 102 can have a filter wheel 114 for receiving different, application-specific color filters. The image capture device can additionally have an illumination unit (not shown) for illuminating the area to be viewed.
[0049] Fig. 3 shows a block diagram of the device components of an embodiment of the device according to Fig. 1. The device 100 comprises a control device 115, which is connected to the image capture device 102, the robotic holding arm 103, and the input unit 106. The control device 115 can comprise a central control unit with a processor and associated storage means (not shown). According to the invention, the control device is designed such that it performs the adjustment of the mechanical scaling function 110 and the focusing device 112 by controlling the holding arm 103 and the focusing device 112 without regulation by recourse to sensor values of the holding arm 103, in particular without actual values for position or movement data of the holding arm 103.In particular, the control device 115 is intended to control the holding arm 103 and the focusing device 112 solely based on an input command acquired by the input unit 106 for scaling the captured image or for changing the scaling of the captured image. The control unit 115 is configured to control the mechanical scaling function 110 of the holding arm 103 by varying an axial distance d between the image capture device 102 and the area 101 to be viewed.
[0050] The control device 115 is configured such that it can control an optical scaling function 108 of the image capture device 102, in particular by adjusting the lens optics, and / or an electronic scaling function 109, in particular by displaying a partial section of the digital image area on the display unit 105. In this case, the control device 115 is preferably configured to read out a respective actual value of the optical and / or electronic scaling means 108, 109, in particular a current magnification or zoom factor. For this purpose, the optical and / or electronic scaling means 108, 109 can have an internal storage and / or control unit, which stores a respective minimum and maximum scaling factor in addition to an actual value for a given scaling factor.
[0051] The control device 115 is further designed for the selective or combined control of the scaling means 108, 109 and the mechanical scaling function 110 in order to provide an extended scaling range or an extended overall zoom range (see also Fig. 5). In this case, the control device 115 enables focus tracking of the recorded image by preferably simultaneously or sequentially controlling the focusing device 112. The focusing device 112 preferably also has an internal memory and / or control unit, which can provide a current actual value of a focus setting of the focusing device 112 to the control device. Furthermore, a respective focus characteristic curve 116 for the respective image capture device oran optics or identification number of the image capture device 102 is stored, from which the relationship between a respective focus setting or a position of the focus lens Fp in the optics of the image capture device 102 and the respective distance d of the image capture device 102 or of the optics of the image capture device 102 to the area 101 to be viewed is derived. In particular, the stored focus characteristic curve can be used to determine the position of the focus lens to which the respective focusing device is to be set in order to achieve a sharp focus of the recorded image at a specific distance d between the capture optics and the area 101 to be viewed.
[0052] An embodiment of the steps for scaling and focus tracking with the device according to the invention is described below in connection with Fig. 4.
[0053] Fig. 4 shows a flow chart 200 for an inventive control of optical scaling means 108 of an image capture device 102 and the mechanical scaling function 110 for providing an extended scaling, with tracking of a focus setting, for example for an enlargement of the recorded image.
[0054] It is assumed that, before using the device, a manual and / or electronic adjustment of the position of the image capture device 102 relative to the area 101 to be viewed takes place, in particular by a desired alignment of the robotic support arm 103 and the image capture device 102. The user can select a desired viewing angle and a desired distance d between the image capture device 102 and the area 101.
[0055] In a first step 201, an initial focus adjustment of the captured image is carried out by manual and / or electronic adjustment of a focusing device 112 assigned to the image capture device 102. Furthermore, an initial scaling or magnification factor can additionally be set by manual and / or electronic adjustment, for example of the optical scaling means 108.
[0056] In step 202, the control device 115 reads out the scaling means 108 and the focusing device 112 to provide actual values for a current scaling factor of the image capture device 102 and a focus setting or a set position of the focus lens of the optics of the image capture device 102. As a result, the focus distance or the distance d between the image capture device 102 and the viewing area 101 can be determined using the stored focus characteristic curve (position Po, see Fig. 6).
[0057] Step 202 can be triggered, for example, by pressing a foot switch 107, whereby the foot switch 107 can serve as a trigger for changing a scaling by inputting an input to the input unit or the 3D joystick 106. Alternatively, step 202 can also be triggered directly at the input unit 106.
[0058] In step 203, the input of a desired scaling factor or a desired change in the scaling factor is detected by the input unit 106. The input unit 106 can detect a target value for a desired magnification, which is transmitted to the control device 115. In step 204, based on the user input or the desired magnification, a new scaling or magnification factor is calculated, which is to be set by the device. A comparison is made with the detected actual scaling factor and a minimum and maximum value for the scaling factor of the scaling means 108. Magnification by the scaling means 108 of the image capture device 102 can occur until a stored maximum value of the scaling means 108 is reached, without the mechanical scaling function 110 being activated (see iteration loop 205).
[0059] If the desired scaling value exceeds the maximum value of the scaling means 108 (checked in step 206), a necessary distance change d1 to the area to be viewed 101 is calculated in step 207, which is necessary to achieve the desired remaining magnification. Based on this, a setpoint value is then output in step 208 to control the robotic holding arm 103 and thus to change and, in particular, shorten the distance d along the optical axis 111 toward a position Pi of the holding arm 103.
[0060] In step 209, the focusing device 112 is controlled by the control device 115 to adjust the image sharpness. Based on the input command for magnifying the captured image detected by the input unit 106, the focus lens is adjusted using the stored focus characteristic curve. In particular, the control device 115 determines the necessary adjustment of the focusing device or the position Fi of the focus lens in the optics of the image capture device 102 based on the previously determined distance change d1 and thus the new position Pi (see Fig. 6).
[0061] If, starting from the new position Pi, a reduction in the scaling factor is desired again or is to be detected by the input unit 106, the control device 115 preferably controls the mechanical scaling function 110 to reduce the scaling factor. In particular, the axial distance d is increased until the previously controlled distance change d1 is reached. The scaling means 108 of the image acquisition device 102 are then controlled.
[0062] If a further reduction of the scaling factor is desired or detected by the input unit 106, the scaling means 108 are controlled to reduce the scale until a stored minimum value of the scaling means is reached. Once the minimum value of the scaling means 108 is reached, the mechanical scaling function 110 is then controlled by again increasing the axial distance d, i.e., the image capture device 102 is moved further away along the optical axis 111.
[0063] The control by the control device 115 described above is explained again below with reference to the schematic representation in Fig. 5, which shows a scaling range 120 of the exemplary optical scaling means 108 of the image capture device 102. Starting from an initially defined distance dini between the image capture device 102 and the area 101 to be viewed, the optical scaling means 108 are initially controlled upon magnification (arrow 121) or reduction (arrow 122) until a maximum scaling factor Smax or a minimum scaling factor Smin is reached.If the maximum scaling factor is exceeded or the minimum scaling factor is undershot, the mechanical scaling function 110 is activated, either to enlarge by shortening the axial distance d between the image acquisition device 102 and the operating area 101 in the area 123 or to reduce by increasing the axial distance in the area 124.
[0064] If a respective return of the scaling factor takes place, ie from the areas outside the minimum and maximum scaling factors Smin, Smax of the image capture device 102 to the area 120, the mechanical scaling function is first controlled to reduce or increase the scaling factor until a distance is reached which lies within the area 120, at which the optical scaling means 108 can be used.
[0065] The control described above can be dependent on the activation of a foot switch. For example, if the foot switch has been released after a reduction (arrow 122) to range 124, then, in deviation from the control described above and, for example, in the case of a (further) enlargement of a scaling factor from range 124, the optical scaling means 108 can be activated immediately. Only when a maximum possible scaling factor of the optical scaling means 108 is reached can the mechanical scaling function 110 be activated for further enlargement.
[0066] List of reference symbols
[0067] 100 device
[0068] 101 area to be considered
[0069] 102 Image capture device
[0070] 103 robotic holding arm
[0071] 103a end section
[0072] 104 cars
[0073] 105 Display
[0074] 106 Input unit
[0075] 107 foot switches
[0076] 108 optical scaling / zoom
[0077] 109 electronic scaling tools / zoom
[0078] 110 mechanical scaling function
[0079] 111 optical axis
[0080] 112 Focusing device
[0081] 113 Mirror unit
[0082] 114 Filter wheel
[0083] 115 Control device
[0084] 116 Focus characteristic
[0085] 120 zoom range of the optical / electronic scaling means
[0086] 121 magnification
[0087] 122 Reduction
[0088] 123 Magnification range mechanical scaling function
[0089] 124 Reduction range mechanical scaling function d axial distance d1 distance change dini initial distance
[0090] Smax maximum scaling factor opt. scaling means
[0091] Smin minimum scaling factor opt. scaling center
[0092] Po,1 Holding arm positions Fp Position focus lens
Claims
Patent claims 1. An imaging device (100) for the scalable visual representation of an area to be viewed (101), in particular an operating area, comprising: an optical image capture device (102) for capturing a recorded image of the area to be viewed (101), optical and / or electronic scaling means (108, 109) associated with the image capture device (102) for scaling the captured image, and an adjustable focusing device (112) associated with the image capture device, a robotic holding arm (103) for moving the image capture device (102) relative to the area to be viewed (101), wherein the holding arm (103) is designed to provide a mechanical scaling function (110) by adapting an axial distance (d) between the image capture device (102) and the area to be viewed (101),an input unit (106) for detecting a user-side input command for scaling the captured image on a display unit (105), a control device (115) for adjusting the scaling means (108, 109) of the image capture device (102) and the mechanical scaling function (110) of the holding arm, as well as the focusing device (112), characterized in that the control device (115) is designed to perform the adjustment of the mechanical scaling function (110) and the focusing device (112) by controlling the holding arm (103) and the focusing device (112) solely based on the input command for scaling the captured image detected by the input unit (106). Device according to claim 1, characterized in that the control device (115) is designed to control the scaling means (108, 109), the mechanical scaling function (110) of the holding arm (103) and the focusing device (112) based only on the input command for scaling the recorded image detected by the input unit (106), and in particular without reading out or evaluating position or movement data of the robotic holding arm (103). Device according to claim 1 or 2, characterized in that the input unit (106) is designed to detect a target value for the scaling of the recorded image and in that the control device (115) is designed to control the scaling means of the image capture device (108, 109), the mechanical scaling function (110) of the holding arm (103), and the focusing device (112) based on the target value detected by the input unit (106).Device according to one of claims 1 to 3, characterized in that the mechanical scaling function (110) of the holding arm (103) is carried out by a change in the position of the image capture device (102) along its optical axis (111). Device according to one of the preceding claims, characterized in that the scaling means (108, 109) is assigned a preferably internal storage and / or control unit for providing a minimum and maximum scaling factor as well as a current actual scaling factor to the control device (115). Device according to one of the preceding claims, characterized in that the focusing device (112) is assigned a preferably internal storage and / or control unit for providing a minimum and maximum scaling factor. maximum focus value / focus setting and a current actual focus value / focus setting is assigned to the control device (115). Device according to one of the preceding claims, characterized in that the control device (115) and / or a preferably internal storage and / or control unit assigned to the focusing device (112) is designed to provide a focus characteristic curve of the image capture device (102) for assigning a respective focus setting to a respective distance (d) from the area to be observed (101).Device according to one of the preceding claims, characterized in that the control device (115) is designed to compare a target value for the scaling detected by the input unit (106) with a minimum and maximum scaling factor, as well as the actual scaling factor of the scaling means (108, 109) and, based thereon, to control a selective or combined control of the optical and / or electronic scaling means (108, 109) and of the robotic holding arm (103) to provide the mechanical scaling function (110).Device according to one of the preceding claims, characterized in that the control device (115) is designed to first control the optical and / or electronic scaling means (108, 109) until a minimum or maximum scaling factor of the scaling means (108, 109) is reached and only then to carry out a further adjustment of the scaling by controlling the robotic holding arm (103). Device according to one of the preceding claims, characterized in that the control device (115) is designed to determine a necessary change in the holding arm position, in particular a position change of the, based on a target value for the scaling of the recorded image. on the holding arm arranged image capture device (102), along an optical axis (111) of the image capture device (102) and based on this change in the holding arm position, in particular a change in the axial distance (d) to the area to be viewed (101 ) to control the focusing device (112).
11. Device according to claim 10, characterized in that the control device (115) is designed to determine, when scaling the recorded image, a simultaneous and / or subsequent control of the focusing device (112) for tracking the focus based on a stored focus characteristic of the respective image capture device (102).
12. Device according to one of the preceding claims, characterized in that the input unit (106) enables a variable, in particular deflection-dependent, setpoint detection of a scaling factor and optionally for the selective control of the image acquisition device (102) and / or the movable holding arm (103).
13. Device according to one of the preceding claims, characterized in that the image acquisition device (102) comprises a stereo exoscope with preferably at least optical scaling means (108) and a motor-driven focusing device (112).
14. Device according to one of the preceding claims, characterized in that the movable robotic holding arm (103) has its own control unit which is designed for bidirectional data communication with the control device (115) of the device (100).
15. Device according to one of the preceding claims, characterized in that the control device (115) is designed, upon termination to activate a preferably one-time, ie non-continuous, autofocus function of the image capture device (102) by means of a holding arm movement.
16. Method for the scalable visual representation of an area to be viewed, in particular an operating area, comprising at least the following steps: - capturing a recorded image of the area to be viewed (101) with an optical image capture device (102) for display on a display unit (105), - initial focus adjustment of the captured image by manual and / or automatic adjustment of a focusing device (112) associated with the image capture device, - detecting an input command for scaling the recorded image by means of an input unit (106), - scaling of the recorded image on the display unit (105) by setting optical and / or electronic scaling means (108, 109) associated with the image capture device and setting a mechanical scaling function (110) by moving the image capture device (102) along an optical axis (111) of the image capture device (102) by means of a robotic holding arm (103), - Focusing the scaled recorded image by adjusting the focusing device (112), characterized in that the adjustment of the mechanical scaling function (110) and the focusing device (112) is carried out by controlling the holding arm (103) and the focusing device (112) solely based on the input command for scaling the recorded image detected by the input unit (106).
17. Method according to claim 16, characterized in that when scaling the recorded image and adjusting the focusing device (112) the scaling means (108, 109), the robotic holding arm (103) and the focusing device (112) are controlled solely on the basis of the input command for scaling the recorded image detected by the input unit (106) and in particular without reading out or evaluating position or movement data of the robotic holding arm (103).
18. Method according to claim 16 or 17, characterized in that scaling of the recorded image and focusing of the recorded image preferably take place simultaneously or sequentially.
19. Method according to one of claims 16 to 18, characterized in that based on a detected target value for scaling the recorded image, scaling is first carried out by controlling the scaling means (108, 109) of the image capture device (102) and only when a maximum scaling factor of the image capture device is exceeded is scaling carried out by changing the axial distance (d) between the image capture device (102) and the area to be viewed (101) by changing the position of the holding arm.
20. Method according to one of claims 16 to 19, characterized in that a control of the focusing device (112) when changing the holding arm position comprises at least the following further steps: - Determination of an initial focus setting before executing a movement of the holding arm (103), - Calculating the necessary change in the holding arm position along the optical axis (111) of the image capture device (102) to provide the detected target value for scaling the captured image, - Calculating a target value for the focus adjustment based on the calculated change in the holding arm position and on the basis of a focus characteristic curve stored for the image capture device (102), - Control of the focusing device (112) to set the determined target value.