Method for controlling a medical visualization system, and medical visualization system

Abstract

The invention relates to a method for controlling a medical visualization system and to a medical visualization system, the method having the step of: - receiving a surroundings image (A10) recorded by a surroundings detection device (11) and a microscopy image (A11) of an examination region recorded by the image detection device (11) for microscopy imaging purposes, - generating an augmented surroundings image (AA10) by superimposing the surroundings image (A10) with the microscopy image (A11) in a microscopy image region (B11), a sub-region (TB11) of the examination region which can be recorded by the image detection device (11) for microscopy imaging purposes being imaged into the microscopy image region (B11), and by superimposing the surroundings image region (A10) with at least one operating element (BE1,..., BE6) in an image region which lies outside the microscopy image region (B11), - actuating a display device (9) of the medical visualization system (1) so as to output the augmented surroundings image (AA10), - determining an image position (BLO) of an imaged object (O) by evaluating at least one surroundings image (A10), and - generating an input signal (ES) on the basis of the image position (BLO) of the imaged object (O) and the image position of the at least one operating element (BE1,..., BE6).

Description

[0001] Method for controlling a medical visualization system and medical visualization system

[0002] The invention relates to a method for controlling a medical visualization system and a medical visualization system.

[0003] Surgical microscopes are used, among other things, to prepare for and perform medical operations on a patient. These microscopes are used by a user, such as a surgeon or assistant, during a procedure to provide a magnified view of an area of ​​examination, particularly within or on the patient's surgical site. For this purpose, a surgical microscope may include an objective lens or lens system to create a true optical image of the area under examination. The objective lens may include optical elements, such as a lens, for beam guidance, shaping, and / or direction.

[0004] Surgical microscopes are used in medical facilities, as well as in laboratories and industrial applications. Examples of medical applications include neurosurgery, ophthalmic surgery, otolaryngology (ENT), plastic or reconstructive surgery, and orthopedic surgery. This list is not exhaustive. Generally, they are used in all areas of surgery where a magnified, high-resolution view of the surgical field is required to perform precise procedures.

[0005] A distinction can be made between analog and digital surgical microscopes. Unlike digital surgical microscopes, analog surgical microscopes do not capture images that are then displayed, for example, on a screen to magnify the examination area. Instead, they provide the user with a directly visual magnification of the examination area. Here, radiation reflected or scattered from the area of ​​application passes through the objective lens into at least one beam path and to at least one output section, through or into which the user looks to visually perceive the radiation and thus also the typically magnified representation of the examination area. An exemplary embodiment of an output section is a so-called eyepiece, into or through which the user looks to optically perceive the examination area with at least one eye. Such an eyepiece is usually arranged on a microscope head.Analog surgical microscopes usually also include handles arranged on a microscope head for positioning, such handles also having operating elements for controlling the surgical microscope, e.g. in the form of switches.

[0006] Digital surgical microscopes comprise, or at least include, an image acquisition device for microscopic imaging that captures radiation in a beam path of the surgical microscope to generate a magnified image. This image can be displayed to the user or multiple users on one or more display devices. This enables high-resolution visualization. The image can be generated as a transmittable image signal, which encodes or represents the image. Purely digital surgical microscopes, unlike analog surgical microscopes, do not have an output section for visually detectable radiation, specifically no eyepiece. The image signal can then be transmitted as a data signal, either wired or wirelessly.

[0007] Digital surgical microscopes enable the capture, storage, and further processing of images and videos. By applying image processing techniques, contrast, brightness, and other parameters can be adjusted to optimize the image quality of the generated images. Hybrid surgical microscopes can include at least one image acquisition unit and at least one output section. For example, the radiation guided in a beam path of the surgical microscope can be split by a beam splitter, with one portion directed to the output section and another portion captured by the at least one image acquisition unit.

[0008] The operating microscope can constitute a medical visualization system, or the medical visualization system can encompass the operating microscope. The components of the medical visualization system described below can be components of the operating microscope or components designed differently from the operating microscope.

[0009] Microscopy systems are also known that include a so-called tracking camera for determining the position of at least one object to be examined. Such a microscopy system is disclosed, for example, in WO2022 / 234099 A1. Such a tracking camera can be part of a position detection device or tracking system of the microscopy system. EP 3 285 107 B1 also discloses a surgical microscope with a field of view and with an optical imaging system designed to image an inspection area that is at least partially within the field of view. This publication further discloses a method for gesture control of the surgical microscope, wherein a projection unit can project optional and / or interactive images onto a real image of an inspection area. According to the teaching of this publication, the inspection area is the area of ​​an examination region that is imaged by an optical system for magnification.However, this projection has the disadvantageous effect of potentially impairing the visual perception of a magnified representation of the inspection area.

[0010] The digital surgical microscopes described allow for a separation of user and device, as there is no longer a need for an output section. For this reason, the arrangement of handles on the microscope head can also be eliminated, which in turn eliminates the controls located there. Even if handles for positioning are still present, they may be difficult or impossible for a user to reach, especially when setting large working distances.

[0011] Alternative operating methods are known, such as those based on voice or gesture control, or those using separately arranged controls like foot and hand switches. Foot switches, however, require floor space, which may be problematic if the space is used for additional foot switches, for example, for an electrosurgical unit. Furthermore, foot switches may not be precise when operating while standing, and fine adjustments, in particular, cannot be made with sufficient accuracy. A hand switch is typically attached to a standard rail on an operating table, which makes it difficult to reach for certain types of surgery, such as cranial procedures.

[0012] Furthermore, this presents a sterility issue. Control panels are also connected to the operating microscope either via cables (creating an obstruction and tripping hazard) or wirelessly, which poses both a tripping hazard and an additional obstruction in the operating room, as well as a risk of failure. In the case of wireless connections, it must also be ensured that sufficient power is always available for operation and that any necessary batteries are adequately charged. This requires a corresponding infrastructure, which must be provided either separately or as part of the system, although there is still a risk that the necessary charging process will be forgotten by the operating room staff. Voice control is particularly well-suited for clear, short, and one-off commands. However, continuous changes to zoom, focus, and / or light intensity values ​​may be difficult to implement via voice control.Gesture control requires the reliable recognition of various hand movements, which cannot be easily guaranteed in a surgical setting. Furthermore, the microscopic field of view, particularly during microsurgical procedures, is too small to provide sufficient space for the execution of visually controlled gestures, especially when a hand or finger cannot be fully visualized.

[0013] The technical problem therefore arises of creating a method for controlling a medical visualization system and a medical visualization system that enable improved control of the microscopy system.

[0014] The solution to the technical problem is provided by the articles with the features of the independent claims. Further advantageous embodiments of the invention are described in the dependent claims.

[0015] A method for controlling a medical visualization system is proposed. This system includes, in particular, an operating microscope or can be formed by the operating microscope. Operating microscopes and their technical characteristics have already been briefly explained. The operating microscope can be, in particular, a digital operating microscope, and furthermore, a purely digital operating microscope. Such a visualization system, in particular the operating microscope, can include a microscope head. The objective lens described above can be integrated into the microscope head or attached to it, in particular detachably. The objective lens can be fixed in position relative to the microscope head. In addition to the objective lens, the microscope head can also have or incorporate at least one beam path for microscopic imaging and / or other optical elements for beam guidance, shaping, and / or deflection.In analog and hybrid surgical microscopes, but not in purely digital surgical microscopes, the microscope head may include at least one mounting interface for attaching an output element, such as an eyepiece, which may be detachable. The microscope head may comprise or form a housing, or be arranged within a housing.

[0016] The medical visualization system can include a stand for mounting the operating microscope. The operating microscope, in particular the microscope head, can be mechanically attached to the stand, especially at a free end, thus forming an end effector of the stand. The stand is designed to allow movement of the operating microscope in space, in particular with at least one degree of freedom, preferably with six degrees of freedom, where one degree of freedom can be translational or rotational. Furthermore, the stand can include at least one drive unit for moving the operating microscope. Such a drive unit can, for example, be a servo motor. Naturally, the stand can also include means for force / torque transmission, e.g., gear units.In particular, it is possible to control the at least one drive unit in such a way that the operating microscope performs a desired movement and thus a desired change of position in space, or assumes a desired position and / or orientation in space. For example, the at least one drive unit can be controlled in such a way that an optical axis of an objective lens of the operating microscope assumes a desired orientation. Furthermore, the at least one drive unit can be controlled in such a way that a reference point of the operating microscope, e.g., a focal point, is positioned at a desired position in space. A target position can be specified by a user or another higher-level system. Methods for controlling the at least one drive unit as a function of a target position and a kinematic structure of the stand are known to those skilled in the art.

[0017] The medical visualization system comprises at least one image acquisition device for microscopic, i.e., magnifying, imaging, which is hereinafter referred to as the microscopic image acquisition device, and at least one environmental sensing device, wherein the microscopic image acquisition device and the environmental sensing device are each part of the microscope head. It is also possible that the medical visualization system comprises a stereo system for microscopic imaging, which includes two image acquisition devices.

[0018] In particular, the scanning devices are arranged together in a housing, especially the microscope head. A scanning area of ​​the microscopic image acquisition device is located entirely within a scanning area of ​​the ambient scanning device, at least within a predetermined working distance range. The scanning area of ​​the ambient scanning device is larger than the scanning area of ​​the image acquisition device for microscopic imaging, at least within this predetermined working distance range. The working distance range can, for example, include values ​​greater than 100 mm or greater than 200 mm. The fact that the scanning area of ​​the ambient scanning device is larger than the scanning area of ​​the microscopic image acquisition device means that the scanning area of ​​the microscopic image acquisition device is located entirely within the former, with the scanning areas overlapping only in a partial region.In particular, the maximum diameter of the detection area of ​​the environmental detection device (first maximum diameter) in a plane oriented perpendicular to an optical axis of the microscopic image acquisition device and spaced along this axis at a predetermined working distance from the visualization system can be larger than the maximum diameter of the detection area of ​​the microscopic image acquisition device (second maximum diameter), e.g., more than 1.1 times larger. Specifically, the first maximum diameter can be twice the size of the second maximum diameter at a working distance of 200 mm. The working distance can be defined as the distance between a focal plane and an end element of an objective system of the visualization system along the optical axis of the microscopic image acquisition device or the microscope, which may be defined by the objective or objective system of the microscope.

[0019] The environmental sensing device can be part of a position sensing device of the visualization system, whereby the position sensing device can determine the spatial location or pose, i.e., the position and / or orientation, of at least one object to be detected, particularly in three dimensions. Such a position sensing device can also be referred to as a tracking system. Such a tracking system can be an optical tracking system. The tracking system can be a marker-based tracking system that detects active or passive markers. Markers can be arranged on objects or subjects whose pose is to be detected by the tracking system. An optical tracking system can, in particular, include optically detectable markers. An optical tracking system can, in particular, be a system for monoscopic position sensing.The pose of an object can be determined by evaluating a two-dimensional image, in particular exactly one two-dimensional image (monoscopic position detection). Specifically, the position can be determined by evaluating the intensity values ​​of pixels (picture elements) of the two-dimensional image. Thus, the position detection device can include an evaluation or computing unit that can perform position determination by evaluating the images generated by the environmental detection device. The pose of the object to be detected can be determined in a reference coordinate system, particularly a three-dimensional one. The object to be detected can be a user's hand or finger. It can also be an instrument, specifically a medical or surgical instrument. This includes, for example...Instruments such as clamps, holders, syringes, tweezers, spoons, scissors, scalpels, wound retractors, forceps, suction devices, cautery devices, and also refractors, e.g., a brain refractor. The instrument can be, in particular, a hand-held instrument. The object can also be a marker element or a target, whereby a target can comprise several marker elements, especially with a predetermined geometric arrangement. It is possible that one or more marker elements or a target are arranged on an instrument, a finger, or a hand. Then their pose can be determined depending on the pose(s) of the marker element(s). Furthermore, the position detection device, in particular its evaluation device, can detect at least one object to be detected, especially image-based, i.e., via image processing and evaluation methods.

[0020] Furthermore, the medical visualization system can include one or more display devices for showing the images. The display device can be used to show two- or three-dimensional images. A three-dimensional image can, in particular, be or comprise a stereo image pair, wherein the images of this image pair are stereoscopic images. Typical display devices are screens, especially 3D screens, head-mounted displays (HMDs), or digital eyepieces, which can also be referred to as booms.

[0021] The visualization system can also include means for introducing, in particular for reflecting, additional information into a beam path of the environmental detection device and, if applicable, also into a beam path of the image acquisition device for microscopic imaging. For example, such additional information can be projected by means of a projection device onto a projection element, e.g., a particularly radiolucent disc, which is arranged in the beam path. Thus, additional information can be superimposed on an image, in particular in a less transparent or at least partially transparent manner.

[0022] Furthermore, the medical visualization system, in particular the operating microscope, may include one or more of the following elements:

[0023] • at least one white light lighting device,

[0024] • at least one infrared illumination device, • at least one fluorescence illumination device for exciting fluorescence radiation,

[0025] • at least one beam filter to provide excitation radiation with wavelengths from a broader spectrum, e.g. the spectrum of the white light illumination device,

[0026] • at least one fluorescence detection device for detecting fluorescence radiation,

[0027] • at least one filter device for filtering radiation from a broader spectrum, e.g. for detection by an image acquisition device for microscopic imaging,

[0028] • at least one gaze direction detection device,

[0029] • at least one position detection device for determining a pose, i.e. a position and / or orientation, at least of the operating microscope

[0030] • at least one input device for operation,

[0031] • at least one interface for data transmission to or from another system or facility,

[0032] • at least one device for determining depth information, in particular with regard to the elements arranged in the detection range of the operating microscope, which may, for example, be designed as a distance sensor,

[0033] • at least one storage device for storing signals and / or information, especially in a retrievable manner.

[0034] An image acquisition device may, in particular, include a CMOS or CCD sensor.

[0035] The method comprises the step of receiving an environmental image captured by the environmental sensing device and an image of an area of ​​investigation captured by the image acquisition device for microscopic imaging, which is hereinafter also referred to as a microscopic image (receive step). The images can be, in particular, two-dimensional images. The microscopic image can also be a stereo image. The method can also include the step of generating these images. The images can be provided in the form of an image signal or a data signal, with the corresponding signal being received, for example, by the control and evaluation unit described above or by another control and evaluation unit of the visualization system, in particular via a signal interface.The microscopic image can be, in particular, a white light image (VIS image), generated especially with visible radiation, i.e., radiation from a wavelength range between 360 nm and 830 nm. The ambient image can be, in particular, an infrared image (IR image), generated especially with radiation in the infrared range, preferably in the near-infrared range, i.e., radiation from a wavelength range between 800 nm and 900 nm. The microscopic image can be, in particular, a color image, e.g., an RGB color image. The ambient image can be a monochromatic image.

[0036] According to the invention, the method further comprises the following steps.

[0037] In one image generation step, an augmented image of the surroundings is created by superimposing the microscopic image onto the surroundings. The microscopic image is superimposed onto the surroundings within a microscopic image area of ​​the surroundings. This microscopic image area represents a sub-area of ​​the investigation area captured by the environmental sensing device, corresponding to the sub-area that is (also) detectable or captured by the microscopic image acquisition device. The microscopic image can be scaled for this superimposition.

[0038] To generate the augmented environmental image, the microscopic image can be introduced into the beam path of the environmental detection device, e.g., by being reflected, for example, using a projection device. This was explained previously. Alternatively, the augmented environmental image can be generated by computer-aided enhancement of the environmental image, particularly through image processing. In this case, the microscopic image can be superimposed on the environmental image.

[0039] Furthermore, at least one control element is superimposed on the environmental image in an image area of ​​the environmental image that lies outside a microscopy image area. This can be achieved by superimposing a graphical or visually detectable representation of the control element onto the environmental image. A control element can be a graphical element, i.e., a virtual element, that can be displayed on a display device for visual detection. The superimposition of the environmental image with the at least one control element can be carried out in the same way as the superimposition with the microscopic image.

[0040] In one control step, a display unit of the medical visualization system is activated to output the augmented environmental image. This augmented environmental image can, in particular, represent the real examination area, which is enhanced, especially by computer, by overlaying or displaying at least one control element onto the representation of the real examination area. The augmented representation can thus be displayed to the user on the display unit in the form of the augmented environmental image.The user can thus visually perceive information about the area under investigation, information from an enlarged image of a sub-area of ​​the area under investigation, and information about at least one control element, whereby the visual perception of the information from the enlarged image is advantageously not impaired by the representation of the at least one control element.

[0041] If the medical visualization system includes an output section into which or through which the user looks in order to optically perceive the examination area with at least one eye, e.g. an eyepiece, then the augmented environmental image can be provided to a user alternatively or cumulatively via this output section in a visually perceptible manner.

[0042] In one determination step, the image position of at least one object depicted in the environment image is determined by evaluating at least one environment image. The depicted object can, in particular, be an object whose pose is to be captured by the position detection device. However, it is also conceivable that the depicted object is an object whose pose is not to be captured by the position detection device or is not captured at all. Even in this case, the object can be configured as described above and, in particular, be a hand, a finger, or an instrument. A method for object detection, especially image-based object detection, can also be used to detect the at least one object in the environment image for determining the image position.

[0043] The respective image position can be an image coordinate of a reference point of the object or element. It is also conceivable that the image position of the object encompasses the image coordinates of all pixels of the image into which the object is mapped, and the image position of the control element encompasses the image coordinates of all pixels of the image into which the control element is mapped.

[0044] In a signal generation step, an input signal is generated depending on the image position of the depicted object and the image position of at least one control element. For example, an input signal can be generated if at least one of the following image position-specific conditions is met, namely:

[0045] • as soon as a condition arises or exists in which the image positions differ from each other by less than a predetermined amount,

[0046] • as soon as a condition arises or exists in which the image positions differ from each other by less than a predetermined amount for a predetermined period of time,

[0047] • as soon as a condition arises or exists in which the image layers overlap, especially by more than a predetermined amount,

[0048] • as soon as a condition arises or exists in which the image positions overlap for a predetermined period of time, in particular by more than a predetermined amount,

[0049] • as soon as a state is established or when a state exists in which the image positions are in a predetermined relative position to each other or as soon as the relative position assumes a value from a predetermined relative position range, in particular for a predetermined period of time.

[0050] The predetermined time duration can be longer than half a second or longer than one second.

[0051] The input signal can be used to control an element of the visualization system, such as the previously described lighting device(s), and thus constitutes a control signal. For example, the input signal can be used to set a zoom value, a focus value, a working distance value, or one of several operating parameters of the visualization system. The input signal can continuously adjust a value to different values ​​within a range, and in particular, it can be continuously changed. It is also possible for the input signal to set a value to discrete values ​​from a set of possible values. For example, this can be used to set a desired operating state of the visualization system.

[0052] The input signal can be, in particular, a control-specific input signal, especially if multiple controls are present. Specifically, different controls can be assigned different input signals, e.g., input signals for controlling different elements. The input signal can encode the value of an adjustable (operating) parameter. This value can be determined depending on the image position of the displayed object, especially relative to the image position of the control. Alternatively or cumulatively, the input signal can encode a change in the value of an adjustable (operating) parameter, whereby the magnitude of the change can also be determined depending on the image position of the displayed object, especially relative to the image position of the control.

[0053] It is conceivable that different pixels or different sets of pixels or sub-areas of the control element

[0054] - different (absolute) values ​​of a parameter to be set,

[0055] - various percentage values ​​of a maximum value of a parameter to be set or of a difference between the maximum value and a minimum value of a parameter to be set or

[0056] - different levels of change are assigned.

[0057] If at least one pixel, or at least a predetermined number of pixels comprising more than one pixel, overlaps with a pixel(s) of the control element, the value of the parameter to be set, or a change in its value, can be determined based on the value(s) assigned to the overlapping pixel(s). Alternatively, a reference pixel of the displayed object can be defined, and the value of the parameter to be set, or a change in its value, can be determined based on the value assigned to the pixel of the control element that is overlapped by the reference pixel. The reference pixel can be defined, for example, for an area of ​​overlap of the displayed object, such as its geometric center.

[0058] If the currently set value deviates from a value to be set, the input signal can be generated in such a way that the change to the value to be set occurs abruptly or continuously.

[0059] The proposed method advantageously enables simplified and, in particular, contactless control of a medical visualization system, wherein operation is enabled with simultaneous visual acquisition of an enlarged image, an object for operation and also the control element, and the visual acquisition of the information from the enlarged image is advantageously not impaired by the representation of the at least one control element, and thus a reliable input signal can be generated.

[0060] In a further embodiment, a change in the image position of the depicted object is additionally determined, wherein the input signal is additionally generated depending on the change in the image position of the depicted object.

[0061] For example, an input signal can be generated if, in addition to at least one of the image position-specific conditions explained above, at least one of the following image position change-specific conditions is met:

[0062] • there is no change in image position or the change in image position is less than a predetermined threshold, especially for a predetermined duration,

[0063] • Despite the change in the object's position in the image, at least one image-position-specific condition has still been met.

[0064] • The image position change corresponds to a predetermined image position change or does not deviate from such a change by more than a predetermined amount, e.g., an image position change along at least one predetermined and, in particular, control-element-specific image direction, especially by more than a predetermined amount.

[0065] It is possible for different image position changes to be assigned different input signals. Thus, a single control element can be used to generate different input signals. For example, an image position change along a first image direction can generate a first, and specifically control-element-specific, input signal, while an image position change along a second image direction, different from the first, can generate another, and specifically control-element-specific, input signal, with the first input signal being distinct from the second. For example, the first input signal can cause a continuous change of a value in a first direction, e.g., to increase the value, while the second input signal causes a continuous change of the value in a third direction, e.g., to decrease the value.

[0066] This advantageously results in more reliable operation and more diverse possibilities for generating input signals. In a further embodiment, the image position of the depicted object is determined multiple times, at least during a predetermined period, particularly with a predetermined determination frequency. The input signal is only generated if at least one of the previously described image position-specific conditions is met during the predetermined period, i.e., for example, at each determination point during the predetermined period. This advantageously increases the reliability of input signal generation.

[0067] It is conceivable that the change in the image position of the depicted object is also determined multiple times, at least during a predetermined period, and in particular at a predetermined measurement frequency. The input signal is only generated if at least one of the previously described image position-specific conditions is met during the predetermined period, i.e., for example, at each measurement point during the predetermined period. This also advantageously increases the reliability of the input signal generation.

[0068] In a further embodiment, information about the spatial orientation of the depicted object is additionally determined by evaluating at least one image of the surrounding environment, with the input signal being generated depending on the information about the spatial orientation. The spatial orientation can be determined in a three-dimensional reference coordinate system, which is distinct from the image coordinate system. The three-dimensional reference coordinate system can, for example, be a coordinate system of the previously described position detection device.

[0069] For example, an input signal can be generated if at least one of the following spatial conditions is met, namely

[0070] • The spatial orientation corresponds to a predetermined spatial orientation or lies within a predetermined spatial orientation range, e.g. at a predetermined distance from the area of ​​investigation,

[0071] • The spatial position reaches the predetermined spatial position or a spatial position from the predetermined spatial position range and is maintained, if necessary, for a predetermined period of time.

[0072] In particular, a change in the spatial orientation of the depicted object can also be determined by evaluating several sequentially generated environment images, whereby the input signal is additionally generated depending on the information about the change in spatial orientation. For example, an input signal can be generated if at least one of the following spatial orientation-specific conditions is met, namely:

[0073] • The change in spatial orientation corresponds to a predetermined change in spatial orientation or does not deviate from such a change by more than a predetermined amount, e.g. a change in spatial orientation along at least one predetermined and in particular control-element-specific spatial direction, in particular by more than a predetermined amount.

[0074] • there is no change in spatial orientation or the change in spatial orientation is less than a predetermined threshold, in particular for a predetermined duration,

[0075] • Despite the change in the object's spatial position, at least one image position and / or spatial position-specific condition has still been met.

[0076] In particular, this method enables gesture-based generation of an input signal, for example, when the image positions differ from each other by less than a predetermined amount and the change in spatial orientation corresponds to a predetermined change in spatial orientation, such as a click or double-click movement. A click movement can, for example, comprise a first partial movement along a predetermined spatial direction and a further partial movement in the opposite direction. A double-click movement can, for example, comprise a first partial movement along a predetermined spatial direction, a second partial movement in the opposite direction, a third partial movement along the predetermined direction, and a fourth partial movement in the opposite direction.

[0077] If the predetermined change in spatial position occurs along a predetermined spatial direction, this spatial direction can be oriented in particular parallel to the optical axis of the visualization system or parallel to the optical axis of the environmental detection device, or may include a portion of such a spatial direction.

[0078] This also advantageously increases the reliability in the generation of input signals.

[0079] In a preferred embodiment, the augmented environment image is generated such that the control element is superimposed by the depicted object if they are at least partially located in the same image area and their image positions thus overlap. This advantageously results in improved visual detection of the depicted object in the augmented environment image, which can increase the reliability of input signal generation. However, in this case, it may be necessary to adjust the display of the control element so that it does not superimpose on the depicted object.

[0080] Alternatively, the object being depicted is overlaid by the control element if they are at least partially located in the same image area. This advantageously results in a simplified and potentially less computationally intensive generation of the augmented environment image, particularly since the display of the control element does not need to be adjusted, while still ensuring sufficient visual recognition of the object being depicted.

[0081] In another embodiment, the overlaying control element or object is displayed semi-transparently. Thus, the augmented environment image can be generated in such a way that the overlaying control element or object is displayed semi-transparently. In other words, the object / control element in the foreground can be displayed semi-transparently when overlaid, so that a user can advantageously perceive both the object and the control element visually. For example, an alpha blending method can be used for the semi-transparent display. In this case, each or at least the overlaying pixel(s) of the overlaying control element / object can be assigned an alpha value between 0 (exclusively) and 1 (exclusively), where the value 0 or 1 represents, for example, complete transparency and the value 1 or 0 represents complete opacity.If a semi-transparent display is not desired, an alpha value representing complete opacity can be assigned to each or at least the overlapping pixel(s) of the overlapping control element / object.

[0082] In another embodiment, a graphical representation of an element or object in a state in which at least one condition is met differs from a graphical representation in a state in which the at least one condition is not met. Alternatively or cumulatively, a graphical representation of an element or object in a state in which at least one condition is met for a shorter period than a predetermined duration differs from a graphical representation in a state in which the at least one condition is met for a longer period than the predetermined duration.

[0083] The condition is an image-position-specific condition, an image-position-change-specific condition, a spatial-position-specific condition, a spatial-position-change-specific condition, or a combination of at least two of these conditions. For example, the graphical representation of the control element and / or the object in a state where their image positions overlap can differ from a graphical representation in a state where their image positions do not overlap. Thus, the graphical representation of the control element can also change when the state changes. In particular, the graphical representation of a control element or object can change when switching to the state where its image positions overlap. For example, a property of the graphical representation, especially a shape or color, can change.Furthermore, the graphical representation can change if a state, especially an overlap state, is maintained for a longer period of time.

[0084] Such different representations or changes advantageously make it possible to give a user improved visual feedback regarding the fulfillment of a condition for generating an input signal or the successful generation of an input signal, resulting in simplified and more reliable operation.

[0085] For example, a color change can serve as activation confirmation / feedback. A control element can, for instance, change its color when the image layers overlap. Similarly, the control element's representation can change from red to yellow as soon as such an overlap occurs, and then to green if this overlap persists for a predetermined duration. An input signal can only be generated if this overlap persists for a predetermined duration. This change in representation provides the user with visual feedback on the successful generation of an input signal. If the green display is not reached, the user can visually determine that no input signal has (yet) been generated.

[0086] In another embodiment, the input signal serves for the stepless adjustment of an operating parameter of the medical visualization system. For example, the input signal can be used to:

[0087] Magnification factor or zoom factor,

[0088] Working distance or focus position, detection range, illumination intensity,

[0089] The lighting spectrum can be adjusted. A stepless adjustment allows the value of the corresponding operating parameter to be changed continuously, without fixed intermediate steps, and especially without intermediate steps spaced more than a predetermined distance. In contrast to a stepped adjustment, which only allows discrete, fixed values ​​(such as a switch with fixed positions), a stepless adjustment enables any selection within a specific range.

[0090] In particular, stepless adjustment can be achieved by generating input signal(s) depending on a change in the object's image position and / or spatial orientation, and specifically by ensuring that a change in the image position and / or spatial orientation leads to a change in the input signal. For example, the value of the corresponding operating parameter can be increased if a coordinate value of the object's image position is changed in exactly one predetermined image direction. The value of the corresponding operating parameter can then be decreased if the coordinate value of the object's image position is changed opposite to the predetermined image direction. Alternatively or cumulatively, the value of the corresponding operating parameter can be increased if a coordinate value of the object's spatial orientation is changed in exactly one predetermined spatial direction.The value of the corresponding operating parameter can be reduced if the coordinate value of the object's spatial orientation is changed contrary to the predetermined spatial direction.

[0091] This advantageously results in a simple and easily visually perceptible stepless adjustment of an operating parameter.

[0092] Of course, it is also conceivable that the input signal serves not for stepless adjustment, but for step-based adjustment of a parameter of the visualization system. In particular, the input signal can be an activation or deactivation signal, e.g., for a lighting device, a braking device, or another (de)activatable element of the visualization system. The augmented environmental image and / or the microscopic image can also be saved when an input signal is generated.

[0093] In a further embodiment, at least the control of the display device for outputting the augmented environment image (i.e., an output state) and / or at least the generation of the input signal (i.e., a controllable state) are activated when an activation signal is generated and / or deactivated when a deactivation signal is generated. The (de)activation signal can be generated by a higher-level system. Alternatively, the (de)activation signal can be generated via at least one input device for operating the visualization system. Such an input device can be configured to generate the (de)activation signal through haptic actuation, acoustic operation, optical operation, or in another manner. For example, the input device can be configured as a foot switch or a hand switch.The input device can also be designed to generate the (de)activation signal via voice or gesture control and may include, for example, a microphone or a gesture recognition device. Thus, it is conceivable that the (de)activation signal is generated by a predetermined head movement of a user or by actuating a sterile or non-sterile control element. It is also conceivable that an (de)activation signal is generated by evaluating the identity and / or spatial position and / or a change in the spatial position of an object within the environment.

[0094] This advantageously results in improved usability of the medical visualization system, as a user can easily activate and / or deactivate the output state and / or the controllable state.

[0095] Furthermore, a medical visualization system is proposed, comprehensive

[0096] - at least an image acquisition device for microscopic imaging,

[0097] - at least one environmental sensing device,

[0098] - at least one control and evaluation unit and

[0099] - at least one display device, wherein the image acquisition device for microscopic imaging and the environmental sensing device are each part of a microscope head of the medical visualization system. A sensing area of ​​the image acquisition device for microscopic imaging is arranged completely within a sensing area of ​​the environmental sensing device, at least within a predetermined working distance range, wherein the sensing area of ​​the environmental sensing device is larger than the sensing area of ​​the image acquisition device for microscopic imaging. In particular, the environmental sensing device and the microscopic image acquisition device, as well as the beam paths associated with these sensing devices, can be arranged accordingly. According to the invention, the medical visualization system is configured to carry out a method comprising the steps according to one of the embodiments described in this disclosure.

[0100] For this purpose, the visualization system can include a communication interface for receiving the images captured by the recording devices.

[0101] The generation of the augmented image can be performed using the control and evaluation unit. The control and evaluation unit can be configured as, or comprise, at least one computing unit. A computing unit, in turn, can comprise, or be configured as, at least one microcontroller and / or at least one integrated circuit, such as an FPGA. It is possible that the evaluation unit includes at least one graphics processing unit (GPU) for this purpose.

[0102] The previously explained control of the display device for outputting the augmented environment image, the determination of the image position and the generation of the input signal can also be carried out with the control and evaluation device.

[0103] The medical visualization system advantageously enables the execution of a method according to one of the embodiments described in this disclosure, with the advantages already explained.

[0104] In another embodiment, a beam path for the rays captured by the image acquisition device for microscopic imaging is formed separately from a beam path for the rays captured by the ambient light detection device. The beam paths can be located in or formed by the microscope head. The rays captured by the microscopic image acquisition device can pass through at least one optical element that serves to magnify the area under investigation. In this case, the rays captured by the ambient light detection device can pass through no optical element that serves to magnify the area under investigation. This advantageously results in reliable generation of both the ambient image and the microscopic image, since the beam paths are separated.

[0105] In an alternative embodiment, at least one section of a beam path for the rays captured by the image acquisition device for microscopic imaging forms at least one section of a beam path for the rays captured by the environmental sensing device. This advantageously results in a visualization system with reduced installation space requirements. If the visualization system comprises two beam paths for microscopic imaging, for example, if the visualization system includes a stereo system, then at least one section of one of the beam paths of the stereo system can form at least one section of the beam path for the rays captured by the environmental sensing device.

[0106] In this case, too, the rays detected by the environmental sensing device cannot pass through any optical element that serves to magnify the area under investigation. However, it is also possible that the rays detected by the environmental sensing device pass through at least one optical element that serves to magnify the area under investigation. In particular, but not necessarily exclusively, in this embodiment, the visualization system can include a beam splitter that divides the radiation into the rays to be detected by the image acquisition device for microscopic imaging and the rays to be detected by the environmental sensing device, i.e., into the beam paths. The visualization system can also include a negative lens group through which the rays to be detected by the environmental sensing device pass and which can be designed to widen the detection area.The visualization system can also include at least one device for modifying the optical properties of all optical elements in the beam path, particularly in the common section. This device can, for example, be designed as a positioning device for at least one optical element, by which the at least one optical element can be moved into or out of the beam path. Such a movable optical element can, for example, be designed as a negative lens group to widen the detection area.

[0107] Thus, for example, an ambient image and a microscopic image can be generated alternately. The augmented image can then be generated, for example, with a currently generated ambient image and a previously generated microscopic image, or with a currently generated microscopic image and a previously generated ambient image. The components mentioned can be arranged in the microscope head. In the case of a stereo system, it is possible that the microscopic image is generated by one of the two microscopic image acquisition devices of the stereo system and is therefore not a stereo image. In both embodiments, it is possible that both the rays acquired by the microscopic image acquisition device and those acquired by the ambient image acquisition device pass through an optical element that does not serve to magnify the examination area, for example, through an end glass of the objective lens.

[0108] The invention is explained in more detail using exemplary embodiments. The figures show:

[0109] Fig. 1 shows a schematic representation of a medical visualization system according to the invention.

[0110] Fig. 2 shows a schematic view of different detection areas,

[0111] Fig. 3 shows a schematic block diagram of a medical visualization system.

[0112] Fig. 4 shows a schematic flowchart of a method according to the invention and

[0113] Fig. 5 shows a schematic augmented environment image.

[0114] In the following, identical reference symbols denote elements with the same or similar technical characteristics.

[0115] Figure 1 shows a medical visualization system 1 according to the invention in use in an operating room environment. The visualization system 1 comprises a preferably purely digital operating microscope 2 with a microscope head 3, which is arranged on a stand 4 for holding the microscope head 3, in particular at a free end of the stand 4. The stand 4 allows movement of the operating microscope 2 to change its position and / or orientation. The stand 4 shown represents an exemplary kinematic structure for holding and moving the operating microscope 2. It is of course known to those skilled in the art that other kinematic structures can also be used. Drive devices of the stand 4 (not shown) can enable rotational movement of movable parts of the stand 3 about axes of rotation 5, 6, 7.Also shown is an evaluation and control unit 8 of the visualization system 1.

[0116] This can be used to control the drive units and to set operating and / or movement parameters of the operating microscope 2. For this purpose, the control unit 8 can be connected to the operating microscope 2 and / or to the drive units via signal and / or data transmission, with the corresponding data and / or signal connection 12 being represented by a dashed line.

[0117] The medical visualization system 1 further comprises a display device 9, which in the illustrated embodiment is designed as a screen. The visualization system 1 also includes an environmental detection device 10 and an image acquisition device 11 for microscopic imaging, which is hereinafter referred to as the microscopic image acquisition device 11 and by which an examination area can be depicted in magnified form. The detection devices 10 and 11 are part of the microscope head 3. Also shown is a common end glass 17, which is arranged in or on the microscope head 3 and through which the rays detected by the environmental detection device 10 and the microscopic image acquisition device 11 pass. However, the end glass 17 can be designed such that it does not constitute an optical element for magnification.

[0118] The control and evaluation unit 8 can receive and evaluate the images generated by the acquisition units 10 and 11, which can be represented, for example, in the form of image signals. The control and evaluation unit 8 can also control the display unit 9 to output an image.

[0119] Also shown are a user 12, e.g. a surgeon, who is looking at an image displayed on the display device 9, and a patient 13 who is lying on an operating table 14.

[0120] The user 12 holds and moves an instrument 15 within a detection range of the environmental detection device 10. Also shown is an optical axis 16 of the medical visualization system 1, which can correspond to an optical axis of the operating microscope 2 or of an objective lens or the entirety of elements for magnifying the examination area.

[0121] The environmental sensing device 10 is part of an optical position sensing device for determining the spatial orientation of the instrument 15. The position sensing device can include a target 18 with at least one marker element, preferably several marker elements, which is arranged on the instrument 15 at a relative orientation to the instrument 15 that is predetermined, for example, by calibration. By evaluating an image from the environmental sensing device 10, in particular exactly one image, the spatial orientation of the target 18 and, depending on the predetermined relative orientation, also the spatial orientation of the instrument 15, in particular of an instrument tip, can be determined. The control and evaluation device 8 can also be part of the position sensing device. For example,The position of the target 18 in a two-dimensional image coordinate system of the environmental detection device 10 is determined, and based on this image position, a spatial position in a three-dimensional reference coordinate system is then determined. Both a position and an orientation in the three-dimensional reference coordinate system can be determined in this way.

[0122] Figure 1 shows an exemplary reference coordinate system, where a vertical axis (z-axis) of this reference coordinate system can be oriented parallel to and opposite to the force of gravity. A longitudinal axis (x-axis) and a transverse axis (y-axis) of the reference coordinate system can define a plane that is oriented perpendicular to the vertical axis. Furthermore, the longitudinal and transverse axes can also be orthogonal to each other.

[0123] The spatial position in the reference coordinate system can be determined by converting the image coordinate system or a (three-dimensional) coordinate system of the position detection device into the reference coordinate system via a known transformation, for example determined by a registration.

[0124] Fig. 2 shows a schematic view of different detection areas EB10, EB11 of the detection devices 10, 11 at a predetermined working distance D, which can be, for example, 100 mm. It is shown that the detection area EB11 of the microscopic image acquisition device 11 is completely located within the detection area EB10 of the environmental detection device 10 at the predetermined working distance D, with the detection area EB10 of the environmental detection device 10 being larger than the detection area EB11 of the microscopic image acquisition device 11.

[0125] Also shown is a microscopic sub-area TB11 of the examination area, which is imaged by the microscopic image acquisition device 11 into a microscopic image A11 (see Fig. 3), as well as a surrounding sub-area TB10 of the examination area, which is imaged by the surrounding acquisition device 10 into a surrounding image A10. The microscopic sub-area TB11 is smaller than the surrounding sub-area TB10 and is located within the surrounding sub-area TB10. Further shown is an instrument 15 with a target 18, which includes marker elements 19. The marker elements 19 are arranged in the acquisition area EB10 of the surrounding acquisition device 10. Thus, the image position of the marker elements 19 can be determined by evaluating a surrounding image A10, whereby, depending on the image positions, the spatial position of the target 18 and thus of the instrument 15 can be determined.

[0126] Fig. 3 shows a schematic block diagram of a medical visualization system 1, which comprises a control and evaluation unit 8, a display unit 9, an environmental sensing unit 10, and an image acquisition unit 11 for microscopic imaging. An environmental image A10 can be generated by the environmental sensing unit 10, and a microscopic image A11 can be generated by the microscopic image acquisition unit 11. These can be transmitted to the control and evaluation unit 8 in the form of image signals. The control and evaluation unit 8 receives the images A10 and A11 and can then perform a procedure according to the embodiment shown in Fig. 4.

[0127] Fig. 4 shows a schematic flowchart of a method according to the invention for controlling a medical visualization system 1 (see, for example, Fig. 3). In a receive step S1, an environmental image A10, which was acquired by the environmental detection device 10, and a microscopic image A11, which was acquired by the microscopic image acquisition device 11, are received. In a generation step S2, an augmented environmental image AA10 is generated by superimposing the environmental image A10 with the microscopic image A11 in a microscopy image area B11 (see Fig. 5), wherein a partial area TB11 of the examination area, detectable by the microscopic image acquisition device 11, is mapped into the microscopy image area B11.Furthermore, the environment image A10 is superimposed with at least one control element BE, in particular with its graphical representation, in an image area that lies outside the microscopy image area B11. In a control step S3, a control signal SS9 is generated for the display device 9. This triggers the display to output the augmented environment image AA10. In a determination step S4, the image position BLO of a depicted object O, e.g., an instrument 15, is determined by evaluating at least one environment image A10, in particular in a two-dimensional image coordinate system. In a further generation step S5, an input signal ES for the medical visualization system 1 is generated depending on the image position BLO of the depicted object O and the image position BBE of the at least one control element BE. The sequence of steps S1, ... shown in Fig. 4Step S5 is not mandatory in relation to the determination step S4 and the subsequent generation step S5. These can also be carried out before or simultaneously with the generation step S2 and / or the control step S3.

[0128] In determination step S4, a change in the image position BLO of the depicted object O and / or a spatial position of the depicted object O and / or a change in the spatial position of the depicted object O can additionally be determined. These changes can be determined, in particular, by evaluating several sequentially generated environment images A10 or by comparing a current image position BLO with at least one image position BLO determined at an earlier time, or by comparing a current spatial position with at least one spatial position determined at an earlier time. The image or spatial position determined at an earlier time can be stored and then retrieved from a corresponding storage device to determine the change. The input signal ES can then be additionally determined as a function of at least one of the parameters determined in this way.

[0129] Fig. 5 shows a schematic augmented environmental image AA10. The depicted augmented environmental image AA10 was generated by superimposing a microscopic image A11, acquired by a microscopic image acquisition device 11, onto an environmental image A10, which was generated by an environmental detection device 10 (see, for example, Fig. 3). Fig. 5 shows that a microscopic image area B11, in which this superimposition takes place, is rectangular and located centrally within the augmented environmental image AA10. A first image direction xA and a second image direction yA, which define an image coordinate system, are shown schematically.

[0130] Also shown are control elements BE1, ... , BE6 or their graphical representations, which are superimposed on a sub-area of ​​the environment image A10 that is located outside the microscopy image area B11. Also shown is an object O imaged by the environment detection device 10, which, purely by way of example, could be a hand or a finger. Of course, it is also conceivable that the instrument shown, for example, in Fig. 2 could constitute the imaged object O.

[0131] A first, second, and third control element BE1, BE2, BE3 represent a virtual push button. A fourth, fifth, and sixth control element BE4, BE5, BE6 represent a virtual slider. Input signals ES (see Fig. 4) can be assigned to the control elements BE1,..., BE6. For example, pressing the first control element BE1 can generate a first input signal ES, which, for example, activates or deactivates the execution of a function by the medical visualization system 1. For example, the first input signal can activate an activatable feature of the visualization system 1 that is in a deactivated state. Likewise, the first input signal can deactivate an activatable feature of the visualization system 1 that is in an activated state. Such a feature could, for example, be a lighting feature.Furthermore, when the second control element BE1 is activated, a second input signal ES can be generated, and when the third control element BE1 is activated, a third input signal ES can be generated, which can, for example, (de)activate the execution of other function(s) by the medical visualization system 1.

[0132] Such a control element BE1, BE2, BE3 can be activated if at least one of the aforementioned image-position-specific conditions is met. In particular, a control-element-specific input signal ES can be generated as soon as or when the image positions BLO, BBE of the depicted object O and the corresponding control element BE1, BE2, BE3 overlap, especially for longer than a predetermined duration.

[0133] By activating the fourth control element BE4, the fifth control element BE5, and the sixth control element BE6, an input signal can be generated to set the value of an operating parameter of the visualization system 1, whereby the value can, in particular, be a continuously adjustable value. For example, by activating the fourth control element BE4, a focus value can be set; by activating the fifth control element BE5, a zoom value; and by activating the sixth control element BE6, an illumination intensity. Such a control element BE4, BE5, BE6 can also be activated if at least one of the aforementioned image position-specific conditions is met.In particular, a control-element-specific input signal ES can be generated as soon as or when the image positions BLO, BBE of the depicted object O and the corresponding control element BE1, BE2, BE3 overlap, whereby the value of the adjustable operating parameter or a change in value is determined depending on the image position BLO of the depicted object relative to the control element BE4, BE5, BE6. The value can then be encoded by the input signal ES.

[0134] In Fig. 5, the hatched area of ​​a control element BE4, BE5, BE6 represents a currently set value of the corresponding operating parameter. For example, if the depicted object O is moved between the boundaries of the hatched area into the

[0135] If the object O is moved into the image area of ​​control BE4, BE5, BE6, the value to be set can be decreased, especially as long as the depicted object O is still between the boundaries or as long as the depicted object O is positioned within the image area of ​​control BE4, BE5, BE6. If the depicted object O is moved into the image area of ​​control BE4, BE5, BE6 between the boundaries of the unhatched area, the value to be set can be increased, especially as long as the depicted object O is still between the boundaries or as long as the depicted object O is positioned within the image area of ​​control BE4, BE5, BE6.

[0136] It is also possible for such a control element BE4, BE5, BE6 to be activated if at least one of the aforementioned image position-specific conditions and one image position change-specific condition are met. For example, an input signal can be generated if the depicted object O is located within the image area of ​​a control element BE4, BE5, BE6 and is moved along the second image direction yA. Such an input signal ES can, for example, lead to an increase in the value, with the increase depending on the distance traveled along the second image direction yA. Conversely, a movement against the second image direction yA can lead to a decrease in the value.

[0137] Figure 5 shows that the third control element BE3 is overlaid by the depicted object O. It is possible that the depicted object O is shown as partially transparent, in particular or exclusively the overlaid portion of the object O.

[0138] A currently set value or state can be visualized by a graphical representation of the control element BE1, ..., BE6. Furthermore, a graphical representation of a control element BE1, BE6, or the depicted object O, in a state where at least one condition for generating an input signal ES is met, can differ from a graphical representation in a state where at least one condition is not met. Similarly, a graphical representation of a control element or the depicted object O in a state where at least one condition is met for a shorter than a predetermined duration can differ from a graphical representation in a state where at least one condition is met for a longer than the predetermined duration. Reference symbol list

[0139] 1 Visualization system

[0140] 2 Operating microscopes

[0141] 3 Microscope head

[0142] 4 Tripod

[0143] 5, 6, 7 axis of rotation

[0144] 8 Evaluation and control unit

[0145] 9 Display setup

[0146] 10 Environmental sensing device

[0147] 11 Image capture device

[0148] 12 users

[0149] 13 patients

[0150] 14 Operating table

[0151] 15 Instrument

[0152] 16 optical axis

[0153] 17 End glass

[0154] 18 Target

[0155] A10 Environment Image

[0156] A11 microscopic image

[0157] AA10 augmented environment image

[0158] BE1.....BE6 Controls

[0159] B11 Microscopy image area

[0160] BBE Image Days

[0161] BLO image position

[0162] D Working distance range

[0163] EB10, EB11 detection range

[0164] ES input signal

[0165] O object

[0166] S1 receive step

[0167] S2 generation step

[0168] S3 control step

[0169] 54 Determination step

[0170] 55 Production step

[0171] SS9 control signal

[0172] TB10 microscopic subregion

[0173] TB11 Environment Sub-area

Claims

Patent claims 1. Method for controlling a medical visualization system (1), wherein the medical visualization system (1) comprises at least one image acquisition device (11) for microscopic imaging and at least one environment detection device (10), wherein the image acquisition device (11) for microscopic imaging and the environment detection device (10) are each part of a microscope head (3), wherein a detection area (EB11) of the image acquisition device (11) for microscopic imaging is arranged completely within a detection area (EB10) of the environment detection device (10) at least within a predetermined working distance range, and the detection area (EB10) of the environment detection device (10) is larger than the detection area (EB11) of the image acquisition device (11) for microscopic imaging, comprising the step: Receiving an environment image (A10) acquired by the environment detection device (10) and a microscopic image (A11) of an examination area acquired by the image acquisition device (11) for microscopic imaging, characterized in that the method comprises the following further steps: generating an augmented environment image (AA10) by superimposing the environment image (A10) with the microscopic image (A11) in a microscopy image area (B11), wherein a partial area (TB11) of the examination area detectable by the image acquisition device (11) for microscopic imaging is mapped into the microscopy image area (B11), and by superimposing the environment image (A10) with at least one control element (BE1, BE6) in an image area that lies outside the microscopy image area (B11), Controlling a display unit (9) of the medical visualization system (1) to output the augmented environment image (AA10), determining an image position (BLO) of a depicted object (O) by evaluating at least one environment image (A10), Generating an input signal (ES) depending on the image position (BLO) of the depicted object (O) and an image position of at least one control element (BE1 ,... , BE6).

2. Method according to claim 1, characterized in that a change in the image position (BLO) of the depicted object (O) is additionally determined, wherein the The input signal (ES) is additionally generated depending on the change in the image position (BLO) of the depicted object (O).

3. Method according to one of the preceding claims, characterized in that the image position (BLO) of the imaged object (O) is determined at least several times during a predetermined period of time, wherein the input signal (ES) is only generated if an image position-specific condition is met for the entire period of time.

4. Method according to one of the preceding claims, characterized in that additional information about the spatial position of the depicted object (O) is determined by evaluating at least one environment image (A10), wherein the input signal (ES) is additionally generated depending on the information about the spatial position.

5. Method according to one of the preceding claims, characterized in that the augmented environment image (AA10) is generated such that the control element (BE1 , ... , BE6) is superimposed by the imaged object (O) or the imaged object (O) is superimposed by the control element (BE1 , ... , BE6) if these are at least partially arranged in the same image area.

6. Method according to claim 5, characterized in that the superimposed control element (BE1 ,... , BE6) or depicted object (O) is displayed semi-transparently.

7. Method according to one of the preceding claims, characterized in that a graphical representation of a control element (BE1 , ... , BE6) or of the depicted object (O) in a state in which at least one condition is satisfied is different from a graphical representation in a state in which the at least one condition is not satisfied, and / or that a graphical representation of a control element (BE1 ... is different from a graphical representation in a state in which the at least one condition is not satisfied, and / or that a graphical representation of a control element (BE1 , BE6) is different from a graphical representation in a state in which the at least one condition is not satisfied.of the depicted object (O) in a state in which at least one condition is fulfilled for a shorter than a predetermined time period, is different from a graphical representation in a state in which the at least one condition is fulfilled for a longer than the predetermined time period, wherein the condition is an image position-specific condition, an image position change-specific condition, a space position-specific condition, a space position change-specific condition or a combination of at least two of these conditions.

8. Method according to one of the preceding claims, characterized in that a representation of the control element (BE1 ,... , BE6) or of the depicted object (O) according to a first graphic representation takes place when the superimposed state occurs, wherein a representation of the control element (BE1 , ... , BE6) or of the depicted object (O) according to a further graphic representation takes place when the superimposed state is maintained for a predetermined period of time.

9. Method according to one of the preceding claims, characterized in that the input signal (ES) serves for stepless adjustment of an operating parameter of the medical visualization system (1).

10. Method according to one of the preceding claims, characterized in that at least the control of the display device (9) for outputting the augmented environment image (AA10) and / or that at least the generation of the input signal (ES) is activated when an activation signal is generated and / or deactivated when a deactivation signal is generated.

11. Medical visualization system (1), comprising - at least one image acquisition device (11) for microscopic imaging, - at least one environmental sensing device (10), - at least one control and evaluation unit (8) and - at least one display device (9), wherein the image acquisition device (11) for microscopic imaging and the environment detection device (10) are each part of a microscope head (3), wherein a detection area (EB11) of the image acquisition device (11) for microscopic imaging is arranged completely within a detection area (EB10) of the environment detection device (10) at least in a predetermined working distance range, and the detection area (EB10) of the environment detection device (10) is larger than the detection area (EB11) of the image acquisition device (11) for microscopic imaging, characterized in that the medical visualization system (1) is configured to perform a method comprising the steps according to one of claims 1 to 10.

12. Medical visualization system according to claim 11, characterized in that a beam path for the rays captured by the image acquisition device (11) for microscopic imaging is formed separately from a beam path for the rays captured by the environment detection device (10).

13. Medical visualization system according to claim 11, characterized in that at least one section of a beam path for the rays captured by the image acquisition device (11) for microscopic imaging forms at least one section of a beam path for the rays captured by the environment detection device (10).

Images