Tracking moving objects during auto-configuration of surgical microscopy system

Abstract

To provide tracking of moving objects during an auto-configuration of a surgical microscopy system.SOLUTION: Techniques which relate to an automatic configuration of one or more components of a surgical microscopy system for imaging a moving object (242) are disclosed. In the described techniques, the initially determined target configuration of the surgical microscopy system is selectively adapted if a movement of the moving object is detected. This selective adaptation takes place when the movement is within a predefined spatial limit (551) and / or a predefined temporal limit.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] Various embodiments of the present disclosure relate to techniques for automatically configuring a surgical microscope system for imaging an object. Various embodiments of the present disclosure relate to techniques for taking into account the movement of an object when operating at least one component of a surgical microscope system.

Background Art

[0002] A medical surgical microscope system having a robotic stand for positioning a microscope (also referred to as a robotic visualization system or a surgical visualization system) is known from the prior art (see, for example, (Patent Document 1)).

[0003] The robotic stand can be manually controlled. However, techniques are also known in which the robotic stand is automatically controlled, for example, to enable automatic centering and / or autofocusing on a particular object, such as a surgical instrument (also referred to as a surgical instrument or a surgical tool). The positioning process is initiated by a user command, during which the robotic stand of the microscope and / or the objective optical unit is actuated.

[0004] Techniques for such automatic control are known, for example, from (Patent Document 2). Therein, automatic position correction is described. Deviations in the relative position of the object can be corrected. If automatic position correction is performed each time the position of the observed object changes, the user viewing the observation image may feel discomfort. Therefore, other operating modes of position correction are also disclosed. The conditional automatic mode is a mode in which the position deviation is corrected for the condition that the deviation of the relative position exceeds the allowable range. The specified timing correction mode is a mode in which the position deviation is corrected for the condition that a position correction command is input by the user.

[0005] In the case of such previously known technologies, it has been observed that there may be a discrepancy between the user's expectations regarding automatic configuration and the actual changes produced by the surgical microscope system.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0007] Therefore, an improved technique for automatically operating a surgical microscope system is needed. In particular, an improved technique related to the automatic configuration of a surgical microscope system for imaging an object is needed.

Means for Solving the Problems

[0008] This object is achieved by the features of the independent claims. The features of the dependent claims define embodiments.

[0009] Aspects related to the automatic configuration of a surgical microscope system having a stand and a microscope carried by a robotic stand are described below.

[0010] In principle, the stand can be robotic or at least partially robotic. It is also conceivable that the stand is not robotic.

[0011] The surgical microscope system is configured to image a predetermined object. This means that the predetermined object is arranged in a specific way, for example, within the field of view of the microscope camera of the microscope.

[0012] Automatic configuration may include, for example, automatic centering with respect to a specific reference point. The reference point can be, for example, the tip or the center of the object. The reference point can include, for example, the geometric centroid, the center point, or the center of activity of a plurality of objects. During such automatic centering of the field of view, the reference point is then arranged at the center of the microscope image. For this purpose, at least a partially robotic stand is actuated as appropriate.

[0013] Alternatively or additionally, the configuration may include setting the focus of the microscope to such a reference point (automatic focus). For example, the focus lens can be moved so that such a reference point is arranged on the focal plane.

[0014] Alternatively or additionally, it is also possible to set a specific orientation of the microscope of the surgical microscope system (automatic orientation) in relation to such a reference point. For example, a specific orientation of the microscope can be set (i.e., the optical axis can be rotationally positioned). For example, it is conceivable to rotate the microscope without translating it.

[0015] Also, a combination of translational and rotational movements (a combination of automatic positioning and automatic orientation) can be implemented.

[0016] Alternatively or additionally, the zoom of the microscope can also be set, for example, so that a specific object or a plurality of objects appear to fill the entire screen (automatic zoom).

[0017] With the above-described technology, it becomes possible to perform such an automatic setting of the surgical microscope system that particularly well meets the expectations of the user. In particular, this method also makes it possible to provide a seamless user experience that enables reliable system behavior for a surgeon in a high-stress situation.

[0018] For this purpose, the moving object that forms the basis of the automatic setting of the surgical microscope system is tracked during the automatic setting of the surgical microscope system. This means checking whether the position and / or orientation of the object has changed compared to the initial position and / or orientation, which means checking whether the object is moving.

[0019] Then, it is checked whether the movement meets one or more criteria. For example, it is possible to check whether the position and / or orientation is changing within a predetermined spatial limit. Alternatively or additionally, it is also possible to check whether the position and / or orientation is changing within a predetermined time limit.

[0020] For example, if the position changes outside the time limit, it is not considered in the context of the automatic configuration. For example, if the orientation changes outside the time limit, it is not considered in the context of the automatic configuration. For example, if the position changes outside the spatial limit, it is not considered in the context of the automatic configuration.

[0021] A computer-implemented method for controlling a surgical microscope system is disclosed. The surgical microscope system comprises a stand, for example a robotic or partially robotic stand. The surgical microscope system also comprises a microscope carried by the stand. The microscope may comprise, for example, a camera for capturing microscope images. The microscope may be, for example, a stereo microscope.

[0022] The method includes receiving a user command. The user command requests a configuration of a surgical microscope system for imaging a predetermined object. The method also includes first determining a target configuration of the surgical microscope system based on the user command. The method also includes actuating at least one component of the surgical microscope system to achieve the target configuration. The method also includes determining whether the position and / or orientation of a movable object is changing within a predetermined time limit and / or within a predetermined space limit. This determination is carried out in coordination with the actuation of at least one component. The method also includes adapting the target configuration if the position and / or orientation of the movable object is changing within a predetermined time limit and / or within a predetermined space limit.

[0023] A data processing device for controlling a surgical microscope system is disclosed. The data processing device comprises a processor. The processor is configured to load and execute program code from a memory. By executing the program code, the processor is caused to implement the method disclosed above for controlling a surgical microscope system.

[0024] Also disclosed is a surgical microscope system having such a data processing device.

[0025] The above features and the features described below can be used without departing from the scope of the invention not only in the explicitly set corresponding combinations, but also in further combinations or alone.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0027] The characteristics, features, advantages of the present invention described above, and the ways in which they are realized will become clearer and more clearly understood in connection with the following description of exemplary embodiments, which will be described in more detail in connection with the following drawings.

[0028] Hereinafter, the present invention will be described in more detail based on preferred embodiments with reference to the drawings. In the figures, the same reference numerals denote the same or similar elements. The figures are schematic diagrams of various embodiments of the present invention. The elements shown in the figures are not necessarily shown to scale. Rather, the various elements shown in the figures are drawn so that their functions and general purposes are understandable to those skilled in the art. The connections and couplings between the functional units and elements shown in the figures can also be implemented as indirect connections or couplings. The connection or coupling can be implemented in a wired or wireless manner. The functional units can be implemented as hardware, software, or a combination of hardware and software.

[0029] The technologies related to the operation of the surgical microscope system will be described below. With the technologies to be described, it becomes possible to automatically configure one or more components of the surgical microscope system. In this way, the surgical microscope system can be placed in a state where a specific object, for example, a surgical instrument, is appropriately imaged. For example, it is possible to perform automatic alignment, and / or automatic centering, and / or automatic orientation, and / or automatic focusing, and / or automatic zoom in relation to a reference point defined in relation to one or more objects. In particular, a technology regarding a method by which the surgical microscope system can be automatically configured to image a movable object is disclosed.

[0030] FIG. 1 schematically shows an aspect regarding an exemplary surgical microscope system 80. The surgical microscope system 80 is used to microscopically image an examination area during a surgical intervention. For this purpose, a patient 79 is placed on an operating table 70. A site 78 with a surgical instrument 78 is shown.

[0031] The surgical microscope system 80 includes a robotic stand 82 equipped with a positionable head portion 81. Depending on the variant, the robotic stand 82 can have different degrees of freedom. The positioning of the head portion 81, that is, a robotic stand 82 having six degrees of freedom for translation along the x-axis, y-axis, z-axis, and rotation about each of the x-axis, y-axis, z-axis is known. The robotic stand 82 can have a handle 82a as shown in FIG. 1.

[0032] In all variants, it is not necessary for the stand 82 to be robotic. It is also conceivable that the stand can be operated manually only.

[0033] The head portion 81 includes a microscope 84 having optical components 85, such as an illumination optical unit, an objective optical unit, a zoom optical unit, etc. (only schematically shown in FIG. 1).

[0034] In the illustrated example, the microscope 84 also comprises a microscope camera 86 (here a two-channel stereo camera, although a mono-optical unit is also conceivable), whereby an image of the examination area can be captured and reproduced, for example, on the screen 69. Thus, the microscope 84 is also referred to as a digital microscope. Also shown is the field of view 123 of the microscope camera 86.

[0035] In the example of FIG. 1, the microscope 84 also comprises an eyepiece 87 with an associated field of view 122. The eyepiece 87 is optional. For example, the detection beam path can be split using a beam splitter (shown in FIG. 1), so that both the capture of an image by the camera 86 and the observation through the eyepiece 87 are possible. It is not necessary for the microscope 84 to have an eyepiece in all variants. A pure digital microscope 84 without an eyepiece is also possible.

[0036] In the example of FIG. 1, the head portion 81 of the surgical microscope system 80 carried by the stand 82 also includes an environmental camera 83. The environmental camera 83 is optional. The environmental camera 83 in the example of FIG. 1 is shown as being integrated into the microscope 84, but it can also be arranged separately from the microscope 84. The environmental camera can be, for example, a CCD camera. The environmental camera can also have depth resolution. Instead of or in addition to the environmental camera, other auxiliary sensors, for example, distance sensors (such as time-of-flight cameras, or ultrasonic sensors, or sensors with structured illumination) can also be considered. The field of view 121 of the environmental camera 83 is shown in FIG. 1.

[0037] Thus, the surgeon has multiple options for visualizing the examination area using the eyepiece 87, using the microscope image recorded by the camera 86, or using the overall image recorded by the environmental camera 83. The surgeon can also directly visualize the examination area (without magnification).

[0038] Various components of the surgical microscope system 80, such as the robotic stand 82, the microscope 84, or one or more further components, such as the environmental camera 83, are controlled by the processor 61 of the data processing device 60.

[0039] The processor 61 can be in the form of, for example, a general-purpose central processing unit (CPU), and / or a field programmable logic module (FPGA), and / or an application specific integrated circuit (ASIC). The processor 61 can load program code from the memory 62 and execute it.

[0040] The processor 61 can communicate with various components of the surgical microscope system 80 via the communication interface 64. For example, the processor 61 can operate the stand 82 to move the head portion 81 relative to the operating table 70, for example, in a translational and / or rotational manner. The processor 71 can operate the optical components 85 of the microscope 84 to perform, for example, zooming and / or autofocusing. Images from the environmental camera can be read and evaluated if available.

[0041] The processor can evaluate a sequence of images captured by a camera of the surgical microscope system to track an object. For this purpose, for example, it is possible to use a machine learning algorithm that detects corresponding objects in various images or, optionally, locates them. Alternatively or additionally, it is conceivable to determine an optical flow based on the sequence of images, and regions where the optical flow exhibits large values are regions where a moving object is likely to be located. The corresponding techniques are known in principle in the prior art (see, for example, (Patent Document 3)), and the implementation form of the specific algorithm used to track an object is not decisive for the technique described herein.

[0042] The data processing device 60 also includes a user interface 63. Commands from a surgeon or a user of the surgical microscope system 80 in general can be received via the user interface 63. The user interface 63 can have various configurations. For example, the user interface 63 can include one or more of the following components, namely, a handle on the head portion 81, a foot switch, a voice input, an input via a graphical user interface, etc. The user interface 63 can provide a graphical interaction via menus and buttons on the monitor 69.

[0043] The following describes the technology in terms of how the surgical microscope system 80 is used to enable an automatic configuration for imaging a predetermined object (e.g., a surgical instrument). Such a configuration may include, for example, aligning the microscope with the surgical instrument. For example, it is possible to perform alignment in the lateral direction, i.e., in the X direction and / or the Y direction, using the surgical instrument. Alignment may include, for example, centering with respect to a reference point of the object. For example, centering with respect to the tip of the surgical instrument may be triggered. Also, as an alternative or addition to such lateral alignment with the object, it is also possible to set the focus with respect to the object. For example, the objective optical unit of the microscope 84 may be actuated to move the lens element. It is also possible to use a microscope with a fixed focal length, in which case the Z position of the microscope 84 can be changed by the robotic stand 82. Alternatively or additionally, it is also possible to tilt (auto-align) the central axis of the head portion 81, and thus the microscope 84, in particular the optical component 85, in order to image a specific object. It is also possible to rotate it. Such a technique may have the advantage that a specific posture of the microscope 84 with respect to the object, i.e., a specific alignment and orientation, becomes possible. In this way, for example, deep channels (see site 78 in FIG. 1) that occur during a surgical intervention can be imaged. Generally, such an automatic configuration for imaging a predetermined object may be referred to as an automatic XYZ configuration, which indicates that the XY range and Z configuration of the captured image are set. Automatic XYZ may mean, for example, automatic positioning (especially automatic centering), and / or automatic focus, and / or automatic orientation, and / or automatic zoom, etc.

[0044] For the purpose of the automatic XYZ configuration, the target configuration of one or more components of the surgical microscope system 80 is determined, and then one or more components are actuated to change from the actual configuration to the target configuration.

[0045] In various embodiments, the automatic XYZ configuration is triggered by a user command. A scenario has been observed where a user command is received and then the scene changes. This means that one or more objects move and then have a position and / or orientation different from that at the time when the user command was triggered. Thus, the user command is triggered or issued in a first state of the scene, but then the scene transitions to a second state. For example, since automatic centering is performed with respect to the old or new position of an object while the user expects the exact opposite behavior, this can result in an undesirable system behavior in the reference implementation form. In other words, this means that due to changes in the underlying scene, the behavior of the system and the user's expectations will be at odds with each other.

[0046] Regarding the automatic XYZ configuration triggered by a user command, a technique will be described below that enables better meeting the user's expectations than the reference implementation form. Deterministic system behavior is enabled to provide reproducible and understandable results even in very different situations and / or in a very different and dynamic scene. Such a technique is based on the knowledge that in a surgical environment, especially in a high-stress situation under time pressure as usually occurs, the behavior of the automatic controller system needs to exactly match the user's expectations.

[0047] FIG. 2 is a flowchart of one exemplary method. The method of FIG. 2 relates to a technique for implementing an auxiliary function. The auxiliary function comprises or is composed of an automatic XYZ configuration of a surgical microscope system.

[0048] In particular, FIG. 2 relates to a technique for implementing an auxiliary function in relation to a moving object. An example of a moving object is a surgical instrument particularly handled by a surgeon.

[0049] Examples of surgical instruments generally include scalpels, forceps, scissors, needle holders, clamps, suction devices, trocars, coagulators, electrosurgical knives, dissectors, drills, spreaders, osteotomes, suture materials, knot pushers, periosteal elevators, hemostatic forceps, lancets, drains, suture cutters, spatulas, ultrasonic aspirators.

[0050] User commands are received in box 3203. The user commands implicitly or explicitly request, for example, the configuration of a microscope camera for imaging a given object, such as a surgical instrument. The user commands may request specific auxiliary functions. The user commands are received by a user interface (see FIG. 1). The user commands may, for example, request automatic alignment or autofocus. The user commands may trigger an automatic XYZ configuration.

[0051] The target configuration of the surgical microscope system is first determined in box 3204. This target configuration is determined based on user commands from box 3203. The target configuration can be determined for a scene having one or more objects currently being imaged by a camera of the surgical microscope system. This target configuration relates to one or more components of the surgical microscope system. For example, the target configuration may indicate how the robotic stand is intended to position and / or orient the head portion. Also, for the microscope, one or more settings may be indicated, such as the magnification of the zoom lens optics to be used, or the settings of the autofocus and / or filters to be used.

[0052] Next, in order to achieve the target configuration, at least one component of the surgical microscope system is actuated in box 3205. Depending on the implementation form variant, the actuation in box 3205 may take different forms. For example, box 3205 may include transferring set values to the control hardware of the surgical microscope system. The control hardware can then convert these set values into a sequence of specific control commands, such as analog values. In this way, the actual configuration of the surgical microscope system is gradually changed to the target configuration. However, it is also conceivable that an appropriate sequence of set values is determined in box 3205 and transferred directly to one or more corresponding components.

[0053] Depending on the auxiliary function, different components can be controlled in box 3205. For example, the field of view of the microscope can be automatically centered on a predetermined object. Such centering can be performed using two dimensions, i.e., XY translation of the head part. In such a situation, a robotic stand can be actuated to perform such movement. Alternatively or additionally, it is also conceivable that automatic centering is performed in six dimensions, for example, to examine a site, i.e., using rotation, movement, and translation of the head part. Alternatively or additionally, automatic focusing on a specific predetermined object can be performed. In such a case, the movable optical unit of the microscope's objective lens can be actuated. However, in such an example, it is also possible to perform Z positioning of the head part (especially when the optical component has a fixed focal length). In yet another scenario, it is conceivable to set the field of view of a set of objects (e.g., objects recognized as relevant) by adapting the zoom magnification. For example, the zoom magnification is selected as large as possible, so that all selected objects are still visible (as a result, only the optical unit is moved, and the robotic stand becomes unnecessary). Therefore, some objects can be arranged close to the edge of the zoomed field of view.

[0054] By the operation in box 3205, one or more components of the surgical microscope system are adjusted (to achieve the target configuration). The adjustment of one or more components is not carried out immediately, but rather requires a specific period (i.e., a specific waiting time). The reasons are, for example, the limitations (acceleration, maximum speed) of the actuator, which is a servo motor. Further reasons are the mass inertia and the required positioning accuracy, which can cause the adjustment to be slow. For example, it may also be necessary to limit the moving speed to avoid the risk to the surrounding staff as a result of a collision in the case of exposed components (especially the robotic stand that carries and moves the head part in the operating room). The normal time for moving the robotic stand to a specific position is on the time scale of several seconds to dozens of seconds. The autofocus by adjusting the lens elements of the objective lens also generally requires a specific time, usually up to 1 to 2 seconds. This also applies to the zoom function.

[0055] Based on the knowledge that in various embodiments, due to such waiting times in the implementation of a specific configuration of the surgical microscope system, the scene based on the target configuration from box 3204 may also change during the set period (or even immediately after the setting is completed). Specifically, this means that the moving object can move away from the reference position (e.g., the center of the camera's field of view) used as a criterion for determining the target configuration. Therefore, the target configuration from box 3204 is out of date. Generally speaking, this is due to the fact that, on the one hand, the time scale on which the moving object moves and, on the other hand, the time scale on which one or more components of the surgical microscope system are adjusted to achieve the target configuration are equivalent.

[0056] Therefore, box 3210 checks whether the position of the movable object has changed. This can be carried out, for example, during or in conjunction with the operation of box 3205.

[0057] For example, a specific tolerance range is accepted during the checking of box 3210, and as a result, minor movements can be ignored (thus, frequent readjustments are avoided).

[0058] If no (significant) movement is identified in box 3210, the target configuration underlying the operation of box 3205 is not adapted (the "no" path from box 3210).

[0059] If the position of the movable object has changed (significantly), box 3215 is performed (the "yes" path from box 3210). Box 3215 determines whether the position and / or orientation of the movable object has changed within a predetermined time limit and / or a spatial limit. Thus, it is checked whether the adaptation of the target configuration is acceptable because the position and / or orientation of the movable object has changed only within the time limit and / or the spatial limit. Therefore, particularly large changes, and / or changes not in a time context particularly close to the operation in box 3205 can be avoided. On the other hand, some movements in a sufficiently close spatial and / or time context are considered during the operation. If the adaptation is acceptable, the target configuration is adapted in box 3220, and then the operation in box 3205 takes into account the adapted target configuration.

[0060] Next, the effects of such a method will be described based on specific embodiments. This is implemented separately from the reference implementation form. FIG. 3 shows the initial state. The surgical instrument 244 is disposed offset with respect to the center 821 of the microscopic image 820. Then, an automatic centering command (lateral XY centering) is received, and the command requires that the tip of the surgical instrument 244 be centered. This is indicated by the arrow in FIG. 3. While the head portion is being moved to perform this automatic centering, the surgical instrument 244 is also moved (see the arrow in FIG. 4). Therefore, this means that the scene on which the automatic centering is based also changes during the movement of the head portion. This results in the center 126 not coinciding with the tip of the surgical instrument 244 after the positioning process (without further means for adapting to the target configuration, i.e., when following the reference implementation form such as the "designated timing correction mode" described in (Patent Document 2)). As a result, under at least certain conditions, the user's expectations (e.g., "centering with respect to the tip of the tool") may not be met. In the examples of FIGS. 3 and 4, therefore, centering is performed only once, and after activating the auxiliary function (e.g., via a voice command or a button on the foot control panel), the head portion is centered with respect to the operation of the surgical instrument present during activation. Since the surgeon generally continues his work while the surgical microscope is still performing its movement, as a result, the field of view 123 of the surgical microscope 84 no longer corresponds to the position of the surgical instrument after the movement of the surgical microscope. For correct centering, centering is activated again only once, and the operation of the surgical instrument is paused. However, this is disadvantageous because it results in interrupting the surgical workflow and possibly the surgeon's concentration. Therefore, in another (disadvantageous) reference scenario (e.g., the "conditional automatic mode" described in (Patent Document 2)), it is also conceivable that the movement of the surgical instrument 244 is continuously followed.Therefore, continuous centering is performed for the operation of the surgical instrument, that is, after the auxiliary function is activated (e.g., via a voice command or a button on the foot control panel), the microscope is continuously centered for the operation until the auxiliary function becomes deactivated again. However, this reference implementation form is considered to often cause confusion because it is not possible to distinguish between a desirable movement and an undesirable movement of the object, and as a result, the surgical microscope will perform movements that disrupt the surgeon's concentration and focus (therefore, the risk to the patient will also increase).

[0061] Such problems of the above-described reference scenario are alleviated by considering at least the temporal and / or spatial limitations in box 3215. The movement of the object results in the adaptation of the target configuration only within the temporal limitation and / or within the spatial limitation. This is shown in FIG. 5.

[0062] FIG. 5 shows an aspect related to the spatial limitation 551. In the illustrated example, the spatial limitation is determined in relation to the target configuration initially determined based on the user command (see box 3204). Specifically, a predetermined object, here the tip of the surgical instrument, is intended to be placed at the center of the field of view 123 of the microscope camera, and thus at the center of the microscope image, using the initial target configuration. Thereby, the reference position of the predetermined object is defined, and in this embodiment, this reference position is the center of the microscope image. The spatial limitation is defined as the maximum distance from the reference position related to the initial target configuration.

[0063] In the example of FIG. 5, the tip 242 moves away from this reference position, specifically, by a distance less than the spatial limit 551 (shown by the arrow). Thus, the target configuration (initially determined based on the user command) is adapted such that the center of the field of view follows the surgical instrument 242 that moves and the moving tip 242 is finally positioned at the center of the microscopic image 820 (see FIG. 6). If the movement of the tip 242 is greater than the spatial limit, it is possible to output an error and / or terminate the automatic centering (the "no" path from box 3125 in FIG. 2).

[0064] The scenario where the spatial limit 551 is determined based on the reference position of the object related to the target configuration initially determined based on the user command has been described above. It is also conceivable that the spatial limit 551 is adapted again after the surgical instrument 242 is repositioned within the spatial limit 551. Thus, the spatial limit 551 can be "tightened". This means that the reference position can be adapted according to the correspondingly adapted target configuration. Such iterative adaptation of the spatial limit 551 can continue until the actual position and the target position of the surgical instrument 242 converge stably. This corresponds, for example, to the function of a low-pass filter, where smaller movements of the surgical instrument 242 result in "tightening" of the automatic centering, while larger movements of the surgical instrument 242 do not result in any adaptation of the automatic centering.

[0065] The spatial limits described in FIGS. 5 and 6 can be combined with time limits. However, the time limits can also be used without spatial limits. The aspects of the time limits will be described next.

[0066] FIG. 7 shows aspects related to the time limit 562. The time limit 562 indicates the period during which the movement of a given object that forms the basis of the target configuration of the surgical microscope system can be considered for adaptation to the target configuration. Thus, if the movement occurs within the time limit (for example, on the premise that the movement is also within the spatial limit), it is considered; otherwise, it is not.

[0067] In FIG. 7, (see box 3204 which requires automatic centering so that, for example, the target configuration can be determined first) a user command 561 is received, and then one or more components of the surgical microscope system are adjusted in period 563. This period may depend, for example, on the trajectory along which the robotic stand has moved to properly position the head portion.

[0068] The time limit 562 has a length that is 150% of the length of period 563 in the example of FIG. 7. That is, the time limit 563 is determined by the period for achieving the target configuration initially determined based on the user command 561.

[0069] The object is moved 568 within the time limit 562. For example, a surgical instrument is moved, and since this movement occurs within the time limit, it is considered for adapting to the target configuration. However, a subsequent movement 569 is outside the time limit 562 and is thus not considered for adapting to the target configuration.

[0070] In the example of FIG. 7, the time limit 562 is initially defined specifically by the duration of period 563 which is determined based on the trajectory initially determined based on the user command for achieving the target configuration. However, the time limit can also be adapted dynamically, as shown in FIG. 8.

[0071] FIG. 8 shows aspects related to the repeatedly applied time limits 562, 562-1, 562-2. The limits 562, 562-1, 562-2 are applied (i.e., "tightened") until the actual configuration stably corresponds to the target configuration, i.e., until the target configuration is finally achieved. In the example of FIG. 7, the movement 568 (described in connection with FIG. 7) triggers the adaptation of the target configuration (at this point the target position has not yet been finally reached), and thus the robotic stand is actuated again, for example, to recenter the center of the field of view, and the robotic stand is actuated during period 563-1. The newly made movement 568 in the example of FIG. 8 triggers the re-initialization of the time limit 562-1, and as a result, movement 569 is also considered here. Movement 569 triggers the movement (period 563-2) again, and then triggers the re-initialization of the time limit 562-2. The actual configuration then becomes stably the same as the target configuration, and no further re-initialization of the time limit exists. Thereafter, the object is finally stably positioned, for example, at the center of the microscopic image.

[0072] The scenarios of FIGS. 7 and 8 show an example where the time limit 562 in each case has a start time that coincides with the start time of the period 563 for setting one or more components, but in general, it is also conceivable that the time limits described herein have different start times. For example, the start of the time limit can be offset by a specific predetermined period compared to the reception of the user command 561 and / or compared to the start of the movement of one or more components of the surgical microscope system. In addition to such a time shift, it is also possible to start the time limit in an event-controlled manner. For example, it can be checked whether the microscope has moved a specific distance.

[0073] In connection with the dimensional setting of temporal and / or spatial restrictions, it is possible to firmly pre-define their sizes. However, it is also possible for their sizes to be determined dynamically, for example, based on the type of object, such as the setting of one or more components of a surgical microscope system, for example, the magnification of the microscope, the working distance, the surgical situation (e.g., the type of surgery), the patient's position, etc. Such techniques are based on the knowledge that a particular type of object usually has smaller movements than other types of objects. This means that the tolerance range related to the adaptation of the target configuration can be selected smaller for such an object type than for other object types.

[0074] Accordingly, various scenarios can be implemented using spatial and / or temporal restrictions. Some exemplary scenarios are described below. In a first scenario, a surgeon activates automatic centering once for a particular surgical instrument. Then, the position and / or orientation of the microscope is calculated based on the current position of the surgical instrument relative to the microscope, i.e., the target configuration of the robotic stand is determined first. In this context, an area or volume around the target position or around the tip of the surgical instrument is also defined within which the target pose can change during movement. This means that the spatial restriction is determined in relation to a reference position determined based on the initial target configuration. Next, the robotic movement towards the target configuration is initiated. The movement of the surgical instrument is continuously evaluated (tracked). As long as the tip of the surgical instrument is within the pre-calculated area or pre-calculated volume, the target pose is adapted. When the target pose is reached, the robotic movement ends and no further continuous adaptation of the target configuration is performed there. Accordingly, this corresponds to a spatial restriction combined with a temporal restriction defined by actually reaching the target position. In a further scenario, as already described above, a one-time centering is activated by the surgeon. Then, the target configuration (e.g., the target pose of the robotic stand) is determined. Also, the time it takes for the movement of the surgical instrument to adapt to the target configuration is determined (e.g., based on the estimated movement time to achieve the target configuration). Thereafter, the robotic movement is initiated and the movement of the surgical instrument is continuously evaluated (tracked). Based on the detected movement, the target configuration is adapted within the previously determined temporal restriction. Combinations of these two variants are also conceivable.

[0075] The above features and the features described below can be used without departing from the scope of protection of the invention not only in the explicitly set corresponding combinations, but also in further combinations or alone.

[0076] For example, various aspects related to an automatic XYZ configuration have been described above in connection with surgical instruments. However, also as a general rule, it is conceivable that the automatic XYZ configuration is also implemented with respect to other objects, such as the characteristic anatomical features of a patient.

[0077] Furthermore, aspects related to implementing an automatic XYZ configuration in relation to a single object have been described above. However, it is also generally conceivable that the automatic XYZ configuration is implemented in relation to a plurality of objects. For example, a reference point (e.g., a geometric center point or a center of activity) that depends on the positions of a plurality of objects can be determined, and then the automatic XYZ configuration can be implemented in relation to this reference point. For example, a subset of relevant objects can be selected from all visible objects, and then the automatic XYZ configuration can be implemented in relation to at least one object within this subset.

[0078] Furthermore, various aspects related to a robotic stand have been described above. It is not absolutely necessary for a surgical microscope system to have a robotic stand. The surgical microscope system can also have a partially robotic stand or a manual stand.

Explanation of Reference Numerals

[0079] 60 Data processing device 61 Processor 62 Memory 63 User interface 64 Communication interface 69 Monitor 70 Operating table 78 Site 79 Patient 80 Surgical microscope system 81 Head portion 82 Robotic stand 82a Handle 83 Environmental camera 84 Microscope 85 Optical component 86 Camera 87 Eyepiece 87 121, 122, and 123 fields of view

Claims

1. 1. A computer-implemented method for controlling a surgical microscope system (80) having a stand (82) and a microscope (84) carried by the stand (82), comprising: receiving (3203) a user command (561) requesting configuration of said surgical microscope system (80) to image a given object (231, 232, 233); - initially determining (3204) a target configuration of said surgical microscope system (80) based on said user commands; - actuating (3205) at least one component (82, 83, 84) of said surgical microscope system (80) to achieve said target configuration; determining, in coordination with said actuation (3205) of said at least one component (82, 83, 84), whether said position and / or orientation of said movable object (231, 232, 233) has changed within predefined time limits (562) and / or within predefined spatial limits (551); adapting (3220) the target configuration if the position and / or orientation of the movable object (231, 232, 233) changes within the predetermined time limits (562, 562-1, 562-2) and / or within the predetermined spatial limits (551); 4. A computer-implemented method comprising:

2. The method further comprising: determining a reference position of the predetermined object based on the target configuration initially determined based on the user command; Further comprising: the spatial constraint is determined relative to the reference position; 10. The computer-implemented method of claim 1.

3. The method further comprising: adapting the reference position based on the adapted target configuration if the position of the movable object (231, 232, 233) is changing within the predetermined time limits and / or within the predetermined spatial limits; The computer-implemented method of claim 2 , further comprising:

4. The method further comprising: - initially determining said time limits (562, 562-1, 562-2) based on a period (563) for achieving said initially determined target configuration based on said user commands (561); The computer-implemented method of any one of claims 1 to 3, further comprising:

5. The method further comprising: - applying said initially determined time limits (562, 562-1, 562-2) based on said time period (563) until said target configuration is finally achieved; The computer-implemented method of claim 4 , further comprising:

6. The method further comprising: - determining the type of the moveable object (231, 232, 233); Further comprising: the predetermined time and / or spatial restrictions depend on the type of the predetermined object (231, 232, 233); A computer implemented method according to any one of claims 1 to 5.

7. the user command requests automatic centering of the field of view (122, 123) of the microscope (84) relative to the given object (231, 232, 233); the at least one component comprises the at least partially robotic stand (82); A computer implemented method according to any preceding claim.

8. said user command requests automatic focusing of an image plane of said microscope on said given object (231, 232, 233); the at least one component comprises an objective optical unit (85) of the microscope (84) and / or the at least partially robotic stand (82); A computer implemented method according to any preceding claim.

9. A data processing device (60) for controlling a surgical microscope system (80), comprising a processor (61) configured to load and execute program code from a memory (62), said execution of said program code causing said processor (61) to: receiving (3203) a user command (561) requesting configuration of said surgical microscope system (80) to image a given object (231, 232, 233); - initially determining (3204) a target configuration of said surgical microscope system (80) based on said user commands; - actuating (3205) at least one component (82, 83, 84) of said surgical microscope system (80) to achieve said target configuration; determining, in coordination with said actuation (3205) of said at least one component (82, 83, 84), whether said position and / or orientation of said movable object (231, 232, 233) has changed within predefined time limits (562) and / or within predefined spatial limits (551); adapting (3220) the target configuration if the position and / or orientation of the movable object (231, 232, 233) changes within the predetermined time limits (562, 562-1, 562-2) and / or within the predetermined spatial limits (551); performing the steps consisting of: A data processing device (60).

10. A data processing device (60) according to claim 9, wherein said processor (61) is configured to carry out the method according to any one of claims 1 to 8.

11. A surgical microscope system (80) comprising a data processing device (60) according to claim 9 or 10.

Images