Surgery device and method for adjusting medical apparatus

Abstract

To provide a device that allows an alternative control operation of a graphical user interface by means of an instrument.SOLUTION: The device 10 includes an image processing device 25, an image display device 18, and an image capturing device 14. The image processing device is configured to show a menu 20 in the image (B) of the image display device. Showing the menu can be triggered by a suitable movement of an instrument 11. For example, an activation range (a) can be provided for this purpose at the image border that is monitored by the image processing device for determining whether an instrument tip 19 touches this region (a). If so, a respective activation signal (a) characterizing this gesture can be created and can be used for showing menu in the image (B). Additional gestures of the instrument tip, particularly in a defined detection region (E), allow the control of a pointer 21 in the menu and the selection of functions for control of an apparatus 15.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a device for treating the body of a human or animal patient, in particular a device in which the means for supplying a treatment instrument can be adjusted by manipulating the instrument, as well as a method for this purpose. [Background technology]

[0002] A system, method, and device for tracking an instrument through a digital microscope are known from WO 2022 / 216810. It creates an image and allows the operation of a graphical user interface depicted in the image by the tip of the instrument, which can be recognized by an image analysis device. The graphical user interface depicted in the image is preferably shown in an edge area of ​​the image outside the area of ​​operation of interest. If desired, the user interface can be blank. The graphical user interface can be displayed as soon as the user moves the tip of the instrument into the area of ​​the graphical user interface. Then, the respective field of the graphical user interface can be selected, provided that the instrument tip is present in this area for a minimum time.

[0003] To operate the graphical user interface, the user must move the instrument tip into the graphical user interface located at the image border and keep the instrument tip there during operation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2022 / 216810 Summary of the Invention [Problem to be solved by the invention]

[0005] It is an object of the present invention to provide a device that allows alternative control operations of a graphical user interface by an instrument. It is also an object of the present invention to provide a method for controlling the operation of a graphical user interface. [Means for solving the problem]

[0006] This object is solved by a device according to claim 1 and a method according to claim 15.

[0007] The device according to the present invention includes an image capture device configured to capture images of the surgical site and transmit the images to an image processing device. The image capture device can be an endoscopic camera or a separate camera. Alternatively, an endoscope can be provided having one or more working channels and a camera at its distal end for capturing images or a video stream of the surgical site.

[0008] Furthermore, the instrument may be part of a device, the instrument tip of which may be moved into the field of view of the image capture device. For example, such an instrument may be a separate instrument from the image capture device and may be movable independently of the image capture device. The instrument may also be a rigid or flexible instrument configured as a probe that may be moved into and positioned within the field of view of the image capture device. A positioning device may serve to position the instrument tip within the field of view and may move the instrument tip laterally relative to its longitudinal direction. For this purpose, a means for bending the instrument tip may be provided within or on the instrument. If the instrument is a laparoscopic instrument with a rigid shank, a handle provided at the proximal end may be a positioning means for positioning the distal end.

[0009] An image display device is connected to the image processing device and configured to display images captured by the image capture device. For example, the image display device can be a monitor, VR glasses, etc. The image display device can be located proximate to the surgical site or can be located further from the surgical site, for example, in the case of remote operation.

[0010] The image display device and image processing device are configured to optionally show a menu representing a graphical user interface within the displayed image. Preferably, this menu is positioned outside the center of the captured image so as not to cover the region of interest and surgical target area of ​​human or animal tissue that is normally located in the center of the captured image. The menu shown outside the center includes a pointer symbol. Preferably, the pointer symbol is always present. More preferably, the pointer symbol is movable only within the menu and cannot move beyond its edges.

[0011] The image processing device is configured to detect the instrument, particularly its instrument tip, and reproduce the movement of the instrument tip on a pointer symbol. This movably couples the pointer symbol and the instrument tip. The pointer symbol is thus represented by the distance to the instrument tip. This allows the instrument tip to remain in the center of the image, i.e., within the surgical site area, during menu, i.e., graphical user interface, control operations, while still allowing the instrument tip to be used for menu control. This is particularly advantageous when, from a surgical perspective, longer instrument tip movements should be avoided or when the indicated menu area is difficult to reach. In a preferred embodiment, the image display device and / or image processing device are configured to define a detection area within the image that is located outside the menu. Furthermore, the image display device and / or image processing device can be configured to use the instrument tip and its movement only to move and position the pointer when the instrument tip is located within the detection area.

[0012] Preferably, an edge zone is defined for the image visible on the image display device, and the image display device and / or image processing device monitors whether the instrument tip is positioned within the edge zone. The image display device and / or image processing device may be configured to show a menu that is not normally visible in the image presented on the image display device as soon as the instrument tip is positioned within the edge zone and / or remains there for an extended period of time.

[0013] After showing the menu, a pointer symbol visible in the menu can be moved by the instrument tip in order to select individual menu fields. For this purpose, the image processing device can be particularly configured to move the pointer symbol according to the movement of the instrument tip as soon as the movement of the instrument tip is detected by the image processing device. Thus, the image processing device can be configured to use the movement of the instrument tip to move the pointer symbol when the instrument tip is positioned inside a detection area provided for detection, i.e., for example, arranged centrally. The detection area can preferably be arranged in the center of the image inside which the instrument tip is located in any case when performing a surgical operation.

[0014] The individual menu fields can be assigned to different technical operations, such as in particular selecting or changing the settings of a generator or another device supplying the appliance. For example, menu fields can be assigned to different modes and / or different effect intensities. The menu can contain submenus, for example, to allow changing the effect intensity in a selected mode.

[0015] The movement of the instrument tip can be translated into movement of the pointer proportionally (i.e., keeping the distance to the pointer symbol constant) and, if necessary, using a magnification or demagnification factor that increases or decreases, respectively, the comparison of the path length traveled by the pointer and the path length traveled by the instrument tip. In this way, the path length traveled by the pointer during instrument tip movement can be longer or shorter than the path length of the instrument tip movement. In either case, however, it is preferred that the directions of the instrument tip and pointer paths coincide with each other in the image at any one time.

[0016] The device according to the present invention further comprises an acknowledgment device that serves to select or activate a menu field in which the pointer is located. The acknowledgment device can be a foot switch, a manual switch, or a gesture recognition device that can be part of the image processing device. For example, the gesture that serves to generate an acknowledgment signal can be a puncturing movement of the instrument tip. Alternatively, a circular movement of the instrument tip can be detected as an acknowledgment signal. Other gestures can also be provided for generating an acknowledgment signal. For example, selection of a menu field can be performed by vertical movement of the pointer, and confirmation (i.e., acknowledgment) of the menu field can be performed by horizontal movement of the pointer within the menu field.

[0017] In a preferred embodiment, an activation signal is generated to indicate a menu in that an activation area is touched with the instrument tip. Control of the menu by the instrument tip is preferably performed only when the instrument tip is located within this detection area. If the instrument tip performs a predetermined gesture within the detection area, such as a lancing movement, an acknowledgment signal can be generated to confirm the selected menu field.

[0018] Easy controlled operation of the generator by the surgeon is possible with the device according to the invention and the respective method according to the invention without the surgeon having to take his / her eyes off the surgical site and without the surgeon having to be permanently away from the surgical site.

[0019] Further details and advantages of embodiments according to the invention, as well as modifications thereof, can be derived from the following description and drawings. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a schematic diagram of a device according to the invention; [Figure 2] 1 is a schematic diagram of an image presented by a visual display device showing the tip of the instrument in a controlled operating state; [Figure 3] 1A-1C are schematic diagrams of images presented by a visual display device showing the tip of an instrument in different control operating states; [Figure 4] 1A-1C are schematic diagrams of images presented by a visual display device showing the tip of an instrument in different control operating states; [Figure 5] 1A-1C are schematic diagrams of images presented by a visual display device showing the tip of an instrument in different control operating states; [Figure 6] 1A-1C are schematic diagrams of images presented by a visual display device showing the tip of an instrument in different control operating states; [Figure 7] 1A-1C are schematic diagrams of images presented by a visual display device showing the tip of an instrument in different control operating states; DETAILED DESCRIPTION OF THE INVENTION

[0021] From Fig. 1, a device 10 for surgical treatment of a patient is known, the patient being represented in Fig. 1 only by a tissue surface 12 to be treated by an instrument 11. The tissue surface 12 is the surgical site and may be, for example, the inner surface of a hollow organ 13, such as the stomach, intestine, or other organ or tissue region. In the illustration according to Fig. 1, it is assumed that the surgical procedure is performed endoscopically, and an endoscope is not shown. For this reason, only the instrument 11, which may be configured, for example, as a flexible probe, and the image capture device 14, which may be, for example, an endoscopic camera, are shown.

[0022] The instrument 11 can be configured rigidly, for example, as a laparoscopic instrument, or flexible, for example, as an endoscopic probe. To perform a surgical procedure, the instrument is brought into interaction with, or at least in proximity to, a tissue surface 12 and moved according to the requirements of the surgical process. The movement directions are symbolized in FIG. 1 by arrows representing the x, y, and z movement directions. The instrument 11 configured as a probe can be guided through a working channel of an endoscope, which can be equipped with an image capture device 14. For example, the instrument 11 can be configured rigidly and can include a handle at its proximal end for positioning the rigid instrument 11 inside the patient. During a laparoscopic surgical procedure, the image capture device 14 can be inserted into the patient's body separately and independently of the instrument 11. All elements provided for moving the instrument 11 together (flexible or rigid) form a positioning device for the instrument 11.

[0023] For example, instrument 11 is an RF surgical instrument connected for this purpose to a supply device 15, e.g., in the form of an RF generator 16. However, other applications are possible. For example, device 15 may be a device additionally or alternatively configured for RF generation for gas supply to instrument 11, for purging the surgical site, for gas extraction, or for other purposes.

[0024] For adjustment of the instrument 16, the instrument may have numerous adjustment means 17, such as push buttons, rotary buttons, screens, etc. However, the control operation requires orientation of the instrument 12, either by the surgeon himself or by another operator in person guiding the instrument 11. However, the surgeon's attention is preferably directed to the display device 18, which shows the image B captured by the image capture device 14. In the latter, the distal end of the instrument 11, having the instrument tip 19, is visible within the field of view of the image capture device 14 and is therefore also movable within the image B represented by the display device 18.

[0025] Optionally, graphic elements such as the menu 20 and a pointer 21 present within the menu 20 may be depicted in the image B. The pointer 21 may be a graphical symbol, such as a dot, circle, triangle, or arrow, or a hand symbol or other symbol. The menu may include multiple menu fields 22, 23, and 24, over which the pointer 21 is movable. The menu fields 22, 23, and 24 may be assigned functions such as turning the device 15 on and off, influencing effect intensity, and / or displaying submenus for additional command input. The menu fields 22, 23, and 24 may be rectangular or otherwise shaped areas that may be provided with distinguishing colors, symbols, or labels to clarify the functions selectable via the menu fields. Preferably, the pointer 21 is depicted translucently so that the functions of the menu fields 22, 23, and 24 remain visible and recognizable when the pointer 21 is positioned over the respective menu fields.

[0026] Image processing device 25 is disposed between image capture device 14 and image display device 18, and is configured to locate image tip 19 within an image captured by image capture device 14 and control apparatus 15 based thereon. To this end, image processing device 25 is connected to control device 26, which is configured to generate control commands for apparatus 15 and transmit them to apparatus 15. Control device 26 may further be connected to acknowledgement device 27, which may be provided to generate an acknowledgement signal to cause control device 26 and image processing device 25 to display menu 20 within the image. Furthermore, acknowledgement device 27 may be configured to generate acknowledgement signal q to activate menu field 22, 23, or 24 in which pointer 21 is positioned. Upon receiving acknowledgement signal q, control device 26 provides a control signal to apparatus 15 corresponding to the associated menu field 22, 23, or 24.

[0027] The acknowledgment device 27 may be a manual acknowledgment device, for example, in the form of a push button that can be activated by a hand or a foot. Furthermore, the device may be a device for recognizing gestures of the instrument tip 19. Recognition of such gestures may be performed by the image processing device 25 and / or the control device 26. For example, the image processing device 25 may recognize each gesture g and transmit a corresponding signal to the acknowledgment device 27, which then generates a corresponding activation signal a or an acknowledgment signal q and transmits these signals to the control device 26. Alternatively, the image processing device 25 may be configured to recognize gestures and transmit the corresponding signal g to the acknowledgment device 27 or directly to the control device 26. For example, selection of a menu field by movement of the pointer 21 along the series of menu fields 22, 23, 24 may here be performed, for example, as a vertical movement. The gesture for confirmation (i.e., acknowledgment) of a menu field (e.g., menu field 23) and generation of the acknowledgment signal q may be, for example, a horizontal movement of the pointer 21 within the associated menu field 23.

[0028] The device 10 described so far operates as follows.

[0029] First, Figure 2 shows image B, captured by image capture device 14, in which instrument tip 19 is visible. The instrument tip is depicted approximately in the center of the image and therefore within the typical region of interest of tissue 12 that is to be affected by instrument 11.

[0030] Now, suppose that the operator wants to adjust the device 15 without taking his eyes off the image B. Therefore, the operator moves the instrument tip 19 into an activation area A, which may be, for example, the entire ring-shaped bounding area of ​​the image B. As soon as the instrument tip 19 touches this activation area A, the image processing device 25 or the control device 26 detects this as a gesture g and sends a respective signal to the acknowledgement device 27, as shown in Fig. 3. The acknowledgement device generates an activation signal a, which causes the image processing device 25, either in an indirect path via the control device 26 or also a direct path, to show the menu 20 in the image B, as seen in Fig. 4.

[0031] When displaying the menu 20 in the image B according to Fig. 4, a detection area E can be defined which covers a part of the image B, for example a central area, or alternatively covers the whole image B. This detection area E is monitored by the image processing device 25 for movements of the instrument tip 19 as soon as the instrument tip 19 is located inside the detection area E. The movements of the instrument tip 19 detected by the image processing device 25 are translated into respective movements of the pointer 21.

[0032] For example, this is shown based on the correlation between Figures 4 and 5. If the instrument tip 19 is moved from the position according to Figure 4 to the position according to Figure 5, which is shown by the dashed arrow in Figure 5, the pointer 21 is moved from its previous position, which is shown by the dashed line in Figure 5, and is moved from menu field 22 to menu field 23. This ensures that the direction of movement of the pointer 21 always coincides with the direction of movement of the instrument tip 19. In this way, the pointer 21 can be moved over the different menu fields 22, 23, 24 in order to select one of the menu fields 22 to 24. Preferably, the movement range of the pointer 21 is limited to the menu 20, i.e. the pointer 21 cannot leave the menu 20.

[0033] The selection of menu field 22, 23, or 24 is subsequently signaled by an acknowledgement signal q. This signal can be generated by an acknowledgement device 27 and transmitted to the control unit 26. For example, a push button on the instrument, a foot switch, or gesture recognition can serve to generate the acknowledgement signal q. According to FIG. 5, for example, a puncture gesture can be performed with instrument 11, which is indicated in FIG. 5 by arrow 28. The puncture gesture can be a short, quick movement of the instrument in its longitudinal direction. If this gesture is performed within detection area E, the gesture can be recognized as an acknowledgement signal q and provided to the control device 26. Depending on the meaning of the selected field, here menu field 21, the control device can then transition generator 16 (or another device 15) to the mode assigned to menu field 23, e.g., coagulation.

[0034] If desired, the image processing device 25 can show an additional menu field 29 in the image B according to Fig. 6, for example to allow adjustment of the intensity of the desired effect. This allows the pointer 21 to move up or down a slider 30, which is symbolically shown as a graphic, which can be triggered by a respective movement of the instrument tip 19 in the direction of the arrows 31, 32. After a predetermined time has elapsed, or alternatively by a respective gesture, for example the puncture gesture according to the arrow 28 in Fig. 5, the menu field 29 can be hidden again, and the image according to Fig. 7 can then be seen with the instrument 11 without any additional graphical depiction. The instrument 11 is now ready to treat the tissue surface 12 with the selected settings.

[0035] In the above description, it was assumed that the movements of the instrument tip 19, particularly the vertical movements, are transmitted to the pointer 21 at a 1:1 ratio. However, it is also possible to transmit an enlarged version of the path traveled by the instrument tip 19 on the pointer 21, for example, to enable control of the control menu 20 with minimal instrument movement. Alternatively, the path of the pointer 21 could be a proportionally reduced version of the path traveled by the instrument tip 19, for example, to enable particularly sensitive menu control operations. In particular, it is possible to define different transmission ratios from the instrument tip 19 to the pointer 21 for the main menu 20 and the menu field 29. Furthermore, it is possible to use another selected point of the instrument 11 for control of the menu 20 and the pointer 21 instead of the instrument tip 19. The description of the present invention and its embodiments therefore applies accordingly, provided that a selected position of the instrument 11 replaces the instrument tip 19.

[0036] Furthermore, it is possible to transmit only vertical movements of the instrument tip 19 to the pointer 21, while horizontal movements of the instrument tip 19 are not transmitted to the pointer 21. In this way, control operations of the menu 20 are simplified and operation errors, in particular moving the pointer 21 off the menu 20 or getting the pointer 21 stuck on the edge of the menu, are avoided. The image processing device 25 is not required to limit the movability of the pointer 21 on the menu 20, although it can be configured to do so. This also simplifies control operations and in particular allows the instrument tip 19 to control the pointer 21 at positions within the detection range E, at which the instrument 11 can be best moved.

[0037] The device 10 according to the invention comprises an image capture device 14 to which an image processing device 25 and an image display device 18 are connected via the image processing device. The image processing device 25 is configured to show a menu 20 in an image B of the image display device 18. Showing the menu 20 can be triggered by a suitable movement of the instrument 11. For example, for this purpose, an activation area A can be provided at the image border, which is monitored by the image processing device 25 for whether the instrument tip 19 is touching this area. If so, a respective activation signal A characterizing this gesture can be generated and used to show the menu 20 in the image. A further gesture of the instrument tip 19 within a specifically defined detection area E allows the control of a pointer 21 within the menu 20 and the selection of functions for controlling the device 15. [Explanation of symbols]

[0038] 10 Devices for surgical treatment of patients 11 Equipment 12 Tissue surface 13 hollow organs 14 Image Capture Devices X,y,z moving direction 15 Device for supplying the appliance 11 16 Generator 17 Operational Elements 18 Image display device B. Image on image display device 18 19 Instrument tip 20 Menu 21 Pointer 22~24 Menu Field 25 Image Processing Devices 26 Control Devices / Detection Devices 27 Acknowledgement Device A startup area E Detection area 28 Arrow 29 Menu field for adjusting effect intensity 30,31 Arrows

Claims

1. A device (10) for the surgical treatment of a human or animal patient, comprising: an image capture device (14) configured to capture an image (B) of the surgical site (12) and transmit said image to an image processing device (25); an instrument (11) configured to affect the patient at the surgical site (12) and movably guideable within the surgical site; a) configured to display the image (B) captured from the surgical site (12), and optionally: b) showing at least one menu (20) outside the center of said captured image (B); c) showing a pointer symbol (21) movably associated with said instrument (11); and an image display device connected to said image processing device (25) configured to:

2. 2. The device of claim 1, wherein the instrument (11) comprises a distal instrument tip (19) that can be positioned within the surgical site (12).

3. 3. The device according to claim 2, characterized in that a positioning device is assigned to the instrument (11), the positioning device being configured to move the instrument tip (19) in two mutually independent directions (x, y) within the image (B) captured by the image capture device (14).

4. 3. The device according to claim 2, characterized in that the pointer (21) is movably associated with the instrument tip (19).

5. 5. The device according to claim 4, characterized in that the pointer symbol (21) is positioned away from the instrument tip (19) in at least one direction (x, y) in the image (B).

6. 2. The device according to claim 1, characterized in that the menu (20) comprises a plurality of menu fields (22, 23, 24, 29) to which different technical operations are assigned, the technical operations being the selection or modification of settings of the device (15), the settings being among different modes and effect intensities.

7. 3. The device according to claim 2, characterized in that a detection device (26) connected to the image processing device (25) is part of the device (10), the detection device (26) is configured to detect the position of the instrument tip (19) in the image (B) and transmit the position to the image processing device (25), and the image processing device (25) is configured to move the pointer (21) in response to changes in the position of the instrument tip (19).

8. 8. The device according to claim 7, characterized in that the image processing device (25) is configured to link the movement of the pointer (21) to the movement of the instrument tip (19) using a magnification or reduction factor.

9. 2. The device according to claim 1, characterized in that the device (10) comprises an acknowledgement device (27), configured to generate an acknowledgement signal (q) and connected to the detection device (26) for transmitting the acknowledgement signal (q) to the detection device (26).

10. 10. The device according to claim 5 or 9, characterized in that the detection device (26) is configured to transmit adjustment parameters to the apparatus (15) upon receiving an acknowledgement signal (q), the adjustment parameters being stored for a field (22, 23, 24) in which the pointer (21) is positioned.

11. 10. The device according to claim 9, characterized in that the image processing device (25) is configured to show and hide the menu (20) in response to an activation command (a).

12. 12. The device according to claim 11, characterized in that said acknowledgement device (27) is adapted to generate said activation command (a).

13. 13. A device according to claim 12, characterized in that the acknowledgement device (27) comprises a hand or foot push button.

14. 13. The device according to claim 12, characterized in that the acknowledgement device (27) is configured to detect a predetermined movement of the instrument tip (19) and to generate an acknowledgement signal (q) and / or an activation signal (a) in response to the detected movement.

15. 1. A method for adjusting a device (15) for supplying a surgical instrument (11), said method comprising: An image (B) of the surgical site (12) is captured by an image capture device (14) and transmitted to an image processing device (25); an instrument (11) suitable for affecting the patient at the surgical site (12) is positioned with its instrument tip (19) at the surgical site (12); An image display device (18), a) depicting an image (B) of the surgical site (12) and the instrument tip (19); b) showing at least one menu (12) outside the center of said captured image (B); and c) showing a pointer (21) movably linked to said instrument (11) on said menu (20).

Images