CONTROL ARRANGEMENT FOR CONTROLLING THE MOVEMENT OF A ROBOT ARM

Description

[0001] The invention relates to a control arrangement, a control method for controlling a movement of a robot arm, and a treatment device with the control arrangement.

[0002] Combined surgical holding and camera systems are known from the prior art, which can be positioned and controlled in various ways for microscopic or endoscopic applications. For example, endoscopic holding arms can be moved and adjusted manually or via an automatic positioning system, such as a robotic arm.

[0003] DE 10 2012 220 116 A1 describes a portable medical device comprising a device head, a navigation guide, a camera, and a data processing unit for image data. A navigation device is provided by means of which the device head can be navigated to or within a patient using a predetermined map.

[0004] From DE 10 2007 054 450 A1, a device for providing images to a surgeon is known, wherein a robotic arm is provided with a microscope camera. A navigation system determines both the surgical field and the position and orientation of the camera.

[0005] With navigated instruments, the problem arises that these instruments must be tracked via optical tracking, which makes the system very complex. A constant line of sight between the instrument being tracked and the camera is required; furthermore, electromagnetic tracking is susceptible to interference, depending on which metallic connections are present in the operating area.

[0006] German patent DE 10 2014 188 962 A1 discloses an arrangement for determining the position of a surgical instrument, wherein the instrument can be inserted into a trocar. The instrument has a gyrometer and an accelerometer, and the trocar has a sensor arrangement for determining the penetration depth.

[0007] In some cases, it may be necessary to determine the instrument's position using the inertial sensor, but the camera's viewing angle might not be suitable. The surgeon then has to interrupt the procedure, put the instruments aside, and adjust the camera correctly—this costs time and disrupts the surgical workflow. Furthermore, the surgeon might be tempted to work with less-than-ideal viewing angles or blurry images for convenience.

[0008] Based on this state of the art, the object of the present invention is to propose a control arrangement for controlling the movement of a robot arm, which provides an exact positioning of the robot arm, prevents an interruption of the workflow and is easy to use.

[0009] Patent application DE 10 2016 107 853 A1 discloses a surgical assistance robot for guiding medical instruments, such as an endoscope camera. For this purpose, a medical instrument has a sensor unit that records acceleration and rotation data and is connected to a control unit. Based on this recorded data, the surgical assistance robot can be controlled by the instrument.

[0010] German patent application DE 10 2010 031 943 A1 teaches the guidance of a medical instrument using an orientation device in the form of an inclinometer. This can consist of a spirit level, an accelerometer, or a MEMS.

[0011] Patent application DE 10 2004 052 753 A1 describes an operating room assistance system with motorized kinematics. This system allows an auxiliary instrument to be held and moved during surgery. To guide the instrument to a leading instrument, its position coordinates are detected by electromagnetic and / or optical sensor elements, and the motorized kinematics are controlled accordingly.

[0012] Patent application US 2018 / 0071029 A1 teaches the use of acceleration, force, gyroscope, magnetic sensors and strain gauges in connection with a tracking system in the field of neurosurgery.

[0013] Application DE 10 2008 041 260 A1 discloses the restriction of movement to a defined work area.

[0014] Based on this state of the art, the object of the present invention is to propose a control arrangement for controlling the movement of a robot arm, which provides an exact positioning of the robot arm, prevents an interruption of the workflow and is easy to use.

[0015] This problem is solved by a control arrangement having the features of claim 1.

[0016] It is further an object of the invention to provide a treatment device that enables safe and effective treatment of a patient by allowing precise recording of the working area. This object is achieved by a treatment device with the features of claim 8.

[0017] The further task of proposing a control method for controlling a movement of a robot arm that enables exact positioning of the robot arm without interrupting the workflow of an operation is solved by the control arrangement of dependent claim 9.

[0018] Preferred embodiments of the control arrangement, the treatment device and the control method are described in the dependent claims.

[0019] A first embodiment of the control arrangement according to the invention is designed to control the movement of a robot arm within a predetermined working range of the robot arm. The control arrangement comprises at least: a surgical instrument with an inertial sensor, a robot arm with a camera unit arranged at a distal end of the robot arm, and a release unit that is operationally coupled to the inertial sensor and detects a movement of the surgical instrument based on the inertial sensor signals, wherein, depending on a detected movement of the surgical instrument, a control movement of the robot arm, which is coupled to a control unit, can be triggered or terminated.

[0020] Endoscopic cameras, exoscopes or surgical instruments can be equipped with special sensors - by reading these sensors at specific times or during a movement performed by the user, commands can be passed to the holding arm or a camera unit, thereby achieving exact positioning of the robot arm.

[0021] In a further embodiment of the control arrangement according to the invention, the surgical instrument, the robot arm, the camera unit, and the release unit can be operationally coupled to one another via the control unit. "Operationally coupled" refers to the coupling of a process or work step with a process or work step initiated by it, so that the control unit and the other electronic components communicate wirelessly / wired with each other and exchange data for evaluation, storage, and control.

[0022] "Arrangement" here refers to a connection of electronic components so that they work together, exchange data, and an action by one component leads to a reaction by another component within the arrangement. The components are arranged in space in such a way that they are spatially adjacent to each other and can thus form a working unit, i.e., an arrangement, in which each component has its specific task.

[0023] A "robot arm" within the meaning of the invention is an industrial robot or part thereof and is equipped as a manipulator with tools and actuators. Such robot arms are universally applicable motion machines with multiple axes whose movements, with regard to sequence of movements and paths or angles, are freely programmable and sensor-guided without mechanical or human intervention. In the arrangement according to the invention, the robot arm can carry a tool, a holding device, or the camera unit, or a combination thereof. The robot arm can also be understood as the actual actuator that moves an endoscope or an exoscope attached to the distal end of the robot.

[0024] A "control movement" includes any movement of the robot arm that involves repositioning the arm, the camera unit, changes and adjustments to the camera's degrees of freedom in the camera unit, as well as other movements to improve the view of the work area or image quality.

[0025] Preferably, the inertial sensor and the release unit are operationally coupled with the control unit of the robot arm in order to be able to process a release signal as well as the movement of the instrument, i.e. a signal from the inertial sensor.

[0026] Another embodiment of the control arrangement according to the invention can provide that the surgical instrument has a handle, wherein the inertial sensor is arranged in the handle. This couples the movement of a user's hand with the movement of the handle, thereby enabling more precise determination of the position and orientation of the surgical instrument in the working area.

[0027] In one embodiment of the control arrangement according to the invention, the inertial sensor can be a MEMS sensor comprising one or more sensors selected from the group consisting of accelerometers, gyroscopes, and magnetometers. Advantageously, 3D sensors and combined multi-sensor devices can be integrated directly into the handle of the surgical instrument. Particularly in the field of endoscopy, especially neuroendoscopy, it is advantageous to integrate inertial sensors into certain surgical instruments that are held in the hand for approximately 80% of the operating time. A suction device can be used as the surgical instrument. However, endoscopes, milling cutters, or any other instruments are also suitable.

[0028] In yet another embodiment of the control arrangement, the robot arm can have an actuator at its distal end that can be moved by the robot arm. The actuator can be another surgical instrument, e.g., an endoscope, a medical device such as a trocar, or simply a holding device.

[0029] Furthermore, the robot arm can be configured to control the position and orientation of the camera unit and one or more degrees of freedom of the camera unit, including focus, selection of one or more focus points, selection of an image section, panning (movement in the x / y / z directions), pivoting (movement in the x / y direction around a selected focus point), zoom, and changes in the working distance. Tilting, translation in all three spatial directions, and rotation of the camera unit are also possible using the robot arm. Each of the aforementioned movements and changes in degrees of freedom can be performed individually or in any combination. Thus, the robot arm can be used not only to support the camera unit but also to adjust its various settings and thereby "control" it. The same can be done for an actuator that can be located at the distal end of the robot arm.In addition, and independently of this, a distal tip of the robot arm can also be detected by the camera unit to provide an additional possibility for position determination and to control further degrees of freedom in an endoscopic or exoscopic image.

[0030] Furthermore, in another embodiment of the control arrangement according to the invention, the release unit is mechanically decoupled from the surgical instrument within the control arrangement. Preferably, the release unit can be a foot switch. Alternatively, operating buttons or other electronic triggering units are possible. An ergonomic arrangement is important so that the release unit is easily accessible and usable by the user.

[0031] In yet another embodiment of the control arrangement according to the invention, the working area for the movement of the robot arm is dimensioned such that the robot arm can be moved within predetermined dimensions. The working area is understood to be the spatially defined and fixed-dimensioned area within which the robot arm can move without injuring persons who may be in the vicinity of the working area or touching other objects. The working area in the control arrangement according to the invention preferably comprises the area in which the instrument is used, i.e., the space in which the robot or robot arm, and thus the camera unit connected to the robot arm, cannot collide with patients, surgical personnel, or other objects in the operating room environment. Movement of the robot arm outside the working area is not permitted.The work area can be statically predefined in preoperative planning, as well as dynamically (preferably in real time) detected and adjusted through intraoperative monitoring by suitable sensors.

[0032] In yet another embodiment of the control arrangement, a controller operationally coupled to the control unit is provided for transmitting the signals detected by the inertial sensor, in particular the orientation and degrees of freedom of the surgical instrument. This controller can be located within the control unit or directly connected to the surgical instrument. Based on the detected values, the control unit can then initiate control of the robot arm. This control can be implemented in two ways: In "absolute control," after activation, the control can be purely position-based; that is, for example, a change in angle initiated by the movement of the surgical instrument detected by the inertial sensor is translated into an end position of the robot arm or a specific internal degree of freedom of the camera unit.According to the invention, in a "speed control" system, the aforementioned angular change is converted into a velocity change, whereby the speed of the robot arm's movement or the rate of change of the camera unit's internal degrees of freedom is adjusted. In yet another possibility, a relative control system can be provided, whereby the tilt, rotation, or position of the sensor relative to a predetermined world coordinate system leads to a continuous movement of the robot arm (without an end position), which is faster the greater the sensor's displacement relative to the coordinate system. The movement stops either when the instrument has no relative distance (delta) to the world coordinate system or is interrupted by the release unit.

[0033] A treatment device according to the invention, comprising a treatment table for a patient, includes a robotic arm that is assigned to the treatment table for performing a treatment. Furthermore, the treatment device includes a control arrangement according to the invention for controlling the movement of the robotic arm. Advantageously, an operation can be performed using the control arrangement according to the invention without interrupting the workflow. The release unit allows a surgeon to easily move the robotic arm or adjust the camera unit without interrupting or otherwise delaying the operation. Clear images and good visibility are thus ensured.

[0034] To display the captured images, the control arrangement can include a monitor connected to the camera control unit and the robot arm's control unit. This monitor can continuously display the current camera image, either from the robot arm's work area or from an endoscope or exoscope connected to the robot arm.

[0035] The treatment device may be designed to define a working area for the robot arm's movement, dimensioned to allow the robot arm to move within predetermined dimensions that include the treatment table. These dimensions define a predefined working area and are adapted to the specific treatment device, whether the patient is lying horizontally, elevated, or seated. When moving towards the edge of the working area, the robot arm automatically stops in position at the edge. Any control inputs that would move the robot further outside the working area are automatically suppressed by the control unit. Only reverse movement is possible.

[0036] In a first embodiment, a control method according to the invention for controlling a movement of a robot arm in a control arrangement according to the invention comprises the following steps: a) Activating a movement mode of the robot arm by operating the release unit, while simultaneously a1) acquiring a value from the inertial sensor and determining a reference orientation and / or a reference position of the surgical instrument within the specified work area, and a2) acquiring an image of the work area with the camera unit and determining a reference orientation and / or a reference position of the surgical instrument in the work area, then b) moving the surgical instrument, while simultaneously b1) continuously acquiring values ​​from the inertial sensor and transmitting the acquired values ​​to the control unit, and b2) continuously acquiring images of the work area with the camera unit and determining a relative orientation and / or relative position of the surgical instrument in the work area in relation to the previously acquired reference orientation and / or reference position;and c) from the predetermined reference orientation and / or the predetermined reference position of the surgical instrument and the relative orientation and / or relative position of the surgical instrument, which were determined from the movement of the surgical instrument and the simultaneous acquisition of the work area by the camera unit, determining a movement path of the surgical instrument and function-dependently controlling the robot arm, thereby moving the robot arm within the work area and / or adjusting camera degrees of freedom of the camera unit, d) deactivating the movement mode of the robot arm by operating the release unit again.

[0037] A surgeon can advantageously use an intuitive control method during an operation. The method according to the invention enables repositioning, control of the camera's degrees of freedom or parameters, and direct positioning within the working area without the surgeon having to remove their instruments from the surgical field or change their usual working posture – thus enabling a seamless surgical workflow.

[0038] The following procedure can be implemented for function-dependent control of the robot arm: Upon activation, an initial orientation of the surgical instrument and the camera unit is determined, and this orientation is defined as the initial orientation. Subsequently, a direction of movement for the surgical instrument and a camera image direction for the camera unit are determined by detecting predetermined reference points within the robot arm's working area. From this, a relative orientation with respect to the initial orientation and a tilt angle of the surgical instrument relative to the initial orientation can be determined. The camera unit moves in the same direction as the tilt angle of the surgical instrument, continuing until deactivation or until the boundary of the working area is reached.In this process, the movement speed of the camera unit is chosen to be proportional to a value of the tilt angle or to a ratio of initial orientation and relative orientation.

[0039] In other words, each time the control is activated (e.g., by pressing a foot pedal or an activation button), the instrument's current orientation (inertial sensor) is determined as the neutral starting or reference orientation or position. The instrument's tilt or angle relative to this starting orientation is interpreted as input for the control, allowing the surgical instrument to be used much like a joystick. Moving the instrument to a clearly defined orientation (identifiable by one or more reference points as fixed features on the surgical instrument, such as a hole on a neurosuction device) is defined as a movement that, for example, triggers an "image up" control command. This automatically derives the movements for "image down" (the opposite direction) as well as "image left" and "image right" (each perpendicular to the first).The direction of movement of the surgical instrument is thus registered along with the image direction captured by the camera unit. The camera unit can be controlled in two ways: In a first embodiment, the robot arm can be moved indefinitely in the direction of the control input relative to its current position until the surgical instrument returns to its initial orientation or the control is deactivated, for example, by releasing the foot pedal. The speed of movement of the camera unit is proportional to the tilt angle. This allows for the control of any arbitrary movement without significantly moving the instrument. Even a slight tilt can be sufficient for this. A second possibility is to define the current orientation of the camera unit at the time the control is activated as the initial orientation.The respective tilt angle of the surgical instrument is transferred to the camera unit at a predetermined ratio, e.g., 1:5. If the surgical instrument is then tilted one degree in the "image right" direction relative to its initial orientation or position, the camera tilts five degrees in the "image right" direction relative to its initial orientation; that is, the camera unit tracks the movements of the surgical instrument in a scaled manner.

[0040] By mapping the executed movement patterns to a specific function, intuitive control within the control arrangement can be achieved without affecting the workflow during a medical procedure, thus enabling safe surgery. Further embodiments of the control arrangement and the treatment device, as well as some of the advantages associated with these and other embodiments, will become clearer and more easily understood through the following detailed description with reference to the accompanying figure. The figure is merely a schematic representation of one embodiment of the invention.

[0041] This shows Fig. 1 a schematic view of the treatment device according to the invention,

[0042] In Fig. 1 A treatment device 1 is shown, which includes a robot arm 2 and a treatment table 3. A patient 4 is shown schematically lying on the treatment table 3 as an example; however, the treatment device 1 can also function without a patient 4 for training or other purposes.

[0043] A surgical instrument 5 is provided for performing an operation, and this instrument has a handle 6 for holding and guiding the instrument 5. An inertial sensor 6a is arranged in the handle 6, which serves to detect the orientation or position of the handle 6 and thus of the surgical instrument 5.

[0044] The robot arm 2 has movable segments and a robot head 10 on which a camera unit 9 is mounted. The robot head 10, i.e., the distal end of the robot arm 2, is electronically connected via electrical lines 8 to a camera control unit 13, a monitor 14, and the control unit 12.

[0045] Mechanically independent of the handle 6, a release unit 7 is arranged, which is in Fig.1 The foot switch 7 is used to activate or deactivate a movement mode of the robot arm 2. During this movement mode, the movement of the handle 6 is continuously detected by reading the inertial sensor 6a and consolidated with a movement of the handle 6 detected by the camera unit 9. The camera unit 9 can then be repositioned or settings such as focus or angle can be adjusted.

[0046] The present invention provides a treatment device. The control arrangement according to the invention is designed to control the movement of a robot arm 2 within a predetermined working range of the robot arm 2. According to the invention, the control arrangement 1 comprises at least the features of claim 1. Reference symbol list

[0047] 1 Arrangement 2 Robot arm 3 Treatment table 4 Patient 5 Instrument 6 Handle 6a Inertial sensor 7 Foot switch / release unit 8 Cable 9 Camera unit 10 Robot head 11 Light source 12 Robot / actuator control unit 13 Camera control unit 14 Monitor

Claims

1. A control arrangement which is designed to control a movement of a robot arm (2) within a predefined working region of the robot arm (2), wherein the control arrangement (1) at least has: - a surgical instrument (5) with an inertial sensor (6a), - a robot arm (2) with a camera unit (9) arranged at a distal end of the robot arm (2), - a release unit (7) which is operatively coupled to the inertial sensor (6a) and records a movement of the surgical instrument (5) based on the inertial sensor signals, wherein a control movement of the robot arm (2), which is coupled to a control unit (12) for control, can be triggered or terminated as a function of a recorded movement of the surgical instrument (5), wherein a movement of the robot arm (2) outside the working region can be suppressed, wherein the control arrangement (1) is configured to move the camera unit (9) in an image direction which corresponds to a clearly defined alignment of the instrument (5) and which points in the same direction as an angle of inclination of the surgical instrument (5), wherein the movement is carried out up to deactivation or up to the limit of the working region, wherein a movement speed of the camera unit is selected in proportion to a value of the angle of inclination or to a ratio of initial orientation and relative orientation.

2. The control arrangement according to claim 1, characterised in that the surgical instrument (5) has a handle (6), wherein the inertial sensor (6a) is arranged in the handle (6).

3. The control arrangement according to claim 1 or 2, characterised in that the inertial sensor (6a) is a MEMS sensor having at least one sensor selected from the group of an acceleration sensor, gyroscope and magnetometer.

4. The control arrangement according to at least one of claims 1 to 3, characterised in that the robot arm (2) has, at its distal end, an actuator, which can be moved by means of the robot arm (2).

5. The control arrangement according to claim 4, characterised in that the robot arm (2) is designed to control a position and an orientation of the camera unit (9) and at least one degree of freedom of the camera unit (9) from the group comprising focus, selection of an image section, panning and zoom.

6. The control arrangement (1) according to at least one of claims 1 to 5, characterised in that the release unit (7) is mechanically decoupled from the surgical instrument (5) in the arrangement (1), wherein the release unit is preferably a foot switch (7).

7. The control arrangement (1) according to at least one of claims 1 to 6, characterised in that the working region for the movement of the robot arm (2) is dimensioned in such manner that the robot arm (2) can be moved within predetermined dimensions.

8. A treatment device with a treatment table (3) for a patient (4), wherein the treatment device has - a robot arm (2), which is assigned to the treatment table (3) for a patient (4) in order to carry out a treatment, - and a control arrangement for controlling a movement of a robot arm (2), characterised in that the control arrangement for controlling a movement of a robot arm (2) is a control arrangement (1) according to at least one of claims 1 to 7.

9. The control arrangement (1) according to at least one of claims 1 to 7, which is configured to carry out a control method, the control method comprising the steps a) Activating a movement mode of the robot arm (2) by operating the release unit (7), at the same time a1) Recording a value of the inertial sensor (6a) and determining a reference orientation and / or a reference position of the surgical instrument (5) within the predefined working region, and a2) Recording an image of the working region with the camera unit (9) and determining therefrom a reference orientation and / or a reference position of the surgical instrument (5) in the working region, then b) Moving the surgical instrument (5) while at the same time b1) Continuously recording values of the inertial sensor and relaying the recorded values to the control unit (12), and b2) Continuously recording images of the working region with the camera unit (9) and determining therefrom a relative orientation and / or relative position of the surgical instrument (5) in the working region in relation to the previously recorded reference orientation and / or reference position; and c) From the predetermined reference orientation and / or the predetermined reference position of the surgical instrument (5) and the relative orientation and / or relative position of the surgical instrument (5), which were determined from the movement of the surgical instrument (5) and the simultaneous recording of the working region by the camera unit (9), Determining a movement path of the surgical instrument (5) and activating the robot arm (2) depending on the function, thereby moving the robot arm (2) within the working region and / or adjusting camera degrees of freedom of the camera unit (9), d) Deactivating the movement mode of the robot arm (2) by operating the release unit (7) again.

10. The control arrangement (1) according to claim 9, the control method comprising the step for activating the robot arm (2) depending on the function - When activated, recording a first orientation of the surgical instrument (5) and the camera unit (9) and determining this orientation as the initial orientation; - Determining a movement direction of the surgical instrument (5) and a camera image direction of the camera unit (9) by recording predetermined orientation points in the working region of the robot arm (2); - Determining therefrom a relative orientation in relation to the initial orientation and determining an angle of inclination of the surgical instrument (5) relative to the initial orientation; - Moving the camera unit (9) in the same direction as the angle of inclination of the surgical instrument (5) points, wherein a movement is carried out up to the deactivation or up to the limit of the working region, wherein a movement speed of the camera unit (9) is selected to be proportional to a value of the angle of inclination or to a ratio of the initial orientation and the relative orientation.