Robotic placement of equipment

Abstract

The present invention relates to robotic instrument placement. A system is provided that automatically aligns instruments according to a plan, e.g., a surgical plan, using only instrument tracking feedback by extending a robotic arm into the surgical field. Tracking markers on the robot are not required.

Description

[Technical Field]

[0001] The present invention relates to instrument positioning. In particular, the present invention relates to a system for instrument positioning, a method for controlling a system for instrument positioning, a computer program element, and a computer readable medium. [Background technology]

[0002] Robotic device positioning is becoming more common in applications involving manufacturing and machine manipulation, such as aircraft and construction machinery, for example, for surgical instrument guidance, drilling, nailing, etc. Robotic solutions typically rely on external tracking systems to position the robot and establish a robot-to-object reference frame (e.g., patient reference frame) using optical markers fixed to the object of interest, e.g., the patient or object being drilled. An example of a robotic solution can be found in U.S. Patent Application Publication No. 2015 / 164607. However, reference frames are prone to accidental shifts and occlusions, reducing object tracking accuracy and increasing the risk of registration errors.

[0003] US Patent 2018 / 0000546 discloses a medical robotic system including a robot coupled to an actuator element with the robot configured for controlled movement and positioning.

[0004] U.S. Patent Application Publication No. 2018 / 0185100 discloses a system and method for surgical navigation that provides mixed reality visualization, which depicts virtual images along with real objects to provide improved visualization to the user.

[0005] WO2019 / 204699 discloses a method and system for controlling a robotic arm, comprising tracking the movement of a handheld device using a motion tracking system, and controlling the robotic arm to adjust at least one of a position and an orientation of an end effector of the robotic arm based on the tracked movement of the handheld device. Summary of the Invention [Problem to be solved by the invention]

[0006] There may be a need to improve robotic device placement. [Means for solving the problem]

[0007] The invention is defined by the claims.

[0008] The object of the present invention is solved by the subject matter of the independent claims, and further embodiments are incorporated in the dependent claims. It is to be noted that the below-described aspects of the present invention also apply to systems, methods, computer program elements and computer-readable media.

[0009] According to a first aspect of the present invention, a system for instrument placement is provided. The system includes a robotic system having a robotic arm with control of four or more degrees of freedom (DOF) and a system controller having an interface unit. The robotic arm has an instrument interface connectable to an instrument. The interface unit of the system controller is configured to provide sensor data having instrument pose information and a target position relative to an object. The instrument pose information includes the position and orientation of the instrument. The interface unit of the system controller is further configured to provide image data at the target position. A target trajectory is planned in the image data for placing the instrument at the target position. The system controller is configured to transfer the instrument pose information and the planned target trajectory to an object coordinate system. The system controller is further configured to calculate a position error between the tracked instrument pose information and the planned target trajectory. The system controller is further configured to transfer the position error to a robot coordinate system of the robotic system to control the robotic arm to align the instrument to the planned target trajectory. The position error includes at least one of a translation error and a rotation error.

[0010] In other words, a system with a robotic guidance arm with four or more degrees of freedom for instrument placement is provided that can be used in a hybrid operating room environment with integrated surgical navigation. The system uses instrument pose information, i.e., position and orientation, to provide feedback to the robotic system in maintaining the instrument's trajectory with respect to a target trajectory planned by a user, e.g., a surgeon. Using instrument tracking as direct feedback to achieve alignment with the planned trajectory eliminates the need for manual alignment. The proposed system may be based on 4-DOF or 5-DOF device position feedback without additional tracking bodies on the robot.

[0011] In some examples, the proposed system may be used as a surgical instrument guidance system.

[0012] In some examples, the proposed system may be applied in manufacturing for alignment of parts or machine operations such as aircraft, construction machinery, etc., where drilling, nailing, or concentric alignment is common.

[0013] The instrument has an elongated shape. The system controller is configured to control the robot arm during a first movement of the robot arm for each new targeting session to move the instrument in a predetermined translation to generate a coarse six-degree-of-freedom alignment between the subject coordinate system and the robot coordinate system. The predetermined translation has a predetermined rotational translation.

[0014] Optionally, the predetermined movement may comprise a predetermined translational movement.

[0015] In other words, when only 4-DOF or 5-DOF instrument tracking is available, an approximate alignment method is used to facilitate alignment of the instrument interface, e.g., either on the instrument shaft inserted in the needle guide or on the instrument interface shaft attached to the robot arm, since the object of interest, e.g., the patient, and the robot base are typically static relative to each other during the short procedure time. The position of the instrument interface is also known relative to the robot arm. A coarse six-degree-of-freedom alignment between the object coordinate system and the robot coordinate system is generated based on two poses of the instrument and two poses of the robot arm before and after movement. This is described below, particularly with reference to the exemplary embodiment shown in Figures 3A and 3B.

[0016] The first movement of the robotic arm automatically initiates the coarse alignment process. In general, the predetermined rotational movement for the coarse alignment can be arbitrary, as long as it includes large rotations, since larger angles lead to more accurate alignment, assuming the robotic system movement and instrument tracking feedback can be precisely synchronized. Adding arbitrary translations as well may also be possible. This feature may be used in real time to update or verify that the alignment has not changed due to patient movement relative to the robotic system. Thus, the system controller may perform the coarse alignment automatically, avoiding any input from the user or adding time to the procedure.

[0017] According to one embodiment of the present invention, the sensor data comprises real-time pose information of the instrument relative to the target position, and the system controller is configured to calculate a real-time position error between the tracked pose information of the instrument and the planned target trajectory and transfer the real-time position error to a robot coordinate system of the robotic system for iteratively controlling the robot arm to align the instrument to the planned target trajectory.

[0018] In other words, real-time instrument position feedback may be used to iteratively move an instrument attached to the instrument interface to align it with the target trajectory for precise alignment.

[0019] In one example, during alignment, the system controller may be configured to keep the height of the instrument interface constant relative to the patient, table, imaging axis, robot base, initial tool position, or other physical or virtual reference. This height constraint may provide safer and more intuitive robot behavior. Once aligned, the surgeon can command the robot to lower the robot arm along the guide axis and, if necessary, align to the target.

[0020] According to one embodiment of the present invention, the system controller is further configured to apply a coarse six-degree-of-freedom alignment to control the robot arm to align the instrument to the planned target trajectory when the sensor data does not have instrument pose information.

[0021] The resulting alignment process may be sufficient to position the instrument near the desired trajectory in an open-loop manner, for example, when outside the tracking volume. Thus, as a further option, the system controller may be further configured to control the robot arm to align the instrument to the planned target trajectory based on a coarse six-degree-of-freedom alignment when the sensor data does not have instrument pose information, for example, when outside the tracking volume. Once the instrument is visible within the tracking system, the system controller may use the instrument pose information to iteratively work towards a final high-precision alignment.

[0022] According to one embodiment of the present invention, the system further comprises a tracking system configured to acquire sensor data comprising pose information of the interventional tool and a target position of the object of interest, the tracking system comprising at least one of an optical tracking system, an electromagnetic tracking system, and an acoustic tracking system.

[0023] The optical tracking system may use one or more cameras positioned in an imaging chamber to track pose information of the instrument. The one or more cameras may be capable of detecting infrared, visible light, and / or near-infrared light. Camera tracking markers may be attached to the instrument.

[0024] Electromagnetic (EM) tracking systems are based on the principle of mutual induction, where a field generator generates a known EM field to locate a small EM sensor placed within the tracking volume. EM trackers are gaining popularity due to their lack of line-of-sight constraints, small sensor size, and ease of use. As a result of their submillimeter size, sensors can be easily placed inside the tip of an instrument.

[0025] Acoustic trackers employ high frequency (eg, 20 kHz or greater) ultrasound in the form of a time-of-flight transducer / sensor or phase reference system.

[0026] As known to those skilled in the art, a tracking system can simultaneously track markers on the instrument as well as markers on the object, and thus create a registration between the object coordinate system and the tracking coordinate system.

[0027] According to one embodiment of the present invention, the tracking system is located on or inside the detector of the image acquisition system.

[0028] In one example, a tracking system such as a camera can be attached to a detector of an image acquisition system, for example, the camera may be temporarily attached at a predetermined location on or inside the detector of the image acquisition system during image acquisition, and then detached from the detector after image acquisition.

[0029] In one example, the tracking system may be an integrated part of the detector.

[0030] Locating a tracking system on or within the detector of the imaging acquisition system may allow the tracking coordinate system to be inherently spatially aligned to the image coordinate system, which may, for example, alleviate the need to independently track and align the tracking coordinate system of an external tracking system to the image coordinate system.

[0031] According to an embodiment of the present invention, the system further comprises an image acquisition system configured to acquire image data at the target location of the object of interest, the image acquisition system comprising at least one of an X-ray imaging device, an ultrasound imaging device, and a magnetic resonance imaging device.

[0032] In other words, the instrument placement system may be implemented with different imaging modalities.

[0033] According to one embodiment of the present invention, the robotic arm has a mechanical remote center of motion RCM mechanism with at least one degree of freedom of rotational control. The instrument interface is mounted on the RCM mechanism.

[0034] As used herein, the term "RCM" refers to a remote fixed point, which has no physical rotational joint, about which a mechanism or part of it can rotate.

[0035] The RCM may be robotically translated using a translation module. The inclusion of the RCM may minimize overall joint motion and create an ergonomic robotic configuration and predictable motion for an improved user interface, especially when performing Cartesian rotations. It may allow the instrument to be translated to the entry point primarily by actuating the translation module and then rotated into alignment using one or more distal joints, i.e., the RCM. Advanced link geometries can be optimized to bring the instrument interface closer to the RCM, enabling placement of the instrument interface closer to the target and facilitating the use of short instruments.

[0036] According to one embodiment of the present invention, the system controller is configured to control the robotic arm to align the instrument to a planned target trajectory while the instrument translates within a safety plane to prevent collisions.

[0037] In one example, the safety plane may be a plane above and parallel to the table. For example, the safety plane may be defined by the XY plane of the robot and the current robot guide position. Alternatively, the safety plane position may be defined relative to the tip of the instrument and then transformed to the robot coordinate system, and defined relative to the closest point on the instrument to the target trajectory with some reasonable constraints so that the point is not too far from the robot guide, or relative to the entry point.

[0038] More generally, a safety plane may be any plane defined to prevent robot collisions, such as the robot colliding with an inspection object, itself, and associated equipment. In other words, the safety plane forces the robot to move in a predictable manner to prevent the robot from colliding.

[0039] According to one embodiment of the present invention, the sensor data comprises pose information of the robot arm.

[0040] In other words, hybrid feedback control may be employed that includes both robot end-effector tracking and instrument tracking feedback, allowing for coarse alignment with low precision robot tracking (e.g., without an instrument in the guide), and once the instrument is visible in the camera and near the target, the system controller may use the instrument pose information to effect final high precision alignment. According to one embodiment of the present invention, the device comprises an interventional device.

[0041] According to one embodiment of the present invention, the interventional instrument comprises at least one of an injection needle, an interventional catheter, and an interventional laser device.

[0042] According to a second aspect of the present invention there is provided a method for controlling the above and below mentioned systems, the method comprising: a) receiving, by an interface unit of a system controller of the system, sensor data having instrument orientation information relative to a target position relative to the subject, the instrument orientation information comprising a position and orientation of an interventional instrument; b) receiving, by an interface unit of the system controller, image data at a target location, wherein a target trajectory is planned in the image data for placing an instrument at the target location; c) transferring, by a system controller of the system, the instrument pose information and the planned target trajectory to a target coordinate system and calculating a position error between the tracked instrument pose information and the planned target trajectory, and transferring the position error to a robot coordinate system of the robot system for controlling a robot arm to align the instrument with the planned target trajectory, wherein the position error comprises at least one of a translation error and a rotation error; It has.

[0043] According to a third aspect of the invention, a computer program element for controlling an apparatus is arranged to carry out the above and below methods when executed by a processing unit.

[0044] According to a fourth aspect of the present invention, there is provided a computer readable medium having stored thereon a program element.

[0045] As used herein, the term "elongate" is used in its ordinary sense, i.e., the length dimension is greater than the width or diameter of the device. As an example, an elongate device may be a needle-like device.

[0046] As used herein, the term "instrument" may refer to medical devices such as surgical tools, medical tools, biomedical tools, and diagnostic instruments. In some examples, an instrument may refer to an instrument for mechanical manipulation, such as an instrument for drilling, nailing, and concentric alignment.

[0047] As used herein, the term "object of interest" may refer to a patient, a human subject, or an animal subject. In some examples, the object of interest may refer to an object in manufacturing and machine operations, e.g., an object to be drilled, nailed, etc.

[0048] As used herein, the term "system controller" refers to, may be part of, or include an application-specific integrated circuit (ASIC), electronic circuitry, processor (shared, dedicated, or group) and / or memory (shared, dedicated, or group) executing one or more software or firmware programs, combinatorial logic circuitry, and / or other suitable components that provide the described functionality. The system controller may include a consumer electronics device, a smartphone, a tablet personal computer, a wearable computing device, a personal digital assistant (PDA), a laptop computer, and / or any other similar physical computing device capable of providing the described functionality.

[0049] As used herein, the term "instrument interface" may refer to a mechanical portion for receiving an instrument for placement on a robotic device. For example, the instrument interface may be a needle guide. A surgical device, such as an injection needle, may be inserted into and secured in the needle guide.

[0050] As used herein, the term "unit" refers to, may be part of, or include, an ASIC, electronic circuitry, processors (shared, dedicated, or group) and / or memory (shared, dedicated, or group) executing one or more software or firmware programs, combinatorial logic circuitry, and / or other suitable components that provide the described functionality.

[0051] Throughout this description, different coordinate systems are referenced. The coordinate systems may be Cartesian, polar, spherical, cylindrical, tetragonal, hexagonal, or any other three-dimensional (3D) coordinate system as described above. The coordinate systems referred to below are now defined.

[0052] The term "object coordinate system" refers to a coordinate system whose position is defined with respect to an origin, a preselected point associated with the object, whose relative position remains constant. For example, if the object is a patient, the term "object coordinate system" may be referred to as a patient coordinate system. The object coordinate system may be, for example, a Cartesian coordinate system with orthogonal axes x, y, and z, which may be defined with respect to the preselected point and may use natural language relative positioning terms such as left, right, up, down, forward, and backward to identify positions with respect to the preselected point. Alternatively, polar coordinates may be used to define positions in virtual space with respect to a preselected point associated with the object. Alternatively, any other 3D coordinate system may be used. Similarly, the term "robot coordinate system" refers to a coordinate system whose position is defined with respect to an origin, a preselected point associated with the robot, whose relative position remains constant. The robot coordinate system may be, for example, a Cartesian coordinate system with orthogonal axes x, y, and z, which may be defined relative to a preselected point thereof, and may use natural language relative positioning terms such as left, right, up, down, forward, and backward to identify positions relative to the preselected point. Alternatively, polar coordinates may be used to define positions in virtual space relative to a preselected point associated with the robot. Alternatively, any other 3D coordinate system may be used.

[0053] Similarly, the term "tracking coordinate system" refers to a coordinate system whose position is defined relative to an origin, a preselected point associated with the tracking system, and whose relative position to that origin remains constant. The tracking coordinate system may, for example, be a Cartesian coordinate system with orthogonal axes x, y, and z, which may be defined relative to that preselected point and may use natural language relative positioning terms such as left, right, up, down, forward, and backward to identify positions relative to the preselected point. Alternatively, polar coordinates may be used to define positions in virtual space relative to a preselected point associated with the tracking system. Alternatively, any other 3D coordinate system may be used.

[0054] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0055] Exemplary embodiments of the present invention are described below with reference to the following drawings: [Brief explanation of the drawings]

[0056] [Figure 1] FIG. 1 illustrates a perspective view of an exemplary hybrid operating room (OR) for performing minimally invasive robotic surgical procedures. [Figure 2] FIG. 1 illustrates a top view of an exemplary system for instrument placement in an illustratively illustrated hybrid OR, according to some embodiments of the present disclosure. [Figure 3A] 1 shows an example of a coordinate frame within the system for instrument placement. [Figure 3B] 1 shows an example of an error metric for alignment. [Figure 4] 1 shows a schematic diagram of an example of a robot arm. [Figure 5] 1 illustrates a flow diagram of a method for controlling a system according to some embodiments of the present disclosure. [Figure 6] 10 shows a flow diagram of a method for controlling a system according to some other embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0057] In the following, embodiments are demonstrated in the context of robotically guided pedicle screw placement surgery, but are generalizable to any robotic device placement application where precise (e.g., less than 2 mm) positional guidance is required. Examples of robotic device placement applications may include, but are not limited to, neurosurgery, tumor biopsy, resection, and many other minimally invasive surgical and percutaneous applications. Further examples of robotic device placement applications may include applications in manufacturing and machine operation, such as aircraft, construction machinery, etc.

[0058] Spinal fusion is a common surgical approach to address degenerative spinal disease and deformity, with approximately 450,000 procedures performed annually in the United States. In posterior spinal fusion, screws are placed within the pedicles of vertebrae and connected by rods to stabilize the vertebrae. Large-diameter screws are desirable for mechanical stability. However, maximizing screw diameter relative to pedicle size can increase the risk of pedicle fracture and damage to critical structures such as spinal nerves, spinal cord, or blood vessels. Accurate pedicle screw placement is therefore essential. The current trend toward minimally invasive surgery (MIS) also emphasizes the need for precision. MIS is an attractive technique associated with shorter hospital stays, less postoperative pain, less blood loss, reduced risk of infection, and lower costs. However, pedicle screw placement in MIS relies heavily on tools for guidance, as small, puncture-like incisions provide little or no visual feedback to the surgeon. Typically, guidance is provided by fluoroscopy, which has the drawbacks of radiation exposure to the patient and staff, and providing only a 2D view for a 3D targeting task.

[0059] Surgical navigation systems provide a way to plan the surgical approach and translate the preoperative plan into a desired intraoperative trajectory. Furthermore, such systems visually guide the surgeon (freehand) within the surgical field. However, surgical navigation requires the surgeon to manually keep instruments aligned with a virtual plane displayed on a screen outside the surgical field while performing the surgical procedure. Robotic aids are naturally suited to this alignment and stabilization task, and many commercial and academic solutions exist. Compared to the freehand approach, robotic use results in greater precision in pedicle screw placement, reduced radiation exposure to the surgeon, and earlier hospital discharge.

[0060] In pedicle screw fixation procedures, the primary assistance provided by the robot is the precise alignment of the guide or instrument with a planned trajectory defined in the navigation software. Once the instrument (e.g., needle) is aligned with the target and held firmly in this alignment by the robot, the surgeon advances the needle or drill into the bone and further into the interior of the vertebra. After the holes are created, the surgeon places screws inside these pilot holes and secures adjacent screws with rods to fuse the vertebrae into the desired configuration.

[0061] Robotic solutions typically rely on external tracking systems to localize the robot and establish a robot-to-patient (dynamic) reference frame using optical markers firmly fixed to the patient's spine or iliac crest. These dynamic reference frames tend to be mounted far from the target vertebrae and are prone to accidental shifts and occlusions, reducing patient tracking accuracy and increasing the risk of registration errors. In addition to occupying significant space in often crowded ORs, existing systems require a manual registration step using specialized hardware that must be installed on the robot or invasively on the patient.

[0062] Because the desired tool position is defined in a tracking coordinate system, the robot must somehow be tracked and / or aligned to the tracking coordinate system to allow for the transformation of the target position relative to its own coordinate frame (CF) in order to move the instrument to the target trajectory. This is often done using large markers on the robot base, which requires a large tracking field of view and adds another transformation between the target and the instrument being placed. Others add large 6-DOF device tracking markers and calculate the tracker position through standard kinematic inverse transformations, which requires multiple calibrations and high accuracy. Other solutions rigidly fixate the robot to the patient or table and then perform the alignment step. These approaches require very good forward kinematics and very high-quality parts, assemblies, calibrations, and general stiffness of the robot and fixation.

[0063] FIG. 1 shows a perspective view of an exemplary hybrid OR 100 for performing minimally invasive robotic surgical procedures, where the robot can be used for precise alignment of guides or instruments with planned trajectories defined in navigation software.

[0064] The exemplary hybrid OR 100 is a single room with a dedicated C-arm X-ray imaging system 102 capable of two-dimensional (2D) or three-dimensional (3D) imaging. The C-arm imaging system 102 has a support structure 104 that can translate through azimuth and elevation axes around an object of interest 106. For example, the C-arm X-ray imaging system 102 may be supported from the ceiling of an X-ray facility. The support structure holds a rotating anode X-ray source 108 and an X-ray detector 110.

[0065] The exemplary hybrid OR 100 also has a patient support 112 for supporting the object of interest 106. The C-arm 102 is configured to translate around the object of interest 106 not just in the sense of flat rotation (in the sense of a CT scanner), but also by tilting.

[0066] A typical hybrid OR 100 typically has an external display 114 that allows the surgeon to view the internal surgical site to provide surgical navigation.

[0067] The C-arm X-ray imaging system 102 is controlled from a control console 116 having, for example, a display screen 118 and, optionally, a computer device 120 that serves as a stator control system and is controllable via a keyboard 122 and a mouse 124 .

[0068] Hybrid ORs can improve facility utilization by encompassing many procedures, from endovascular to minimally invasive or open surgery, and enable the exploration of new procedures that take advantage of high-quality intraoperative imaging and high levels of device integration. Incorporating robotic guidance into this environment can provide added precision and streamline surgical workflow by accurately transferring the surgical plan to the patient. However, this can be challenging due to geometric constraints from existing OR equipment, limited surgical workspace and imaging volume, and well-established surgical workflows.

[0069] For example, for spine surgery in a hybrid OR, a robotic guidance system for pedicle screw placement could be quickly set up near the surgical field, low-profile, and sterile. Additionally, it may be necessary to provide visual access for the surgeon and optical tracking system. The robot may need to reach all planned trajectories for a given volume without adjusting its base position to minimize workflow disruption. It may also need a quick and simple method for retraction from the surgical field whenever the surgeon uses it, and it may need to fit under the detector during a conventional cone-beam computed tomography (CBCT) scan without inducing significant image artifacts. The surgeon should not be required to manually align the reference frames for the surgical plan, patient tracking, and robot. Patient tracking must be noninvasive and robust to partial occlusions. Most importantly, the robot must be highly accurate in transferring the navigation plan to the patient (<0.5 mm and <0.5 degrees of the planned trajectory) to minimize clinically unacceptable pedicle breakages greater than 2 mm.

[0070] Based on the above requirements, a system is proposed to provide fully integrated intraoperative 3D imaging and planning, and automatic alignment of instruments to a planned trajectory using a robot with at least four degrees of freedom control.

[0071] FIG. 2 shows a top view of an exemplary system 10 for instrument placement in the illustrated hybrid OR 100.

[0072] The system 10 includes a robotic system 12 having one or more robotic arms 14, such as robotic arms 14a, 14b, and 14c of FIG. 2. The robotic system 12 may be mounted to the side of a patient support 112, outside the surgical field, for example, via a rail-mount adapter 16. Each robotic arm 14 has four or more degrees of freedom of control. The robotic arms 14 have an instrument interface, e.g., a needle guide, connectable to an instrument 18, such as instruments 18a, 18b, and 18c of FIG. 2. As an example, the central robotic arm 14b may support an endoscopic camera 18b. The robotic arms 14a and 14c may support an interventional instrument, such as an interventional laser device, for manipulating tissue.

[0073] The instrument 18 may be any instrument or tool connectable to a robotic arm that can be manipulated thereby. Examples of the instrument 18 may include, but are not limited to, surgical tools, medical tools, biomedical tools, and diagnostic tools. Surgical tools may include, for example, irrigation and infusion needles, tips and tubes for introducing fluids, scopes and probes (e.g., fiber optic endoscopes and tactile probes), ultrasonic tissue disruptors, drills, cryotomes, and cutting laser guides. Diagnostic instruments may include, for example, ultrasound instruments, computed tomography (CT) scanners, and magnetic resonance imagers (MRI).

[0074] Generally, the robotic arm 14 may be divided into two modules, including a translation module and a rotation module, such as rotation modules 20a, 20b, and 20c. An example of the two modules is shown in Figure 4. The translation module may be a 3DOF translation module that allows the instrument 18 to be translated to the entry point and then rotated with the rotation module for alignment.

[0075] The system 10 further includes a system controller 22 having an interface unit 24. For example, the system controller 22 may be the exemplary control console 116 of FIG. 1 . The interface unit 24 of the system controller 22 is configured to provide sensor data having instrument pose information and a target position relative to the object of interest 106. The instrument pose information includes the position and orientation of the interventional instrument. The sensor data may be obtained from a tracking system 26. For example, the tracking system 26 may be an optical tracking system having one or more cameras or optical sensors that may be disposed in a hybrid OR to capture sensor data having instrument pose information. Non-invasive, disposable passive skin markers may be distributed around the target position, e.g., around the surgical incision, to provide the required occlusion redundancy and tracking robustness. By tracking these markers on the patient during image acquisition, the object coordinate system may be aligned with the image coordinate system used for the planning and / or tracking coordinate systems. Alternatively or additionally, the tracking system may include an electromagnetic tracking system and / or an acoustic tracking system.

[0076] For example, for an optical tracking system, markers coated with retroreflective material can be used to reflect light generated near the camera lens. The camera's threshold can be adjusted so that only bright reflective markers are sampled, ignoring skin and fabric. Alternatively, fiducial markers can be used. The marker's centroid is estimated as its location within the captured two-dimensional image. The grayscale value of each pixel can be used to provide sub-pixel accuracy by finding the centroid. The position of each marker can be used to define the axes of the object coordinate system relative to the tracking coordinate system. The origin of the object coordinate system can be defined arbitrarily, and the axis direction can be defined based on the marker's position around the target location. Thus, the tracking coordinate system can be aligned with the object coordinate system based on the marker's position (and possibly the marker's orientation).

[0077] Instead, an electromagnetic tracking system could calculate the position and orientation of the electromagnetic tracker around the target location by the relative magnetic flux of three orthogonal coils in both the transmitter and receiver. The relative strength of the voltages or currents in the three coils allows the electromagnetic tracking system to calculate both range and orientation by carefully mapping the tracking volume. Similarly, the tracking coordinate system can be aligned to the target coordinate system based on the measured position and orientation of the marker. Other tracking systems can also be used.

[0078] Preferably, as shown in FIG. 2, the tracking system 26 may be located on or within the detector 110 of the image acquisition system. Locating the tracking system on or within the detector essentially spatially aligns the tracking coordinate system to the image coordinate system. This may, for example, alleviate the need to independently track and align the tracking coordinate system of an external tracking system to the image coordinate system. Tracking the instrument 18 in this manner may provide more accurate tracking of the instrument compared to, for example, using a shape sensor on the robot, where errors are compounded back to the robot's fixed base.

[0079] The interface unit 24 of the system controller 22 is further configured to provide image data at the target location. The image data may be acquired by an image acquisition system, such as an X-ray imager, an ultrasound imager, or a magnetic resonance imager, as shown in FIG. 1. Target trajectories 28, e.g., skin entry points and target points, are planned in the image data by a user 30, e.g., a surgeon, for placing instruments at the target location. For example, an intraoperative CBCT acquisition is performed, upon which the surgeon plans 3D instrument trajectories, such as pedicle screw positions.

[0080] The system controller 22 is configured to transfer the attitude information of the instrument 18 and the planned target trajectory 28 to the object coordinate system. This is accomplished by registering the object coordinate system with the tracking coordinate system and the image coordinate system. There are many methods known to those skilled in the art. One of these methods involves using a tracking system with markers on the object and the instrument, as previously mentioned.

[0081] The system controller 22 is further configured to calculate a position error between the tracked attitude information of the instrument and the planned target trajectory. The position error includes at least one of a translation error and a rotation error. The error metric for alignment is the angle between the target and the trajectory, while the translational alignment is the length of the vector from the instrument to the target trajectory vector.

[0082] FIG. 3A shows an example of a coordinate frame in the system 10. The target and entrance are defined in an object coordinate system. In the illustrated example, the object coordinate system is also referred to as a patient coordinate system. The tracking pose information of the instrument is based on the instrument tip (P D ) and its main axis ( The target trajectory may have the location of the skin entry point (P) corresponding to the planned screw trajectory. E ) and the medial target point of the pedicle (P Q ) All of these are transformed into the patient coordinate system (S). The main assumptions are that the instrument axis is collinear with the robot guide axis and that the patient fiducial is not significantly moved relative to the robot base during the targeting session.

[0083] Figure 3B shows an example of an error metric for alignment. The error metric for alignment is the angle between the target and the trajectory, while translational alignment is the angle between the target and the safety plane defined by the robot's XY plane and the current robot guide position. Ris the length of the vector from the instrument to the target trajectory vector at π. Alternatively, the safety plane position may be defined relative to the tip of the instrument, then transformed to the robot coordinate system and closest to the point on the instrument to the target trajectory with some reasonable constraints so as not to stray too far from the robot guide, or relative to the entry point.

[0084] In Figure 3B, TIFF0007786370000002.tif64 is the translation from the instrument axis to the target trajectory axis in the plane π1, TIFF0007786370000003.tif69 is the 3DOF rotation (axis-angle) between the device axis and the target trajectory axis.

[0085] The system controller 22 is configured to transfer the position errors to the robot coordinate system of the robotic system to control the robotic arms to align the instruments to the planned target trajectories. For example, as shown in Figure 2, the system controller 22 may use one or more robotic arms 14 to direct the movement of robotically controlled devices 18a-18c via control lines 302 that effect instrument movement to align the instruments to the planned target trajectories.

[0086] During operation, a subject of interest 106, e.g., a patient, is delivered and prepared for surgery. An intraoperative CBCT acquisition is performed, upon which a user 30, e.g., a surgeon, plans 3D instrument trajectories, such as pedicle screw locations. The planned trajectory, including skin entry and target points, is transformed into physical patient space and made available to the robotic system along with instrument tracking. Instrument pose information is used by the system controller 22 for automatic alignment to targets defined in the patient coordinate system. Once aligned, the robot holds position, and the surgeon drives instruments into the pedicles to create pilot holes where the screws will be placed. The surgeon can manually move the robotic arm away from the surgical site by enabling "swing arm" mode, or leave the robotic arm in place for additional guidance or verification imaging.

[0087] Thus, by extending the robotic arm 14 into the surgical field, the system 10 can automatically align the instrument 18 according to the surgical plan using only instrument tracking feedback; tracking markers on the robotic arm 14 are not required. The proposed system may provide an accurate and workflow-friendly robotic guidance system for, for example, generating pedicle screw pilot holes in spinal fixation procedures. The robotic arm precisely aligns the instrument with the desired trajectory, and the surgeon hammers or drills the instrument into the pedicle. The system controller uses a servo control method to achieve high alignment accuracy, relying solely on, for example, 4-DOF or 5-DOF instrument tracking feedback; no manual robotic alignment steps are required.

[0088] Optionally, the sensor data includes pose information of the robot arm. This may enable hybrid feedback control including both robot end-effector tracking and equipment tracking feedback. This may allow for coarse alignment using low-precision robot tracking even without an in-guide instrument; once the device is visible in the tracking system and close to the target trajectory, the proposed system can be used to servo for final high-precision alignment.

[0089] Robot System FIG. 4 shows a schematic diagram of an example of a robotic arm 14. The robotic arm 14 may feature five active joints and a cycled passive rotation inside a single-axis instrument guide as the end effector. The example robotic arm 14 is divided into two modules. The 3-DOF translation module includes a vertical linear stage (J0), a shoulder rotation joint (J1), and a horizontal linear stage (J2). It carries a 2-DOF rotation module, i.e., a mechanical remote center of motion (RCM) mechanism. In some instances (not shown), aligning to a predetermined plane (e.g., 4-DOF) may be sufficient, and therefore, a 1-DOF rotation module may be carried for these instances. The RCM may have two spherical linkages, J3 and J4, both with a 65° angle, with J3 mounted at 60° relative to the vertical. This continuous kinematic architecture is selected to meet the minimum number of degrees required to position a needle-like instrument within Cartesian space. The inclusion of the RCM may minimize overall joint motion and create an ergonomic robotic configuration and predictable motion for an improved user experience, especially when performing Cartesian rotations. It allows the instrument to be translated to the entry point by primarily actuating the first three joints and then rotated to alignment using only the two distal (RCM) joints. The updated link geometry is optimized to bring the instrument guide closer to the RCM, allowing for guide placement closer to the skin incision, improving the accuracy of trajectory transfer to the patient, and facilitating the use of shorter instruments.

[0090] The robotic system 10 may include a rail-mounting adapter 16 that can be attached to a standard OR table rail, or a rail integrated into a custom plate placed under the patient to provide rigidity and positioning flexibility. The link lengths and spherical link angles provide sufficient coverage for at least six levels and ±60° to cover abnormal trajectory angles beyond the range of a healthy patient: -14° to 20° in the sagittal plane and ±22° to 35° in the transverse plane. The advanced link allows for the installation of variable-diameter guides that can lock instruments in place or provide lateral exit capability. The robot body prototype was machined from aluminum channels and 3D printed for non-structural members. Common off-the-shelf mechatronic components were used for the motor and transmission. A 6-DOF force-torque (FT) sensor may be integrated into the robot's advanced link to provide FT measurements at the instrument guide. FT is used for admittance (hands-on placement) control methods and for monitoring loads within the guide during intervention. It provides a GUI, forward / inverse kinematics, and various control modes (force control, ARSN align servo, remote joystick, insertion, etc.). The joystick is used for remote placement input and may be used to trigger auto-alignment and enable hands-on placement mode. In some embodiments, the FT interface can also be used to trigger auto-alignment by detecting when the user is holding the instrument guide. Once the guide is no longer being held, auto-alignment stops.

[0091] System Controller The system controller 22 may be controlled by a trajectory alignment algorithm to align the instrument to the planned target trajectory 30. The goal of the trajectory alignment algorithm is to align the instrument with the planned target trajectory with as much precision as possible without adding additional (alignment) steps to the surgeon's workflow. For instruments with 4 or 5 DOF, instrument tracking feedback may be inherently insufficient for 6 DOF Cartesian robot positioning without accurate coordinate system (CS) alignment, especially since the exact position, e.g., depth, of the instrument within the instrument interface is unknown.

[0092] For instruments with elongated shapes, the proposed system controller 22 may be configured to control the robot arm during its first movement for each new targeting session to move the instrument in a predetermined motion to generate a coarse six-degree-of-freedom alignment between the subject coordinate system and the robot coordinate system without a 6-DOF reference body on the robot. The predetermined motion may include a predetermined rotational motion and, optionally, a predetermined translational motion. In general, the predetermined rotational motion for the coarse alignment may be arbitrary, as long as it includes a large rotation, since larger angles lead to more accurate alignment, assuming the robot system motion and instrument tracking feedback can be precisely synchronized. Arbitrary transformations may also be added. This feature may be used in real time to update or verify that the alignment has not changed due to patient movement relative to the robot system.

[0093] For example, referring now to FIG. 3B, a coarse alignment ( S RTo generate T, the tracked needle is fixed to the guide (to prevent it from slipping off), and upon commanding alignment to the first target in a targeting session, the robot initiates movement by, for example, rotating 15 degrees around its RCM in a predetermined direction (towards the opposite side of the table). Of course, the robot may also initiate movement by rotating other degrees around its RCM in a predetermined direction, such as 5 degrees, 10 degrees, 20 degrees, 30 degrees, 40 degrees, 50 degrees, 60 degrees, 70 degrees, 80 degrees, or 90 degrees. Two robot poses ( S R T1 and S R T2) and two needle positions ( S N1, S N2) were collected before and after the move, S R T is calculated using the temporary coordinate system (D). Note that the patient coordinate system is referred to as (S) and the robot coordinate system is referred to as (R). S R T) is calculated as follows:

[0094] Translational component ( S P D ) is the robot's RCM position ( R P RCM = R P D ) using a least squares approach to find the needle axis ( S N1, S N2).

[0095] ( S D R) is calculated by multiplying the rotational component of the two line normals using the standard cross product method. Created from TIFF0007786370000004.tif719. The same applies to the robot guide z-axis normal. From TIFF0007786370000005.tif716 R D This is done for R.

[0096] The (object) registration between the patient coordinate system and the temporary coordinate system is S D It is called T.

[0097] The (robot) alignment between the robot coordinate system and the temporary coordinate system is R D It is called T.

[0098] The alignment between the patient coordinate system and the robot coordinate system is S R It is called T.

[0099] S D T and R D T is [ S D R, S P D ] and [ R D R, R P D ]. Then, S R T is simply S D T R D T -1 is.

[0100] The resulting alignment process may take several seconds and may be sufficient to position the instrument near the desired trajectory in an open-loop manner, for example, when outside the tracking volume, but due to limited alignment data, small robot inaccuracies may not be sufficient for precise alignment.

[0101] For precise alignment, real-time needle position feedback may be used to repeatedly move the needle mounted within the guide to align with the target trajectory. Once aligned, the surgeon can lower the robot along the guide axis (with force control via an insertion constraint along the axis) and command the robot to align with the target as needed.

[0102] Optionally, the sensor data may include real-time pose information of the instrument with respect to the target position. Optionally, the height of the robot guide, i.e., the equipment interface, is kept constant during alignment. The height constraint may provide safer and more intuitive robot behavior. The system controller may be configured to calculate a real-time position error between the tracked pose information of the instrument and the planned target trajectory, and transfer the real-time position error to a robot coordinate system of the robotic system for iteratively controlling the robot arm to align the instrument to the planned target trajectory.

[0103] For example, referring to FIG. 3B, the goal position of the guide in the robot coordinate system for each iteration ( R G T) may be calculated as follows:

[0104] The robot's XY plane ( R [0,0,1] T ) plane defined by R π1 and the current robot guide position R P RCM Think about it.

[0105] R P Q1 , i.e., the target line ( R P Q , TIFF0007786370000006.tif66) and R The intersection point between π1 and R P N1 , i.e., the needle axis and R Calculate the intersection point of π1.

[0106] The displacement error vector is TIFF0007786370000007.tif65= R P Q1 - R P N1 and the goal translation ( R P G )= The file is TIFF0007786370000008.tif720. R P Q1 and R P N1 is in the XY plane of the robot coordinate system (almost parallel to the table), so the guided translation of the robot is only within this plane.

[0107] The rotation error is TIFF0007786370000009.tif66 and target trajectory axis tif65 is calculated in the robot guide frame by generating the axis angle rotation between TIFF0007786370000011.tif621 and the angle between them is α. Normalize τ and then Set TIFF0007786370000012.tif65 to 0.0 and normalize again.

[0108] This is because the robot cannot perform such rotations, in addition to being arbitrarily defined by the 4-DOF or 5-DOF target and instrument definitions. Remove rotation around TIFF0007786370000013.tif78.

[0109] The desired goal is R G R= R E R E G R, where E G R is the approximate rotation error expressed as a rotation matrix The file is TIFF0007786370000014.tif711.

[0110] The rotation and translation goals are defined as a single homogeneous transformation [ R R G , R G] and sent to the robot position controller.

[0111] Each servo iteration may be executed after the previous robot command has been completed. In a typical setup, the robot: Fewer than five iterative steps are required to align the tracked needle with the target within an error tolerance of TIFF0007786370000015.tif89<0.25 mm and α<0.25.

[0112] Optionally, the robot's XY plane ( R [0,0,1] T ) safety plane defined by R π1 and the current robot guide position R P RCM may be updated at each iteration defined by a mesh of triangles, each defining a plane along which the robot can translate. The mesh may be offset, e.g., 5 cm, from a patient surface model, constraining the robot's motion to this offset. Such a model may be an approximation from stickers placed on the patient.

[0113] In some examples, the system controller 22 may calculate an approximate alignment and interactively control the robot toward the target in two separate steps. Alternatively, the system controller 22 may simultaneously drive the robotic system toward the target while continuously updating / improving the alignment between the target and target coordinate systems.

[0114] 5 shows a flow diagram of a method 200 for controlling a system as described above and below, according to some embodiments of the present disclosure. The method comprises the following steps:

[0115] In step 210, i.e., step a), sensor data is received by an interface unit of a system controller of the system. For example, a tracking system, such as an optical tracking system, is provided to acquire the sensor data and transmit the acquired sensor data to the system controller. The sensor data comprises instrument pose information with respect to a target position relative to the subject. The instrument pose information comprises the position and orientation of the interventional instrument.

[0116] In step 220, i.e., step b), image data at the target location is received by the interface unit of the system controller. A target trajectory is planned in the image data for placing the instrument at the target location. Steps a) and b) can be performed in various orders, such as a) → b), b) → a), or a) and b) simultaneously.

[0117] In step 230, i.e., step c), the instrument pose information and the planned target trajectory are transferred to a target coordinate system by a system controller of the system. A position error between the tracked instrument pose information and the planned target trajectory is calculated. The position error is transferred to a robot coordinate system of the robot system for controlling the robot arm to align the instrument with the planned target trajectory. The position error includes at least one of a translation error and a rotation error.

[0118] FIG. 6 shows a flow diagram of a method 200 according to some other embodiments of the present disclosure. In the method 200, a coarse alignment step 202 may be provided before steps 210 to 230 for instruments having an elongated shape. In this step, during the first movement of the robot arm for each new targeting session, the robot arm is controlled to move the instrument in a predetermined motion to generate a coarse six-degree-of-freedom alignment between the subject coordinate system and the robot coordinate system. The predetermined motion includes at least one of a predetermined translational motion and a predetermined rotational motion. The first movement of the robot arm automatically initiates the coarse alignment process. Thus, the system controller may automatically perform the coarse alignment, avoiding user input or adding time to the procedure. The resulting alignment process may take several seconds and may be sufficient to position the instrument near the desired trajectory in an open-loop manner, for example, when it is outside the tracking volume; however, due to limited alignment data, small robot inaccuracies may not be sufficient for precise alignment.

[0119] In other words, together with the registration process 202, the method may further include step 240. In this step, the instrument is aligned with the planned target trajectory based on a coarse six-degree-of-freedom registration between the subject coordinate system and the robot coordinate system. In other words, the coarse six-degree-of-freedom registration is used to position the instrument near the desired trajectory in an open-loop manner, for example, when outside the tracking volume. Once the instrument is detected, for example, when it is within the tracking volume, step 230 may be executed.

[0120] In another exemplary embodiment of the invention, a computer program or a computer program element is provided, characterized in that it is configured to perform, on a suitable system, the method steps of the method according to one of the previous embodiments.

[0121] Thus, a computer program element may be stored in a computing unit that may be part of an embodiment of the present invention. This computing unit may be configured to perform or direct the performance of the steps of the above-mentioned method. Furthermore, it may be configured to operate the components of the above-mentioned apparatus. The computing unit may be configured to operate automatically and / or to execute a user's order. The computer program may be loaded into the working memory of a data processor. The data processor may thus be configured to perform the method of the present invention.

[0122] This exemplary embodiment of the present invention encompasses both computer programs that use the present invention from the beginning and computer programs that, through updates, turn existing programs into programs that use the present invention.

[0123] Furthermore, the computer program element may be capable of providing all the steps necessary to fulfill the procedures of the exemplary embodiments of the methods described above.

[0124] According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is presented, the computer readable medium having stored thereon a computer program element, the computer program element being as described by the preceding section.

[0125] The computer program may be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0126] However, the computer program may also be presented over a network such as the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, the computer program element being configured to perform a method according to one of the aforementioned embodiments of the present invention.

[0127] It should be noted that the embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method-type claims, and other embodiments are described with reference to apparatus-type claims. However, those skilled in the art will understand from the above and below description that, unless otherwise specified, any combination of features belonging to one type of subject matter, as well as any combination between features relating to different subject matters, is disclosed in the present application. However, all features can be combined to provide a synergistic effect that is greater than the simple sum of the features.

[0128] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered exemplary or explanatory and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the dependent claims.

[0129] In the claims, the word "comprise" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. 1. A system for placing an instrument, the system comprising: a robotic system having a robotic arm with control of four or more degrees of freedom (DOF); a system controller having an interface unit; and the robotic arm has an instrument interface connectable to an instrument; The device has an elongated shape; the interface unit of the system controller is configured to provide sensor data having pose information of the instrument and a target position relative to an object of interest, the pose information of the instrument comprising a position and orientation of the instrument; the interface unit of the system controller is further configured to provide image data at the target location, and a target trajectory is planned within the image data for placing the instrument at the target location; The system controller transferring the pose information of the instrument and the planned target trajectory to a target coordinate system; calculating a position error between the attitude information of the instrument and the planned target trajectory; controlling the robot arm during a first movement of the robot arm for each new targeting session to move the instrument in a predetermined motion to generate a coarse six degrees of freedom alignment between the object coordinate system and the robot coordinate system, the predetermined motion having a predetermined rotational motion, the coarse six degrees of freedom alignment between the object coordinate system and the robot coordinate system being generated based on pose information before and after the motion for each of the instrument and the robot arm; transferring the position error to the robot coordinate system of the robot arm to control the robot arm to align the instrument with the planned target trajectory, the position error comprising at least one of a translation error and a rotation error. It is configured as follows: system.

2. Generating a coarse six-degree-of-freedom alignment between the object coordinate system and the robot coordinate system comprises: collecting two robot poses and two tool positions before and after said predetermined movement; determining the instrument translation components based on calculating the closest point between two instrument line vectors representing the instrument axis from the instrument positions before and after the rotation using a least squares method; creating an instrument rotation component from the normals of said two instrument line vectors using standard cross product methods; creating an object registration between the object coordinate system and a temporary coordinate system based on the instrument translation component and the instrument rotation component; determining robot translation components based on calculating the closest point between two robot line vectors representing robot axes from the robot pose before and after rotation using a least squares method; creating robot rotation components from the normals of the two robot line vectors using standard cross product methods; generating a robot alignment between the robot coordinate system and the temporary coordinate system based on the robot translational component and the robot rotational component; and generating a coarse six-degree-of-freedom alignment between the object coordinate system and the robot coordinate system based on the object alignment and the robot alignment; The system of claim 1 , wherein the system is based on

3. the sensor data comprises real-time pose information of the instrument relative to the target location; the system controller is configured to calculate a real-time position error between the pose information of the instrument and the planned target trajectory, and transfer the real-time position error to a robot coordinate system of the robotic system for iteratively controlling the robot arm to align the instrument with the planned target trajectory.

3. The system according to claim 1 or 2.

4. the system controller is further configured to apply the coarse six-degree-of-freedom alignment to control the robot arm to align the instrument with the planned target trajectory when the sensor data does not have pose information of the instrument.

4. The system of claim 1, 2 or 3.

5. The system comprises: a tracking system configured to acquire the sensor data comprising the pose information of the instrument and the target position of the object of interest; The tracking system further comprises: an optical tracking system; an electromagnetic tracking system; an acoustic tracking system; having at least one of: A system according to any one of claims 1 to 4.

6. The system comprises: an image acquisition system configured to acquire the image data at the target location of the object of interest; the image acquisition system further comprises: an X-ray imaging device; an ultrasound imaging device; a magnetic resonance imaging device; having at least one of:

6. A system according to any one of claims 1 to 5.

7. the tracking system is located on or within a detector of the image acquisition system; The system according to claim 6 dependent on claim 5.

8. the robot arm has a mechanical RCM mechanism with at least one degree of freedom of rotational control; the instrument interface is mounted on the mechanical RCM mechanism; A system according to any one of claims 1 to 7.

9. the system controller is configured to control the robotic arm to align the instrument to the planned target trajectory while translating the instrument within a safety plane to prevent collisions. A system according to any one of claims 1 to 8.

10. the sensor data includes posture information of the robot arm; 10. A system according to any one of claims 1 to 9.

11. the device comprises an interventional device; 11. A system according to any one of claims 1 to 10.

12. The interventional device comprises: A syringe needle and an interventional catheter; an interventional laser device; having at least one of: The system of claim 11.

13. A method of controlling a system according to any one of claims 1 to 12, comprising: receiving, by an interface unit of a system controller of the system, sensor data comprising pose information of an instrument relative to a target position relative to an object of interest, the pose information of the instrument comprising a position and orientation of the instrument; receiving, by the interface unit of the system controller, image data at the target location, wherein a target trajectory is planned in the image data for placing an instrument at the target location; transferring, by a system controller of the system, the pose information of the instrument and the planned target trajectory to a target coordinate system; calculating a position error between the attitude information of the instrument and the planned target trajectory; controlling the robot arm during a first movement of the robot arm for each new targeting session to move the instrument in a predetermined movement to generate a coarse six degrees of freedom alignment between the object coordinate system and the robot coordinate system, the predetermined movement comprising a predetermined rotational movement, the coarse six degrees of freedom alignment between the object coordinate system and the robot coordinate system being generated based on pose information before and after the movement for each of the instrument and the robot arm; transferring the position error to a robot coordinate system of the robot system for controlling the robot arm to align the instrument with the planned target trajectory, the position error comprising at least one of a translational error and a rotational error; A method having the following.

14. A computer program element for controlling an apparatus, configured to carry out the method of claim 13 when executed by a processing unit.

15. A computer readable medium storing the computer program of claim 14.

Images