Method and system for tracking multiple optical markers in robotic surgical procedures

Abstract

The robotic surgical system includes a first robotic arm, a second robotic arm, a first camera on the first robotic arm, and a second camera on the second robotic arm. A controller receives images from the first camera and the second camera, kinematically positions the first robotic arm and the second robotic arm in a surgical coordinate space, kinematically tracks the position of the first camera in the surgical coordinate space, and optically tracks the position of the second camera using the first camera. A position of a secondary marker within the field of view of the second camera is calculated based on the position of the second camera as tracked by the first camera.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 63 / 442,457, filed January 31, 2023, which is incorporated herein by reference.

[0002] (Field) The disclosed technology relates to systems and methods for surgical robotic registration and navigation, and more particularly, to systems and methods that use navigational markers that are attachable to bones and other patient anatomy. [Background technology]

[0003] Surgical and other robotic systems often utilize cameras or sensors to track objects within the robotic space surrounding the surgical robot. In some robotic surgical procedures, radiopaque (RO) markers may be attached to the patient's bones or other anatomical structures, the patient may be imaged by computed tomography (CT) scanning, and the marker locations may be used to "register" the patient's image within the robotic surgical space. For example, as taught in co-owned PCT Application No. PCT / IB2022 / 052297 (published as WO2022 / 195460), the entire disclosure of which is incorporated herein by reference, one or more RO markers shown in the patient CT scan may be screened by cameras located on the arms of a multi-arm surgical robot to establish the initial positions of the markers within the robot's surgical space, with a coordinate system defined relative to the robot's chassis or cart. During subsequent surgical procedures, changes in the patient's position can be calculated based on observed changes in marker positions over time.

[0004] The surgical robot described in PCT Publication No. WO 2022 / 195460 and other commonly used surgical robots typically rely on a single camera and a single marker to track patient position during robotic surgery. While practical, the use of a single marker and a single camera requires a relatively large marker because the camera may not always be close enough to or properly aligned with the marker to allow the marker to have a reduced size. In other words, the marker target needs to be large to allow the camera to accurately determine its location from a distance.

[0005] In currently available robotic surgical systems, when the camera is placed a standard 1.5 m to 2.5 m from the patient and patient markers, navigation markers typically measure at least 7 cm to 15 cm in width to provide the necessary 1 mm to 2 mm accuracy at the tool tip. Such large markers are disadvantageous because they can interfere with the surgeon's view of and access to the surgical site and they can easily deflect due to their own weight, causing a loss of accuracy.

[0006] The use of registration markers as a first step in robotic surgical procedures is well known. This step is performed to align the coordinate system of a robotic surgical system, such as a robotic spinal surgery system, with the patient's anatomy. A navigation marker may then be positioned in place of the registration marker to synchronize the robotic navigation system with the patient's anatomy.

[0007] Although generally successful, the use of both registration and navigation markers has several drawbacks. First, the redundancy adds time, cost, and complexity to the procedure. Second, performing a CT scan for registration exposes the patient to radiation.

[0008] It would therefore be desirable to provide improved robotic surgical systems and methods. In particular, it would be desirable to provide robotic surgical systems and methods that do not require an initial alignment step for robotic navigation or any other purpose. Furthermore, it would be desirable to provide surgical robotic systems and methods that allow for the viewing and tracking of relatively small markers that do not interfere with the surgical procedure without any substantial loss of accuracy or precision. Still further, it would be desirable for surgical robotic systems and methods to allow for the placement of multiple navigational markers even after the surgical procedure has begun. At least some of these objectives will be met by the invention(s) described and claimed herein. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] International Publication No. 2022 / 195460 Summary of the Invention [Means for solving the problem]

[0010] The disclosed systems and methods address the above-mentioned shortcomings. By employing multiple repositionable navigation cameras and / or other sensors and multiple navigation markers fixed to different locations on the patient, smaller navigation markers can be used while maintaining accurate scanning and / or tracking of the patient's anatomy. In some embodiments, multiple robotic arms can be operated within the surgical field, with at least one arm holding a navigation camera or other sensor and at least one other arm holding a tool or end effector, and the arms are manipulated by a robotic system controller. Such systems and methods are useful in orthopedic surgical procedures in which markers are fixed to the patient's bony anatomy, for example, to individual vertebrae of the patient's spine.

[0011] While the disclosed technology will find particular use with optical cameras, the principles of the technology can be applied to any sensing technology and are particularly useful for sensing technologies that are limited by line-of-sight visibility and / or by proximity between the sensor and marker. Suitable sensing technologies include laser scanning or tracking, such as light detection and ranging sensors (LiDAR), magnetic sensing, scanning and tracking, ultrasonic sensing, scanning and tracking, and the like.

[0012] In some embodiments, the placement of multiple "mini" markers on a patient's bone anatomy will allow a small navigation camera (also referred to herein as a secondary camera) mounted on a surgical robotic arm, often along with a tool or end effector, to access and track areas of the surgical field that would be inaccessible to a larger navigation camera (also referred to herein as a primary camera) mounted on a dedicated director arm. Accordingly, systems and methods are presented for fixing multiple miniature markers, sometimes referred to herein as secondary markers, on a patient's bone anatomy, which can be tracked by one or more small navigation cameras mounted on the robotic arm carrying the tools and / or end effectors used to perform the procedure.

[0013] In some embodiments, the small or secondary camera(s) may be removably mounted on a surgical robotic arm (e.g., they may be add-on devices), while the larger or primary camera may be attached to a dedicated director arm. The larger or primary camera may be configured to track an attached larger or primary navigation marker and provide a director view of most or all of the surgical field. In addition, the primary camera may track the secondary camera such that the position of the secondary marker within the surgical field may be tracked by kinematically tracking the position of the primary camera (based on the kinematics of the director arm) and optically tracking the position of the secondary camera using the primary camera. All movement, tracking, and calculations may be performed by the robotic controller.

[0014] In a surgical robotic system in accordance with the disclosed technology, multiple surgical robotic arms may be mounted on a single chassis (typically a single mobile chassis or cart). The phrase "single chassis" means that the chassis provides a single rigid platform when present under the operating table, which in turn provides a single surgical coordinate space. For example, a "single chassis" may comprise two, three, or more mobile or other components, subassemblies, or the like, which may be joined on-site beneath the table to form a single chassis in accordance with the disclosed technology. In other examples, such separate components, subassemblies, or the like may be pre-assembled at the surgical facility or elsewhere before being moved to a location beneath the surgical table. While such a single chassis will typically have a unitary construction, in other cases, the platform may comprise a two, three, or more component structure, which is assembled on-site at the surgical facility.

[0015] Multiple surgical robotic arms may carry and deploy various surgical tools, end effectors, navigation cameras, and the like, as well as a robotic controller. The system may include a display and user interface mounted on or within a single chassis. The controller may automatically control the movement of some or all of the surgical robotic arms, director arms, and other robotic system components based on information provided by the primary and secondary navigation cameras. In some embodiments, the controller may display images from the cameras and allow the surgeon to manually control some or all of the surgical tools or end effectors.

[0016] The robotic system of the disclosed technology is advantageous because multiple navigation cameras do not interfere with the surgeon's line of sight and workflow and can be optimally placed for patient safety. For example, a larger primary navigation camera can be positioned away from the surgical site where the procedure is being performed, while multiple secondary or "mini" navigation markers can be placed inside the body where interference with the procedure is minimized. While the primary camera will often not be able to see the secondary navigation markers, the secondary camera can see and track the secondary markers, while the secondary camera itself can be tracked by the primary camera.

[0017] This approach can be useful in robotic spine surgery, where individual vertebrae are frequently misaligned during procedures such as the placement of pedicle screws on multiple vertebrae for fusion or other purposes. By placing secondary markers on at least some of the vertebrae, misalignments can be tracked and the robotic arm can be repositioned during the procedure.

[0018] Thus, provided herein are systems and methods for precise surgical navigation in robotic surgical systems, optionally for spinal surgery. In some embodiments, the precise navigation system is provided in the context of a multi-arm robotic surgical system comprising at least two robotic arms. In one such multi-arm robotic surgical system, at least one arm is involved in the surgical task and at least one arm carries at least one camera and is used to operate as part of the robotic navigation system. The at least two robotic arms are optimally mounted on a single chassis that houses a central controller that governs the movement of the robotic arms. In an alternative embodiment, a multi-arm robotic surgical system mounted on a single chassis can have at least three arms, at least two arms are involved in the surgical task and at least one arm carries at least one camera and is used to operate as part of the robotic navigation system. Those skilled in the art will understand that, for the present purposes of disclosing systems and methods for precise surgical navigation, it is also possible to envision a surgical robotic system in which multiple surgical arms are mounted on a single chassis and the navigation arm is brought into the surgical field on a separate cart or chassis, with communication and cooperation provided between the surgical arm chassis and the navigation chassis. Those skilled in the art will understand the advantages and disadvantages of this configuration compared to a robotic system based on a single chassis in which all arms, including the navigation arm, have a central controller. In an alternative configuration for use with the disclosed technology, multiple robotic arms (including the navigation arm) can be brought to the surgical field on individual carts, although this may have disadvantages when compared to a single-chassis design.

[0019] In some embodiments, one arm of a surgical robotic system may hold a conventional or "primary" navigation camera on a "director arm," and one or more additional arms of the surgical robotic system may hold a tool or end effector. These arms are often referred to as "working arms," ​​"acting arms," ​​or "tool arms." In accordance with the disclosed technology, a secondary camera, usually a smaller camera, may be attached to one or more of the other robotic arms, often with a tool or end effector also held by the robotic arm. The other arms will, of course, be deployed closer to the surgical site during the surgical procedure and therefore be able to get close enough to track secondary markers with minimal additional interference to the procedure.

[0020] In some embodiments, the smaller navigation camera may be held by a "dedicated" robotic arm that does not hold a surgical tool. This arrangement may be desirable when an additional arm is available, as it allows optimal positioning of the secondary navigation camera so that it can be positioned within the surgical field regardless of the placement or movement of surgical tools.

[0021] In some embodiments, secondary navigation markers or "mini" navigation markers may be placed directly on a portion of the patient's anatomy, such as on the patient's vertebrae, during a robotic spine surgical procedure. The markers may optionally incorporate radiopaque elements that would make them suitable for use in a conventional initial registration step in a robotic surgical procedure, although this is not necessary in embodiments in which a separate conventional registration step has already been performed. Miniature markers may also be deployed within systems that incorporate one or more separate registration markers. The miniature markers may be visible to a conventional navigation camera on the director arm, a smaller navigation camera on the end effector arm, or both cameras.

[0022] In some embodiments, small navigational markers may also be placed directly on tools or end effectors mounted on a robotic arm that primarily functions to perform surgical steps in a robotic surgical procedure. In certain embodiments, small navigational markers may be placed on a secondary navigation camera that is itself attached to the surgical arm, often to a tool or end effector on the surgical arm of a surgical robotic system. In this manner, the secondary camera can be optically tracked by the primary camera. In some embodiments, the secondary camera can be kinematically tracked based on the position of the supporting robotic arm, although kinematic tracking is less accurate and more difficult to implement.

[0023] In some embodiments, the secondary navigation camera is mounted on a surgical arm, typically on the tool / end effector, and is visible to a primary navigation camera held by a dedicated supervisory robotic arm or other robotic arm at a convenient distance (typically 0.5 m to 1.5 m) from the surgical field. This allows for an integrated approach, where the end effector-mounted secondary navigation camera can be positioned to have an optimal view of small secondary navigation markers placed on the patient anatomy. The primary navigation camera maintains an overall view of the surgical field, which importantly includes the secondary navigation camera.

[0024] The robotic controller is configured to kinematically coordinate the movements of all of the robotic arms (both director and working arms) relative to each other and the patient's anatomy, without necessarily requiring initial alignment of the coordinate systems of the miniature markers with the patient's anatomy. Those skilled in the art will understand that this coordination of the robotic arms and system navigation is also possible in embodiments where there is one director arm, one working arm (which does not have a miniature navigation camera mounted thereon), and one additional robotic arm (which holds a small navigation camera in proximity to the surgical field, as long as the small navigation camera has suitable miniature markers attached to it).

[0025] As a further advantage, the methods of the disclosed technology do not require prior alignment of the secondary markers, although prior alignment of the secondary markers can be performed in some circumstances. Alignment of the primary markers with the patient's preoperative computed tomography (CT) or other scan, in contrast, would typically still be performed. Eliminating the need to align the secondary markers is advantageous, for example, because the secondary markers are often placed during the procedure and therefore will not be in a fixed position relative to the preoperative scan. For example, in a procedure on a patient's spine, the secondary markers will often be placed only after the procedure has begun and the surgical site has been gradually opened.

[0026] However, the disclosed technology allows the secondary camera to "optically register" the secondary markers once they are implanted during the procedure. After each secondary marker is attached to an exposed bony or other anatomical structure, the primary camera scans the secondary marker, and the controller can "register" the optically determined marker positions within the surgical coordinate space. Because the primary markers would typically be conventionally registered with the preoperative images, the controller can then relate the secondary marker positions to the images. More importantly, the secondary camera will be able to track the secondary markers during the course of the procedure and determine how their relative positions may change, for example, as individual vertebrae twist relative to one another and change alignment.

[0027] In accordance with embodiments of the disclosed technology, larger conventional navigation markers may be placed on a patient's anatomy of interest, such as on the patient's bony anatomy, in particular examples, on the vertebrae of the patient's spine. The markers may have radiopaque elements and therefore may be used in a conventional registration step using a CT scan. This initial registration step serves to align the navigation components of the robotic system to the patient's anatomy, in representative examples, to the sides of the patient's bony anatomy, or specifically to the vertebrae of the patient's spine. Upon registration, the navigation system may then be aligned to the patient's bony anatomy and track that anatomy using, for example, a conventional navigation camera held by the director arm of the robotic system described herein, in embodiments of the disclosed technology.

[0028] In a similar embodiment, secondary navigation markers may be placed on a patient's anatomy of interest after the procedure has begun, often without prior registration. The anatomy of interest may be bone, skin, soft tissue, or, in the specific example provided, adjacent areas of the patient's spine. These secondary navigation markers are not connected to the (usually larger) primary registration marker and will often be outside the field of view of the primary navigation camera. The secondary markers may be registered to the anatomy of interest using a secondary navigation camera, typically held by the working robotic arm. The secondary navigation camera may be positioned to view the secondary navigation markers. While the primary navigation camera held on the director arm views both the primary navigation markers (used in the initial conventional registration) and the navigation markers on the secondary camera (or the arm holding the secondary camera), the secondary camera views the surgical field and secondary navigation markers, which often will not be visible to the primary navigation camera on the director arm. Therefore, the central controller of the robotic system can go through this navigation "loop" (a chain of navigational registrations) to register the secondary markers to the anatomical structure of interest.

[0029] In a first aspect of the disclosed technology, a method for performing a robotic surgical procedure includes providing a surgical robot with at least a first robotic arm, a second robotic arm, a first camera on the first robotic arm, and a second camera on the second robotic arm, and a controller configured to receive images from the first and second cameras and kinematically position the first and second robotic arms. A primary marker is placed at a primary location on a patient's anatomy, and the patient anatomy and the primary marker are scanned with the first camera to generate a primary image. The controller registers the location of the primary marker within a coordinate system of the surgical robot based on the primary image, and one or more secondary markers are placed at a secondary location(s) on the patient anatomy. The one or more secondary markers are scanned with the second camera to generate a secondary image, and the controller registers the secondary location(s) with respect to the primary location within the coordinate system of the surgical robot based on the secondary image.

[0030] In some cases, aligning the secondary location with respect to the primary location includes determining the location of the secondary camera with respect to the primary camera.

[0031] In some cases, determining the location of the secondary camera relative to the primary camera includes scanning the secondary camera with the primary camera.

[0032] In some cases, determining the location of the secondary camera relative to the primary camera includes the controller kinematically determining locations of the first robotic arm and the second robotic arm.

[0033] In some cases, the methods herein further include continuing to scan the primary marker with the first camera during a subsequent portion of the robotic surgical procedure to track the primary location within the coordinate system of the surgical robot.

[0034] In some cases, the methods herein further include continuing to scan the secondary marker with a second camera during a subsequent portion of the robotic surgical procedure to track the secondary location within the coordinate system of the surgical robot.

[0035] In some cases, tracking the secondary location within the coordinate system of the surgical robot includes tracking the position of a second camera relative to the first camera.

[0036] In some cases, the primary marker is larger than the secondary marker, and the primary camera is at a distance from the primary marker when scanning the primary marker that is greater than the distance of the secondary camera from the secondary marker when scanning the secondary marker.

[0037] In some cases, at least the first and robotic arms are mounted on a common chassis that establishes a coordinate system.

[0038] In some cases, the primary marker is fixed to a primary vertebra and the secondary marker is fixed to a secondary vertebra.

[0039] In some cases, the surgical tools are operated by one or more of the robotic arms.

[0040] In a second aspect of the disclosed technology, a method for performing a robotic surgical procedure includes providing a surgical robot with at least a first robotic arm, a second robotic arm, a first camera on the first robotic arm, a second camera on the second robotic arm, and a controller configured to receive images from the first camera and the second camera and kinematically position the first robotic arm and the second robotic arm within a surgical coordinate space, wherein the location of the first camera is kinematically tracked within the surgical coordinate space and the location of the second camera is optically tracked within the surgical coordinate space using the first camera. The location(s) in surgical coordinate space of one or more secondary markers fixed to the secondary location(s) on the patient anatomy are optically tracked using a second camera, and the controller calculates the locations in surgical coordinate space of the one or more secondary markers based on the kinematically tracked position of the first camera and the optically tracked locations of the secondary markers relative to the second camera.

[0041] In some cases, the secondary marker is within the field of view of the secondary camera but not within the field of view of the first camera.

[0042] In some cases, the method of the disclosed technology further includes tracking the location within surgical coordinate space of the primary marker fixed to a primary location on the patient's anatomy using a first camera positioned a first distance from the primary marker.

[0043] In some cases, a first distance between the first camera and the primary marker is greater than a second distance between the second camera and the secondary marker, and the primary marker is larger than the secondary marker.

[0044] In some cases, the first camera is positioned at a distance of up to 1.5 m from the patient anatomy, and the second camera is positioned at a distance of 30 cm or less than 30 cm from the target anatomy.

[0045] In some cases, the primary marker is 10 cm2 The secondary marker has a larger area and is 10 cm 2 having an area of ​​less than

[0046] In some cases, the method of the disclosed technology further includes continuing to scan the primary marker with the first camera during a subsequent portion of the robotic surgical procedure to track the primary location within the coordinate system of the surgical robot.

[0047] In some cases, the method of the disclosed technology further includes continuing to scan the secondary marker with a second camera during a subsequent portion of the robotic surgical procedure to track the secondary location within the coordinate system of the surgical robot.

[0048] In some cases, at least the first robotic arm and the robotic arm are mounted on a common chassis that establishes a coordinate system.

[0049] In some cases, the primary marker is fixed to a primary vertebra and the secondary marker is fixed to a secondary vertebra.

[0050] In some cases, the method of the disclosed technology further includes performing a procedure on at least one of the primary and secondary vertebrae using a surgical tool operated by one or more of the robotic arms.

[0051] In a third aspect of the disclosed technology, a robotic surgical system includes at least a first robotic arm, a second robotic arm, a first camera on the first robotic arm, and a second camera on the second robotic arm, and a controller configured to: (a) receive images from the first camera and the second camera; (b) kinematically position the first robotic arm and the second robotic arm in a surgical coordinate space; (c) kinematically track a position of the first camera in the surgical coordinate space; (d) optically track a position of the second camera using the first camera; and (e) calculate a position of a secondary marker within a field of view of the second camera based on the position of the second camera being tracked by the first camera.

[0052] In some cases, the robotic surgical system further includes a surgical tool deployed on the second robotic arm.

[0053] In some cases, the robotic surgical system further includes an optical marker mounted proximate to the second camera configured to enable optical tracking of the second camera by the first camera.

[0054] In some cases, the first camera is configured to be positioned at a distance of at least 1.5 m from the patient anatomy, and the second camera is configured to be positioned at a distance of 30 cm or less than 30 cm from the patient anatomy.

[0055] In some cases, the surgical robot includes at least a third robotic arm carrying a surgical tool, and the first robotic arm includes a supervisory robotic arm carrying only the first camera.

[0056] In some cases, the robotic surgical system further includes a third robotic arm that also carries a third camera having markers that enable optical tracking by the first camera.

[0057] In some cases, the first camera is configured to track a primary marker that is fixed to the patient anatomy.

[0058] In some cases, at least the first robotic arm and the second robotic arm are mounted on a single chassis that defines a surgical coordinate space.

[0059] In some cases, the robotic surgical system further includes a robotic arm and a controller all mounted on a single chassis.

[0060] In some embodiments, control of the surgical robot of the disclosed technology may also rely on "pose" information provided through the alignment step for planning and / or control. The disclosed technology provides improved placement and design of one or more navigation cameras, as discussed herein, and the methods and systems of the disclosed technology can be applied to a wide variety of different surgical robot architectures and designs.

[0061] Those skilled in the art will appreciate that the specific examples are given with reference to adjacent portions of a patient's bony anatomy (the spine). However, the present systems and methods can be used for adjacent portions of any anatomy of interest (bone, skin, soft tissue) because small navigational markers can be randomly placed without a separate registration requirement. Instead, they are registered to the anatomy of interest through the disclosed navigation loop.

[0062] In some embodiments, another type of navigation loop can be described. Similar to the previous one, only this time there are no markers on the miniature navigation camera. This time, the camera is positioned at a known, predefined location on one of the surgical arms, and the central controller can use that robot's known location in space to be calculated in the overall navigation / alignment loop.

[0063] (Incorporated by reference) Description of the Background Art WO2022 / 195460 is described above. Other co-owned publications and applications include PCT / IB2022 / 052297 (published as WO2022 / 195460), PCT / 2022 / 058988 (published as WO2023 / 067415), PCT / IB2022 / 058972 (published as WO2023 / 118984), PCT / IB2022 / 058982 (published as WO2023 / 118985), PCT / IB2022 / 058978 (published as WO2023 / 144602), PCT / IB2022 / 058980 (published as WO2023 / 152561), PCT / IB2023 / 055047 (published as WO2023 / 223215), PCT / IB2022 / 058988 (published as WO2023 / 237922), PCT / IB2023 / 055439, PCT / IB2023 / 056911, PCT / IB2023 / 055662, PCT / IB2023 / 055663, US 63 / 524,911, and US 63 / 532,753, the entire disclosures of which are incorporated herein by reference.

[0064] PCT Application No. PCT / __________________ (WSGR Ref: 67551-711.602; M&S Ref: P77437WO), entitled "SINGLE ORIGIN MARKER ASSEMBLIES AND METHODS FOR THEIR USE," and PCT Application No. PCT / __________________ (WSGR Ref: 67551-713.602; M&S Ref: P77440WO), entitled "INTEGRATED MULTI-ARM MOBILE MODULAR SURGICAL ROBOTIC SYSTEM," both of which were filed on the same day as the present application for the same applicant, the entire disclosures of which are incorporated herein by reference in their entireties.

[0065] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. [Brief explanation of the drawings]

[0066] The novelty of the disclosed technology is set forth with particularity in the appended claims. A better understanding of the features and advantages of the disclosed technology will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the disclosed technology are utilized, and the accompanying drawings.

[0067] [Figure 1] FIG. 1 illustrates a surgical robotic system with a primary navigation camera mounted on a robotic director arm and a secondary navigation camera also mounted on a robotic worker arm that carries a surgical tool, according to some embodiments.

[0068] [Figure 2]FIG. 2 illustrates a surgical robotic system with a primary navigation camera mounted on a primary robotic director arm and a secondary navigation camera mounted on a dedicated secondary robotic arm that carries only the secondary camera, according to some embodiments.

[0069] [Figure 3] FIG. 3 is a top plan view of an end effector with a secondary navigation camera attached to a gripper tool having a mounting flange that can be removably attached to the distal end of a surgical robotic arm, according to some embodiments.

[0070] [Figure 4A] 4A and 4B are isometric side views of an end effector having a mounting flange that can be removably attached to the distal end of a surgical robotic arm. The end effector carries a secondary navigation camera but does not have any surgical tools. FIG. 4B is rotated 90° about its longitudinal axis relative to FIG. 4A, according to some embodiments. [Figure 4B] 4A and 4B are isometric side views of an end effector having a mounting flange that can be removably attached to the distal end of a surgical robotic arm. The end effector carries a secondary navigation camera but does not have any surgical tools. FIG. 4B is rotated 90° about its longitudinal axis relative to FIG. 4A, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0071] (Detailed explanation) Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0072] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Any reference to "or" herein is intended to include "and / or" unless stated otherwise.

[0073] As used herein, the term "about" refers, in some instances, to an amount that is approximately the stated amount.

[0074] As used herein, the term "about" refers to an amount that is near 10%, 5%, or 1% of the stated amount, including increments therein.

[0075] As used herein, the term "about" in reference to a percentage refers to an amount that is 10%, 5%, or 1% greater or less than the stated percentage, including increments therein.

[0076] As used herein, the phrases "at least one," "one or more," and "and / or" are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, and / or C" means "A only," "B only," "C only," "A and B together," "A and C together," "B and C together," or "A, B, and C together."

[0077] The disclosed systems and methods will now be described with particular reference to the accompanying drawings. Those skilled in the art will recognize that the described embodiments are exemplary in nature and that reasonable departures from the described embodiments are possible while still remaining within the scope of the disclosed technology.

[0078] 1 , according to some embodiments, an exemplary surgical robotic system 100 includes a primary or supervisory navigation camera 102 mounted on the distal end of a supervisory robotic arm 104. The proximal end of the supervisory arm 104 may be mounted on a robotic platform (not shown) that comprises, consists of, or consists essentially of a single cart or chassis, as described in commonly owned PCT Application No. PCT / IB2022 / 052297 (published as WO2022 / 195460), the entire disclosure of which is incorporated herein by reference.

[0079] The director arm 104 can be configured to hold the primary camera 102 at a conventional distance from the surgical site 106, typically within the range of 1.5 m to 2.5 m for a robotic spine procedure as shown, so that the field of view FOV1 of the primary navigation camera 102 can include most or all of the robotic components as well as most or all of the surgical site. In some embodiments, the field of view FOV1 of the primary navigation camera 102 can track a primary navigation marker 110, which is typically used for initial registration of the robot coordinates to the patient anatomy and / or an initial preoperative patient CT or other scan. In some embodiments, the primary navigation marker is typically relatively large, e.g., 10 cm. 2 It will have a larger area.

[0080] 1, the robotic system 100 includes at least one "working" robotic arm 120 having a distal end 122 supporting an end effector 124 with a gripper 126 that carries a surgical tool, such as a paddle 128. The gripper 126 can carry various other surgical tools, such as a cannula for implanting pedicle screws, as described in commonly owned PCT Publication No. WO 2023 / 223215, the entire disclosure of which is incorporated herein by reference.

[0081] Those skilled in the art will understand that although a single working surgical arm 120 is shown, two, three, four, or even more working arms can be incorporated into the robotic systems of the disclosed technology.

[0082] As described in the detailed description above, robotic systems are generally as described in PCT Publication Nos. WO 2022 / 195460 and WO 2023 / 223215 (the entire disclosures of which are previously incorporated by reference). However, the robotic systems of the disclosed technology differ in that they are configured to deploy a secondary navigation camera intended to be positioned much closer to the surgical site 106, typically on the order of 30 cm or less. The secondary navigation camera may be mounted on a “dedicated” robotic arm or on a working arm that also carries and deploys various surgical tools. In some cases, the secondary camera will be an “add-on” device that can be removably secured to the working robotic arm or other robotic arms and / or surgical tools or end effectors. A first example in which the secondary navigation camera is mounted on a working surgical arm is shown in FIG. 1 , and a second example in which the secondary navigation camera is mounted on a dedicated secondary director arm is shown in FIG. 2 .

[0083] As shown in FIG. 1 , the secondary navigation camera 130 may be removably or fixedly (typically removable) mounted at or near the distal end 122 of the working robotic arm 120. The secondary navigation camera will be smaller than the primary navigation camera, having a maximum dimension of 10 cm or less, and will be configured to focus on a narrower field of view FOV2 at closer distances. In a specific case, the secondary navigation camera 130 has one or more “camera” navigation markers 132 mounted thereon, which may be small, typically having a maximum dimension of 5 cm or less. Such small dimensions may allow the secondary navigation camera 130 to which the navigation markers 132 are attached to be manipulated by the working robotic arm 120 with minimal interference with the surgical procedure, for example, reducing or eliminating interference with the operation of the working arm 120 and / or the operator's view of the surgical site 106.

[0084] 1 also illustrates the placement of additional "secondary" navigational markers for use in performing the robotic surgical procedure. Primary navigational marker 110 may be affixed to the first vertebra V1 prior to the procedure so that it can be used in conventional registration of the patient anatomy. Secondary navigational markers 140 and 142 may be deployed on vertebrae V2 and V3, respectively. While these navigational markers may be implanted prior to the procedure and used in registration, more typically they will be deployed during the procedure, for example, after the patient's spine has been sequentially exposed using conventional bone attachment modalities.

[0085] Secondary markers 140 and 142 (and additional secondary markers, if desired) may be "miniaturized" to have a maximum dimension of 2 cm or less and located inside the patient's body, for example, at a location below the patient's skin surface adjacent a surgical incision exposing the spine. Because markers 140 and 142 are effectively "buried" just below the skin surface, the secondary navigation markers may be invisible to primary navigation camera 102, in contrast to the larger primary navigation marker 110, which extends above the patient's skin.

[0086] 1 , camera navigation markers 132 deployed on the secondary navigation camera 130 are visible to the primary supervision camera 102 and may enable the controller to optically track the position of the secondary navigation camera in and around the surgical site 106. The position of the secondary navigation camera 130 can be kinematically tracked by the controller based on the kinematics of the working robotic arm 120, although such kinematic tracking may be less preferred because it is more difficult to implement to achieve the desired accuracy.

[0087] Information from primary supervising navigation camera 102 and secondary navigation camera 132 can be used to optically align and subsequently track the position of primary navigation marker 110 and first and second secondary navigation markers 140 and 142 (as well as any additional secondary navigation markers that may be deployed later). Primary navigation marker 110 may be optically tracked by primary supervising navigation camera 102, and its position kinematically tracked by a controller as described in WO 2022 / 195460 and WO 2023 / 223215 (the entire disclosures of which are previously incorporated by reference herein). Simultaneously, or substantially simultaneously, primary supervising navigation camera 102 can track the position of secondary navigation camera 130 based on observing camera navigation marker 132, which may be within the field of view FOV1 of camera 102.

[0088] The positions of the secondary navigation markers 140 and 142 can in turn be tracked by the secondary navigation camera 130, and the controller can calculate the positions of the markers 140 and 142 in the spaced surgical coordinates based on the position of the camera 130 in the spaced surgical coordinates. In this manner, the controller can both register and track the positions of the secondary markers in the surgical coordinate space without having to rely on kinematically calculated "pose" information, although the use of pose information is not excluded from the disclosed technology.

[0089] 2 illustrates an alternative robotic system 200 constructed in accordance with the principles of the disclosed technology, according to some embodiments. A primary navigation camera 202 and a primary director arm 204 may be arranged to view a primary navigational marker 210 within a surgical site 206, generally as described for similar components in robotic system 100. Robotic system 200 differs in that a secondary navigation camera 230 is primarily mounted on a dedicated secondary director arm 234, rather than on a working robotic arm 220, which carries only an end effector 24, a gripper 226, and a surgical tool 228 on the arm's distal end 220. Mounting the secondary navigation camera 230 on the dedicated robotic arm 234 may be advantageous in that the secondary camera 230 can be positioned to view secondary navigational markers 240 and 242 at all times, regardless of the positioning of the surgical tool 228 mounted on the working arm 220. The camera navigation marker 232 is shown mounted on the secondary navigation camera 230, but can be located anywhere on the arm 234, so long as the marker movement is representative of the camera movement.

[0090] 3, 4A, and 4B, mounting of a secondary navigation camera on a tool and a robotic end effector is shown, according to some embodiments. As shown in FIG. 3, end effector 300 includes a gripper tool 302 and a mounting flange 304. A secondary navigation camera 310 may be mounted on the free end of gripper tool 302 opposite the mounting flange. The dimensions of secondary navigation camera 310 are minimized to reduce the chance of interfering with one of the surgical procedures, e.g., the operation of the robotic arm or the surgical staff's view of the surgical field, and may typically have a maximum dimension of 10 cm or less.

[0091] 4A and 4B show an alternative secondary camera mounting embodiment including an end effector 400 configured to mount to the free end of a robotic surgical arm and a flange 402. A secondary navigation camera 404 may be attached to the end effector 400 by a bracket 406. The bracket 406 of FIGS. 4A and 4B may be configured to mount to the robotic arm in place of another tool, end effector, gripper, or the like. In this manner, secondary camera(s) may be positioned on a multi-armed surgical robot in different arrangements in accordance with the disclosed technology. In other cases, the bracket 406 may be configured to mount to a dedicated navigation or supervision arm that would typically be devoid of other tools, end effectors, grippers, and the like, as shown, for example, in FIG. 2 .

[0092] In addition to the robotic system just described, the disclosed technology also provides a navigation method for performing robotic surgical procedures. In some embodiments, the method may employ a surgical robot having one robotic arm with a dedicated supervision or navigation camera mounted thereon, the distal end of which may be positioned at a convenient distance, typically 1.5 m to 2.5 m, from the surgical site on the patient. The surgical robot may additionally have a surgical arm configured with an end effector capable of holding surgical tools and performing the robotic procedure at the patient's surgical site. In some embodiments, the surgical arm may also carry a secondary navigation camera, typically a small secondary navigation camera with a diameter of up to 10 cm or less. Such a surgical robot is shown and described with reference to FIG. 1 herein.

[0093] The small secondary navigation camera can be attached to the distal end or region of the surgical arm, e.g., the end effector, and positioned to have a direct view of the surgical site. The method can further include embedding multiple navigation markers, e.g., a relatively large navigation marker that can be reused to perform the initial alignment step, as well as one, two, three, or more secondary navigation markers. For example, in a spinal procedure, two, three, or more secondary markers can be used. Such multiple secondary cameras will typically be visible to one or more secondary cameras even when invisible to the primary supervision camera. Even when invisible to the primary navigation camera, the secondary navigation marker can be visible to a smaller secondary navigation camera, which can be moved and positioned within the surgical site to view the secondary marker.

[0094] In some embodiments, in spine procedures, secondary markers are typically placed on the patient's vertebrae so as to be visible to the secondary navigation camera, and often may be located within the surgical cavity blocking view by the primary navigation camera. Primary navigation markers are typically larger than the secondary markers so as to be visible to the primary supervising navigation camera, and may be placed on adjacent vertebrae and extend out of the surgical cavity, with one navigation marker being placed on the secondary navigation camera, for example, attached to an end effector on a surgical arm.

[0095] In this manner, the primary supervising navigation camera can view navigation markers placed on the secondary navigation camera on the surgical arm, and can also view "anatomical" primary navigation markers that extend outside the surgical site. The robotic controller mediates the movement of the supervising arm and surgical arm, and can do so using the navigation information provided, rather than using the pose information typically provided by the initial alignment step and subsequent kinematic tracking of the robotic arm. Optical tracking can continue as the primary supervising navigation camera maintains a simultaneous view of the primary navigation markers and the markers on the secondary navigation camera.

[0096] In some embodiments, the surgical robotic system has a dedicated secondary director robotic arm with a secondary navigation camera mounted thereon in addition to the primary navigation camera on the primary director arm. The working surgical arm configured with an end effector can hold and operate a surgical tool at a patient's surgical site. The secondary navigation camera on the dedicated secondary director arm can be independently moved in proximity to the patient's surgical site to obtain an advantageous view of the surgical tool and secondary markers, regardless of the positioning and operation of the surgical tool.

[0097] In some embodiments, the alignment step can be performed at the beginning of the spinal surgical procedure. In some embodiments, the navigational markers, which are mounted to the vertebrae but configured to extend outward from the surgical site, can incorporate radiopaque components that are visible to CT or X-ray.

[0098] In some embodiments, a registration step can be performed such that the coordinate system of the robotic system is aligned with the patient's anatomy. When such a registration step is performed, the use of orientation information for navigation is also possible during the surgical procedure. However, as discussed herein, those skilled in the art will understand that such orientation information is not required. For example, the supervisory navigation camera can maintain a simultaneous view of navigational markers attached to the patient anatomy and extending from the surgical site and navigational markers mounted on a miniature navigation camera mounted on the end effector, allowing the miniature navigation camera to view anatomical elements adjacent to the patient anatomy on which navigational markers extending from the patient anatomy visible to the supervisory navigation camera are mounted.

[0099] Those skilled in the art will understand that variations on the described embodiments are possible while still remaining within the spirit of the disclosed technology. For example, the disclosed systems and methods can be deployed in conjunction with surgical robotic systems having an overseer arm and multiple surgical arms. Additionally, the disclosed systems and methods can be deployed in surgical fields other than spinal surgery. Any robotic surgical field that would benefit from collaborative navigation without the need for alignment and precise navigation that does not interfere with surgeon workflow or line of sight would benefit from application of the presently disclosed systems and methods. While embodiments of the disclosed technology have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, modifications, and substitutions will now occur to those skilled in the art without departing from the scope of the invention. It should be understood that various alternatives to the embodiments described herein may be employed in practicing the invention. The following claims define the scope of the technology, and methods and structures within the scope of these claims and their equivalents are intended to be covered thereby.

Claims

1. 1. A method for performing a robotic surgical procedure, the method comprising: providing a surgical robot with at least a first robotic arm, a second robotic arm, a first camera or other sensor on the first robotic arm, a second camera or other sensor on the second robotic arm, and a controller configured to receive images from the first camera and the second camera or other sensor and to kinematically position the first robotic arm and the second robotic arm; placing a primary marker at a primary location on the patient's anatomy; scanning the patient anatomy and the primary marker with the first camera or other sensor to generate a primary image, wherein the controller registers the location of the primary marker within a coordinate system of the surgical robot based on the primary image; placing one or more secondary markers on the patient anatomy at secondary location(s); scanning the one or more secondary markers with the second camera or other sensor to generate a secondary image, wherein the controller registers the secondary location with respect to the primary location in the coordinate system of the surgical robot based on the secondary image; A method comprising:

2. The method of claim 1 , wherein registering the secondary location relative to the primary location includes determining a location of the secondary camera or other sensor relative to the primary camera or other sensor.

3. The method of claim 2 , wherein determining the location of the secondary camera or other sensor relative to the primary camera or other sensor comprises scanning the secondary camera or other sensor with the primary camera or other sensor.

4. 3. The method of claim 2, wherein determining a location of the secondary camera or other sensor relative to the primary camera or other sensor includes the controller kinematically determining locations of the first robotic arm and the second robotic arm.

5. 5. The method of claim 1, further comprising continuing to scan the primary marker with the first camera or other sensor during a subsequent portion of the robotic surgical procedure to track the primary location within the coordinate system of the surgical robot.

6. 6. The method of claim 1, further comprising continuing to scan the secondary marker(s) with the second camera or other sensor during a subsequent portion of the robotic surgical procedure to track the secondary location(s) within the coordinate system of the surgical robot.

7. 2. The method of claim 1, wherein tracking the secondary location(s) in the coordinate system of the surgical robot includes tracking a position of the second camera or other sensor relative to the first camera or other sensor.

8. 8. A method according to any one of claims 1 to 7, wherein the primary marker is larger than the secondary marker, and the primary camera or other sensor is at a distance from the primary marker when scanning the primary marker that is greater than a distance from the secondary marker(s) to the secondary camera or other sensor when scanning the secondary marker(s).

9. The method of any one of claims 1-8, wherein the at least the first robotic arm and the robotic arm are mounted on a common chassis that establishes the coordinate system.

10. The method of any one of claims 1-9, wherein the primary markers are fixed to primary vertebrae and the secondary markers are fixed to secondary vertebrae.

11. The method of any one of claims 1-10, wherein the surgical tool is operated by one or more of the robotic arms.

12. 1. A method for performing a robotic surgical procedure, the method comprising: providing a surgical robot with at least a first robotic arm, a second robotic arm, a first camera or other sensor on the first robotic arm, a second camera or other sensor on the second robotic arm, and a controller configured to receive images from the first camera and the second camera or other sensor and to kinematically position the first robotic arm and the second robotic arm in a surgical coordinate space; kinematically tracking the location of the first camera or other sensor within the surgical coordinate space; optically tracking the location of the second camera or other sensor within the surgical coordinate space using the first camera or other sensor; using the second camera or other sensor to optically track location(s) within the surgical coordinate space of one or more secondary markers fixed to secondary location(s) on the patient anatomy; Including, The method wherein the controller calculates the location(s) of the one or more secondary markers in the surgical coordinate space based on the kinematically tracked position of the first camera or other sensor and the optically tracked location of the secondary marker relative to the second camera or other sensor.

13. The method of claim 12 , wherein the secondary marker is within the field of view of the secondary camera or other sensor but not within the field of view of the first camera or other sensor.

14. 14. The method of claim 12 or 13, further comprising tracking a location in the surgical coordinate space of a primary marker fixed to a primary location on the patient's anatomy using the first camera or other sensor located a first distance from the primary marker.

15. 15. The method of claim 14, wherein a first distance between the first camera or other sensor and the primary marker is greater than a second distance between the second camera and the secondary marker, and the primary marker is larger than the secondary marker.

16. 16. The method of claim 15, wherein the first camera or other sensor is located at a distance of up to 1.5 m from the patient anatomy and the second camera or other sensor is positioned at a distance of 30 cm or less than 30 cm from the target anatomy.

17. The primary marker is 10 cm ２ The secondary marker has a larger area than the 10 cm ２ 17. The method of claim 15 or 16, wherein the surface area is less than

18. 18. The method of any one of claims 14-17, further comprising continuing to scan the primary marker with the first camera or other sensor during a subsequent portion of the robotic surgical procedure to track the primary location within a coordinate system of the surgical robot.

19. 20. The method of claim 18, further comprising continuing to scan the secondary marker(s) with the second camera or other sensor during a subsequent portion of the robotic surgical procedure to track the secondary location(s) within the coordinate system of the surgical robot.

20. The method of any one of claims 12-19, wherein the at least first robotic arm and the robotic arm are mounted on a common chassis that establishes the coordinate system.

21. The method of any one of claims 13-20, wherein the primary marker is fixed to a primary vertebra and the secondary marker is fixed to a secondary vertebra.

22. 22. The method of any one of claims 13-21, further comprising performing a procedure on at least one of the primary and secondary vertebrae using a surgical tool operated by one or more of the robotic arms.

23. 1. A robotic surgical system, comprising: a surgical robot comprising at least a first robotic arm, a second robotic arm, a first camera or other sensor on the first robotic arm, and a second camera or other sensor on the second robotic arm; a controller configured to: (a) receive images from the first camera and the second camera or other sensor; (b) kinematically position the first robotic arm and the second robotic arm in a surgical coordinate space; (c) kinematically track a position of the first camera or other sensor in the surgical coordinate space; (d) optically track a position of the second camera or other sensor using the first camera or other sensor; and (e) calculate a position of a secondary marker within a field of view of the second camera or other sensor based on the position of the second camera or other sensor being tracked by the first camera or other sensor; A robotic surgical system comprising:

24. 24. The robotic surgical system of claim 23, further comprising a surgical tool deployed on the second robotic arm.

25. 25. The robotic surgical system of claim 23 or 24, further comprising an optical marker mounted proximate to the second camera or other sensor configured to enable optical tracking of the second camera or other sensor by the first camera or other sensor.

26. 26. The robotic surgical system of claim 23, wherein the first camera or other sensor is configured to be positioned at a distance of at least 1.5 m from the patient anatomy and the second camera or other sensor is configured to be positioned at a distance of 30 cm or less than 30 cm from the patient anatomy.

27. 27. The robotic surgical system of any one of claims 23-26, wherein the surgical robot includes at least a third robotic arm carrying a surgical tool, and the first robotic arm includes a supervisory robotic arm carrying only the first camera or other sensor.

28. 30. The robotic surgical system of claim 27, wherein the third robotic arm also carries a third camera or other sensor having markers that enable optical tracking by the first camera or other sensor.

29. The robotic surgical system of any one of claims 23-28, wherein the first camera or other sensor is configured to track a primary marker fixed to the patient anatomy.

30. 30. The robotic surgical system of any one of claims 23-29, wherein at least the first robotic arm and the second robotic arm are mounted on a single chassis that defines the surgical coordinate space.

31. The robotic surgical system of claim 30, wherein all robotic arms and the controller are mounted on the single chassis.

Images