Method and system for tracking plurality of optical markers in robotic surgical procedure

Abstract

A robotic surgical system includes a first robotic arm, a second robotic arm, a first camera on the first robotic arm, and a second camera on the second robotic arm. The controller receives images from the first camera and the second camera, kinematically locates the first robotic arm and the second robotic arm in a surgical coordinate space, kinematically tracks a position of the first camera in the surgical coordinate space, and optically tracks a position of the second camera using the first camera. A position of the auxiliary marker in a field of view of the second camera is calculated based on a position of the second camera tracked by the first camera.

Description

[0001] Cross-references

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 442,457, filed January 31, 2023, which is incorporated herein by reference. Background Art

[0003] field

[0004] The disclosed technology relates to systems and methods for surgical robotic registration and navigation. More particularly, the disclosed technology relates to systems and methods using navigation markers that can be attached to bones and other patient anatomy.

[0005] Surgical robotic systems and other robotic systems often utilize cameras or sensors to track objects in a robotic space around a surgical robot. In some surgical robotic procedures, radiopaque (RO) markers can be attached to the patient's bony anatomy or other anatomical structures, and the patient can be imaged by a computed tomography (CT) scan, and the marker positions are used to "register" the patient's image in the robotic surgical space. For example, as taught in co-owned PCT application PCT / IB2022 / 052297 (published as WO2022 / 195460) (the entire disclosure of which is incorporated herein by reference), one or more RO markers shown in a patient's CT scan can be screened by a camera located on the arm of a multi-arm surgical robot to establish an initial position of the marker in a robotic surgical space having a coordinate system defined relative to a robot chassis or cart. During subsequent surgical procedures, changes in the patient's position can be calculated based on observed changes in the marker positions over time.

[0006] The surgical robot described in PCT Publication WO2022 / 195460 and other commonly used surgical robots typically rely on a single camera and a single marker to track the patient's position during robotic surgery. While feasible, using a single marker and a single camera requires a relatively large marker because the camera is not always close enough to the marker or properly aligned with the marker to allow the marker's size to be reduced. In other words, the marker target needs to be large to allow the camera to accurately determine its position from a distance.

[0007] In currently available robotic surgical systems, when the camera is placed at a standard position of 1.5 to 2.5 meters from the patient and patient markers, navigation markers typically measure at least 7 to 15 cm across to provide the necessary 1 to 2 mm accuracy at the tool tip. Such large markers are disadvantageous because they can interfere with the surgeon's view of and access to the surgical site, and they can easily deflect under their own weight, resulting in reduced accuracy.

[0008] The use of registration markers as the first step in a robotic surgery procedure is well known. This step is performed to align the coordinate system of a robotic surgical system (such as a spinal surgery robotic system) with the patient's anatomy. Navigation markers can then be positioned in place of the registration markers to synchronize the robotic navigation system with the patient's anatomy.

[0009] While generally successful, the use of both registration and navigation markers does have some drawbacks. First, duplication increases the time, cost, and complexity of the procedure. Second, performing a CT scan for registration exposes the patient to radiation.

[0010] It would therefore be desirable to provide improved robotic surgical systems and methods. In particular, it would be desirable to provide robotic surgical systems and methods that do not require an initial registration step for robotic navigation or for any other purpose. It would be further desirable to provide surgical robotic systems and methods that allow for the observation and tracking of relatively small markers that do not interfere with the surgical procedure without causing any substantial loss of accuracy or precision. It would be still further desirable to provide surgical robotic systems and methods that allow for the placement of multiple navigation markers even after the surgical procedure has begun. The invention described and claimed herein will meet at least some of these objectives. Summary of the Invention

[0011] The disclosed systems and methods address the above-mentioned deficiencies. By employing multiple repositionable navigation cameras and / or other sensors and multiple navigation markers fixed at different locations on the patient, smaller navigation markers can be used while maintaining accurate scanning and / or tracking of the patient's anatomy. In some embodiments, multiple robotic arms can be operated in a surgical field, wherein at least one arm holds a navigation camera or other sensor and at least one other arm holds a tool or end effector, wherein the arms are manipulated by a controller of the robotic system. Such systems and methods are useful in orthopedic surgery, wherein markers are fixed to the patient's bony anatomy, such as individual vertebrae of the patient's spine.

[0012] While the disclosed technology will find particular use with optical cameras, the principles of the technology can be applied to any sensing technology and are particularly useful for sensing technologies that are limited to line-of-sight visibility and / or proximity between the sensor and the marker. Suitable sensing technologies include laser scanning or tracking, such as light detection and ranging sensors (LIDAR); magnetic sensing, scanning, and tracking; ultrasonic sensing, scanning, and tracking; and the like.

[0013] In some embodiments, placing multiple "micro" markers on a patient's bony anatomy will allow a small navigation camera (also referred to herein as an auxiliary camera), mounted on a surgical robotic arm, typically in conjunction with a tool or end effector, to access and track areas of the surgical field that are inaccessible to a larger navigation camera (also referred to herein as a primary camera), mounted on a dedicated monitoring arm. Thus, systems and methods are presented for affixing multiple micro markers (sometimes referred to herein as auxiliary markers) on a patient's bony anatomy, wherein the micro markers can be tracked by one or more small navigation cameras mounted on a robotic arm that carries the tool and / or end effector used to perform a procedure.

[0014] In some embodiments, the small or auxiliary cameras can be removably mounted on the surgical robot arm (e.g., they can be add-ons), while the larger or primary camera can be attached to a dedicated monitoring arm. The larger or primary camera can be configured to track an attached larger or primary navigation marker, providing a monitoring view of most or all of the surgical field. Additionally, the primary camera can track the auxiliary camera, such that the position of the auxiliary marker in the surgical field can be tracked by kinematically tracking the position of the primary camera (based on the kinematics of the monitoring arm) and optically tracking the position of the auxiliary camera using the primary camera. All movement, tracking, and calculations can be performed by the robotic controller.

[0015] In a surgical robotic system according to the disclosed technology, multiple surgical robotic arms can be mounted on a single chassis (typically a single mobile chassis or cart). The phrase "single chassis" means that the chassis provides a single rigid platform when present below the operating table, which in turn provides a single surgical coordinate space. For example, a "single chassis" may include two, three or more mobile components or other components, subassemblies, etc., which can be joined in situ below the table to form a single chassis according to the disclosed technology. In other examples, such individual components, subassemblies, etc. can be pre-assembled at the surgical site or elsewhere before being moved to a position below the operating table. Although such a single chassis will typically have a unitary construction, in other cases, the platform may include two, three or more component structures that are assembled in situ at the surgical site.

[0016] Multiple surgical robotic arms can carry and deploy various surgical tools, end effectors, navigation cameras, etc., as well as a robotic controller. The system can include a display and user interface mounted on or in a single chassis. The controller can automatically control the movement of some or all surgical robotic arms, monitoring arms, and other robotic system components based on information provided by the primary and secondary navigation cameras. In some embodiments, the controller can display images from the cameras, allowing the surgeon to manually control some or all surgical tools or end effectors.

[0017] The disclosed robotic system is advantageous because multiple navigation cameras do not interfere with the surgeon's view and workflow and can be optimally positioned to ensure patient safety. For example, a larger primary navigation camera can be positioned away from the surgical site where the procedure is being performed, while multiple auxiliary or "micro" navigation markers can be placed within the body with minimal disruption to the procedure. While the primary camera would typically be unable to view the auxiliary navigation markers, the auxiliary cameras can view and track the auxiliary markers, while the auxiliary cameras themselves can be tracked by the primary camera.

[0018] This approach may be useful in robotic spinal surgery, where individual vertebrae are often misaligned during surgery, such as placing pedicle screws on multiple vertebrae for fusion or other purposes. By placing auxiliary markers on at least a portion of the vertebrae, the misalignment can be tracked and the robotic arm repositioned during surgery.

[0019] Thus, systems and methods for accurate surgical navigation in a robotic surgical system, optionally a robotic system for spinal surgery, are provided herein. In some embodiments, an accurate navigation system is provided in the context of a multi-arm surgical robotic system comprising at least two robotic arms. In one such multi-arm surgical robotic system, at least one arm is responsible for the surgical task, and at least one arm is used to carry and operate at least one camera as part of the robotic navigation system. The at least two robotic arms are optimally mounted on a single chassis that houses a central controller that controls the movement of the robotic arms. In alternative embodiments, the multi-arm surgical robotic system mounted on a single chassis may have at least three arms, at least two of which are responsible for the surgical task, and at least one of which is used to carry and operate at least one camera as part of the robotic navigation system. Those skilled in the art will appreciate that for the purposes of the present disclosure of systems and methods for accurate surgical navigation, a surgical robotic system is also contemplated in which multiple surgical arms are mounted on a single chassis, and in which the navigation arms are brought to the surgical site on a separate cart or chassis, with communication and coordination provided between the surgical arm chassis and the navigation chassis. Those skilled in the art will appreciate the advantages and disadvantages of this configuration compared to a robotic system in which all arms, including the navigation arm, are based on a single chassis with a central controller. Multiple robotic arms (including the navigation arm) can be brought to the surgical site in alternative configurations on a single cart for use with the disclosed technology, but this may have disadvantages when compared to a single chassis design.

[0020] In some embodiments, one arm of a surgical robotic system may hold a conventional or "primary" navigation camera on a "monitoring arm," and one or more additional arms of the surgical robotic system may hold tools or end effectors. These arms are often referred to as "working arms," ​​"manipulator arms," ​​or "tool arms." In accordance with the disclosed technology, auxiliary cameras, typically smaller, may be attached to one or more of the other robotic arms, typically along with tools or end effectors also held by the robotic arms. The other arms will naturally be deployed closer to the surgical site during the surgical procedure, and thus will be able to get close enough to track auxiliary markers with minimal additional disruption to the procedure.

[0021] In some embodiments, the smaller navigation camera can be held by a "dedicated" robotic arm that does not hold surgical tools. This arrangement can be desirable when additional arms are available because it allows for optimal positioning of the auxiliary navigation camera as it can be positioned in the surgical field without regard for placement or manipulation of surgical tools.

[0022] In some embodiments, auxiliary or "micro" navigation markers can be placed directly on portions of a patient's anatomy, such as on a patient's vertebrae during a robotic spinal surgery procedure. The markers can optionally contain radiopaque elements, which would make them suitable for use in conventional initial registration steps in robotic surgery procedures, but this is not required in embodiments in which a separate conventional registration step has already been performed. The micro-markers can also be deployed in a system that contains one or more separate registration markers. The micro-markers can be visible to a conventional navigation camera on a monitoring arm or a smaller navigation camera on an end effector arm, or both.

[0023] In some embodiments, small navigation markers can also be placed directly on a tool or end effector mounted on a robotic arm that is primarily used to perform surgical steps in a robotic surgical procedure. In certain embodiments, small navigation markers can be placed on a secondary navigation camera that is itself attached to a surgical arm, typically attached to a tool or end effector on the surgical arm of a surgical robotic system. In this way, the secondary camera can be optically tracked by the primary camera. In some embodiments, the secondary camera can be kinematically tracked based on the position of the supporting robotic arm, but kinematic tracking can be less accurate and more difficult to implement.

[0024] In some embodiments, the secondary navigation camera is attached to a surgical arm, typically to the tool / end effector, and is visible to a primary navigation camera held by a dedicated monitoring robotic arm or other robotic arm at a convenient distance from the surgical site (typically 0.5m to 1.5m). This allows for an integrated approach in which the secondary navigation camera mounted on the end effector can be positioned to have the best view of the miniature secondary navigation markers placed on the patient's anatomy. The primary navigation camera maintains an overall view of the surgical site, which importantly includes the secondary navigation camera.

[0025] The robotic controller is configured to kinematically coordinate the movement of all robotic arms (monitoring and working arms) relative to each other and the patient's anatomy without necessarily requiring initial registration of the coordinate system of the micro-markers with the patient's anatomy. One skilled in the art will appreciate that such coordination of the robotic arms with system navigation is also possible in embodiments in which there is a monitoring arm, a manipulator arm (without a micro-navigation camera mounted thereon), and an additional robotic arm that holds the micro-navigation camera close to the surgical site, so long as the micro-navigation camera has appropriate micro-markers attached thereto.

[0026] As a further advantage, the method of the disclosed technology does not require conventional registration of auxiliary markers, although conventional registration of auxiliary markers can be performed in some cases. In contrast, registration of primary markers is still typically performed using the patient's preoperative computed tomography (CT) scan or other scan. For example, eliminating the need to register auxiliary markers is advantageous because auxiliary markers are typically placed during surgery and therefore will not be in place during preoperative scans. For example, in surgery on a patient's spine, auxiliary markers are typically placed only after the surgery has begun and the surgical site has been gradually opened.

[0027] However, the disclosed technology allows the auxiliary camera to "optically register" the auxiliary markers as they are implanted during surgery. After attaching each auxiliary marker to exposed bone or other anatomical structure, the primary camera scans the auxiliary markers, and the controller can "register" the optically determined marker positions in the surgical coordinate space. Since the primary markers will typically already be conventionally registered with the preoperative image, the controller can then associate the auxiliary marker positions with the image. More importantly, the auxiliary camera will be able to track the auxiliary markers during the course of the procedure to determine how their relative positions may change, for example, as individual vertebrae are torqued relative to each other and alter alignment.

[0028] In accordance with an embodiment of the disclosed technology, a large, conventional navigation marker may be placed on an anatomical structure of interest to a patient (e.g., on the patient's bony anatomy, and in a specific example, on a vertebra of the patient's spine). The marker may have a radiopaque element and, therefore, may be used in a conventional registration step using a CT scan. This initial registration step is used to register the navigation components of the robotic system with the patient's anatomical structure, in a representative example, with an aspect of the patient's bony anatomy, or, specifically, with the vertebrae of the patient's spine. After registration, the navigation system is then registered with the patient's bony anatomy and may track the anatomical structure using, for example, a conventional navigation camera held by a monitoring arm of the robotic system as described herein in an embodiment of the disclosed technology.

[0029] In a similar embodiment, auxiliary navigation markers can be placed on the patient's anatomical structures of interest without prior registration, and typically after the procedure has begun. The anatomical structures of interest can be bone, skin, soft tissue, or, in the specific example provided, adjacent areas of the patient's spine. These auxiliary navigation markers are not connected to the primary (typically larger) registration markers and will typically be located outside the field of view of the primary navigation camera. The auxiliary markers can be registered with the anatomical structures of interest using an auxiliary navigation camera, typically held by a working robotic arm. The auxiliary navigation camera can be positioned to observe the auxiliary navigation markers. The primary navigation camera held on the monitoring arm observes the primary navigation markers (for initial routine registration) and the navigation markers on the auxiliary camera (or arm holding the auxiliary camera), while the auxiliary camera observes the surgical field and the auxiliary navigation markers, which will typically not be visible to the primary navigation camera on the monitoring arm. Thus, the central controller of the robotic system can register the auxiliary markers with the anatomical structures of interest through this navigation "loop" (navigation registration chain).

[0030] In a first aspect of the disclosed technology, a method for performing a robotic surgical procedure includes providing a surgical robot having at least a first robotic arm, a second robotic arm, a first camera on the first robotic arm, a second camera on the second robotic arm, and a controller configured to receive images from the first camera and the second camera and kinematically position the first robotic arm and the second robotic arm. A primary marker is placed at a primary location on a patient's anatomy, and the patient's anatomy and the primary marker are scanned with the first camera to generate a primary image. The controller registers a position of the primary marker within a coordinate system of the surgical robot based on the primary image, and places one or more auxiliary markers at auxiliary locations on the patient's anatomy. The one or more auxiliary markers are scanned with the second camera to generate an auxiliary image, and the controller registers the auxiliary locations relative to the primary locations within the coordinate system of the surgical robot based on the auxiliary image.

[0031] In some cases, registering the secondary position relative to the primary position includes determining a position of the secondary camera relative to the primary camera.

[0032] In some cases, determining the position of the secondary camera relative to the primary camera includes scanning the secondary camera with the primary camera.

[0033] In some cases, determining the position of the secondary camera relative to the primary camera includes the controller kinematically determining positions of the first robotic arm and the second robotic arm.

[0034] In some cases, the methods herein further include continuing to scan the primary marker with the first camera during a subsequent portion of the robotic surgical procedure to track the primary position in the coordinate system of the surgical robot.

[0035] In some cases, the methods herein further include continuing to scan the auxiliary marker with the second camera during a subsequent portion of the robotic surgical procedure to track the auxiliary position in the coordinate system of the surgical robot.

[0036] In some cases, tracking the auxiliary position in the coordinate system of the surgical robot includes tracking the position of the second camera relative to the first camera.

[0037] In some cases, the primary marker is larger than the secondary marker, and the primary camera is located at a greater distance from the primary marker when scanning the primary marker than the secondary camera is located at a greater distance from the secondary marker when scanning the secondary marker.

[0038] In some cases, at least the first and robotic arms are mounted on a common chassis that establishes a coordinate system.

[0039] In some cases, primary markers are fixed to the primary vertebrae and secondary markers are fixed to the secondary vertebrae.

[0040] In some cases, the surgical tools are operated by one or more of the robotic arms.

[0041] In a second aspect of the disclosed technology, a method for performing a robotic surgical procedure includes providing a surgical robot having at least a first robotic arm, a second robotic arm, a first camera on the first robotic arm, a second camera on the second robotic arm, and a controller configured to receive images from the first camera and the second camera, kinematically position the first robotic arm and the second robotic arm in a surgical coordinate space, kinematically track a position of the first camera in the surgical coordinate space, and optically track a position of the second camera in the surgical coordinate space with the first camera, optically track a position of one or more auxiliary markers fixed at auxiliary locations on a patient's anatomy in the surgical coordinate space with the second camera, and calculate a position of the one or more auxiliary markers in the surgical coordinate space based on the kinematically tracked position of the first camera and the optically tracked position of the auxiliary markers relative to the second camera.

[0042] In some cases, the secondary marker is in the field of view of the secondary camera but not in the field of view of the primary camera.

[0043] In some cases, the method of the disclosed technology further includes tracking a position of a primary marker fixed at a primary location on the patient's anatomy in a surgical coordinate space with a first camera located at a first distance from the primary marker.

[0044] In some cases, a first distance between the first camera and the primary marker is greater than a second distance between the second camera and the secondary marker, and the primary marker is larger than the secondary marker.

[0045] In some cases, the first camera is located at a distance of up to 1.5 m from the patient anatomy, and the second camera is positioned at a distance of 30 cm or less from the target anatomy.

[0046] In some cases, the primary marking area is larger than 10 cm 2 , and the area of ​​the auxiliary mark is less than 10cm 2 .

[0047] In some cases, the method of the disclosed technology further includes continuing to scan the primary marker with the first camera during a subsequent portion of the robotic surgical procedure to track the primary position in the coordinate system of the surgical robot.

[0048] In some cases, the method of the disclosed technology further includes continuing to scan the auxiliary marker with the second camera during a subsequent portion of the robotic surgical procedure to track the auxiliary position in the coordinate system of the surgical robot.

[0049] In some cases, at least the first and robotic arms are mounted on a common chassis that establishes a coordinate system.

[0050] In some cases, primary markers are fixed to the primary vertebrae and secondary markers are fixed to the secondary vertebrae.

[0051] In some cases, the method of the disclosed technology further includes performing surgery on at least one of the primary vertebra and the auxiliary vertebra using a surgical tool operated by one or more of the robotic arms.

[0052] In a third aspect of the disclosed technology, a robotic surgical system includes at least a first robotic arm, a second robotic arm, a first camera on the first robotic arm, and a second camera on the second robotic arm; and a controller configured to (a) receive images from the first camera and the second camera, (b) kinematically position the first robotic arm and the second robotic arm in a surgical coordinate space, (c) kinematically track the position of the first camera in the surgical coordinate space, (d) optically track the position of the second camera using the first camera, and (e) calculate a position of an auxiliary marker in a field of view of the second camera based on the position of the second camera tracked by the first camera.

[0053] In some cases, the robotic surgical system further includes a surgical tool disposed on the second robotic arm.

[0054] In some cases, the robotic surgical system further includes an optical marker attached proximate the second camera, the optical marker configured to allow optical tracking of the second camera by the first camera.

[0055] In some cases, the first camera is configured to be positioned at a distance of at least 1.5 m from the patient anatomy, and the second camera is configured to be positioned at a distance of 30 cm or less from the patient anatomy.

[0056] In some cases, the surgical robot includes at least a third robotic arm carrying a surgical tool, and the first robotic arm includes a monitoring robotic arm carrying only the first camera.

[0057] In some cases, the robotic surgical system further includes a third robotic arm also carrying a third camera having markers that allow optical tracking by the first camera.

[0058] In some cases, the first camera is configured to track primary markers affixed to the patient's anatomy.

[0059] In some cases, at least the first robotic arm and the second robotic arm are mounted on a single chassis that defines a surgical coordinate space.

[0060] In some cases, the robotic surgical system further includes all of the robotic arms and controllers mounted on a single chassis.

[0061] In some embodiments, control of a surgical robot of the disclosed technology may also rely on "pose" information provided by the registration step for planning and / or control. As discussed herein, the disclosed technology provides improved placement and design of one or more navigation cameras, and the methods and systems of the disclosed technology may be applied to a variety of different surgical robot architectures and designs.

[0062] Those skilled in the art will appreciate that specific examples have been given with reference to adjacent portions of a patient's bony anatomy (spinal column). However, the present system and method can be used adjacent portions of any anatomical structure of interest (bone, skin, soft tissue) because the small navigation markers can be randomly placed without individual registration requirements. Instead, they are registered to the anatomical structure of interest through the disclosed navigation loop.

[0063] In some embodiments, another type of navigation cycle can be described. Similar to the previous navigation cycle, this time there are no markers on the miniature navigation camera. This time, the camera is positioned in a known, predefined location on one of the surgical arms, and the central controller can use the known position of its robot in space to calculate throughout the navigation / registration cycle.

[0064] Incorporation by reference

[0065] Description of Background Technology

[0066] WO2022 / 195460 has been described above. Other commonly owned publications and applications include PCT / IB2022 / 052297 (published as WO2022 / 195460); PCT / 2022 / 058988 (published as WO2023 / 067415); PCT / IB2022 / 058972 (published as WO2023 / 118984); PCT / IB2022 / 058982 (published as WO2023 / 118985); PCT / IB2022 / 058978 (published as WO2023 / 144602); PCT / IB2022 / 05898 0 (published as WO2023 / 152561); PCT / IB2023 / 055047 (published as WO2023 / 223215); PCT / IB2022 / 058988 (published as WO2023 / 237922); PCT / IB2023 / 055439; PCT / IB2023 / 056911; PCT / IB2023 / 055662; PCT / IB2023 / 055663; US63 / 524,911; and US63 / 532,753, the entire disclosures of which are incorporated herein by reference.

[0067] The entire disclosures of PCT application PCT / ___________________ entitled “SINGLE ORIGIN MARKER ASSEMBLIES AND METHODS FOR THEIR USE” (WSGR Ref: 67551-711.602; M&S Ref: P77437WO) and PCT application PCT / ______________________ entitled “INTEGRATED MULTI-ARM MOBILE MODULAR SURGICAL ROBOTIC SYSTEM” (WSGR Ref: 67551-713.602; M&S Ref: P77440WO), both of which were filed on the same day as the present application by the same applicant, are incorporated herein by reference in their entireties.

[0068] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The novel features of the disclosed technology are particularly set forth in the appended claims. A better understanding of the features and advantages of the disclosed technology will be obtained by reference to the following detailed description of illustrative embodiments in which the principles of the disclosed technology are utilized, and the accompanying drawings, in which:

[0070] Figure 1 A surgical robotic system is shown that includes a primary navigation camera mounted on a robotic monitoring arm and a secondary navigation camera mounted on a robotic working arm that also carries surgical tools, according to some embodiments.

[0071] Figure 2 A surgical robotic system according to some embodiments is shown that includes a primary navigation camera mounted on a primary robotic monitoring arm and a secondary navigation camera mounted on a dedicated secondary robotic arm that carries only the secondary camera.

[0072] Figure 3 is a top plan view of an end effector according to some embodiments, the end effector including an auxiliary navigation camera attached to a gripper tool having an attachment flange that can be removably attached to a distal end of a surgical robotic arm.

[0073] Figure 4A and Figure 4Bis an isometric side view of an end effector having an attachment flange that can be removably attached to the distal end of a surgical robotic arm. The end effector carries an auxiliary navigation camera but does not have any surgical tools. According to some embodiments, Figure 4B Relative to Figure 4A Rotate 90° around its longitudinal axis. DETAILED DESCRIPTION

[0074] Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0075] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Any reference herein to "or" is intended to include "and / or" unless stated otherwise.

[0076] As used herein, the term "about" in some instances refers to an amount that is approximately the stated amount.

[0077] As used herein, the term "about" refers to an amount that is approximately 10%, 5%, or 1% of the stated amount, including increments therein.

[0078] As used herein, the term "about" when referring to a percentage refers to an amount that is 10%, 5%, or 1% greater or less than the stated percentage, including increments therein.

[0079] As used herein, the phrases "at least one," "one or more," and "and / or" are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, and / or C" means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.

[0080] The disclosed systems and methods will now be described with specific reference to the accompanying drawings. Those skilled in the art will recognize that the described embodiments are representative in nature and that reasonable deviations may be made from the described embodiments while remaining within the scope of the disclosed technology.

[0081] like Figure 1 As shown, Figure 11 is a representative surgical robotic system 100 according to some embodiments, which includes a primary or monitoring navigation camera 102 mounted on the distal end of a monitoring robotic arm 104. The base end of the monitoring arm 104 can be mounted on a robotic platform (not shown) that includes, consists of, or consists essentially of a single cart or chassis, as described in commonly owned PCT application PCT / IB2022 / 052297 (published as WO 2022 / 195460), the entire disclosure of which is incorporated herein by reference.

[0082] The monitoring arm 104 can be configured to maintain the primary camera 102 at a conventional distance from the surgical site 106, which in the case of robotic spinal surgery as shown is typically in the range of 1.5 m to 2.5 m, so that the field of view FOV1 of the primary navigation camera 102 can include most or all of the robot components and most or all of the surgical site. In some embodiments, the field of view FOV1 of the primary navigation camera 102 can track a primary navigation marker 110, which is typically used to initially align the robot coordinates with the patient's anatomy and / or an initial preoperative patient CT scan or other scan. In some embodiments, the primary navigation marker is typically relatively large, for example having an area greater than 10 cm2.

[0083] like Figure 1 As further shown, the robotic system 100 includes at least one "working" robotic arm 120 having a distal end 122 supporting an end effector 124 that includes a gripper 126 that carries a surgical tool, such as a paddle 128. The gripper 126 may also carry various other surgical tools, such as cannulas for implanting pedicle screws, as described in commonly owned PCT Publication No. WO2023 / 223215 (the entire disclosure of which is incorporated herein by reference).

[0084] Those skilled in the art will appreciate that while a single working surgical arm 120 is shown, two, three, four, or even more working arms may be incorporated into the robotic system of the disclosed technology.

[0085] As described in the detailed description thus far, the robotic systems are generally as described in PCT publications WO2022 / 195460 and WO2023 / 223215 (the entire disclosures of which have been previously incorporated herein by reference). However, the robotic systems of the disclosed technology differ in that they are configured to deploy an auxiliary navigation camera that is intended to be positioned closer to the surgical site 106, typically as close as 30 cm or less. The auxiliary navigation camera can be mounted on a "dedicated" robotic arm, or on a working arm that also carries and deploys various surgical tools. In some cases, the auxiliary camera will be an "add-on" device that can be removably secured to a working robotic arm or other robotic arm and / or surgical tool or end effector. Figure 1 A first example is shown in which an auxiliary navigation camera is mounted on a working surgical arm, and Figure 2 A second example is shown where the auxiliary navigation camera is mounted on a dedicated auxiliary monitoring arm.

[0086] like Figure 1 As shown, the auxiliary navigation camera 130 can be removably or fixedly (typically removably) attached at or near the distal end 122 of the working robotic arm 120. The auxiliary navigation camera can be smaller than the primary navigation camera and have a maximum dimension of no more than 10 cm, and will be configured to focus on a narrower field of view FOV2 at a closer distance. In certain cases, the auxiliary navigation camera 130 will have one or more "camera" navigation markers 132 placed thereon, and the markers 132 can be very small, typically having a maximum dimension of 5 cm or less. Such a small size can allow the auxiliary navigation camera 130 with attached navigation markers 132 to be manipulated by the working robotic arm 120 with minimal interference with the surgical procedure, for example, reducing or eliminating interference with the operation of the working arm 120 and / or the operator's observation of the surgical site 106.

[0087] Figure 1 Also shown is the placement of additional "secondary" navigation markers used in performing a robotic surgery procedure. Prior to the procedure, a primary navigation marker 110 can be affixed to the first vertebra V1 so that it can be used for conventional registration of the patient's anatomy. Secondary navigation markers 140 and 142 can be deployed on vertebrae V2 and V3, respectively. While these navigation markers can be implanted and used for registration prior to the procedure, it is more common to deploy these navigation markers during the procedure (e.g., after the patient's spine has been gradually exposed) using conventional bone attachment means.

[0088] The auxiliary markers 140 and 142 (and additional auxiliary markers as needed) can be "miniaturized" (having a maximum dimension of 2 cm or less) and located at locations within the patient's body, for example, below the surface of the patient's skin adjacent to a surgical incision that exposes the spine. Because the markers 140 and 142 are effectively "buried" below the skin's surface, the auxiliary navigation markers may not be visible to the primary navigation camera 102 compared to the larger primary navigation marker 110 that extends above the patient's skin.

[0089] refer to Figure 1 , the camera navigation markers 132 disposed on the auxiliary navigation camera 130 may be visible to the primary monitoring camera 102, thereby allowing the controller to optically track the position of the auxiliary navigation camera in and around the surgical site 106. While the position of the auxiliary navigation camera 130 may be kinematically tracked by the controller based on the kinematics of the working robotic arm 120, such kinematic tracking may be less preferred because such kinematic tracking is more difficult to achieve to a desired degree of accuracy.

[0090] Information from the primary monitoring navigation camera 102 and the auxiliary navigation camera 132 can be used to optically register and subsequently track the positions of the primary navigation marker 110 and the first and second auxiliary navigation markers 140, 142 (as well as any additional auxiliary navigation markers that may be deployed later). The primary navigation marker 110 can be optically tracked by the primary monitoring navigation camera 102, the position of which is kinematically tracked by a controller, as described in WO2022 / 195460 and WO2023 / 223215 (the entire disclosures of which have been previously incorporated herein by reference). Simultaneously or substantially simultaneously, the primary monitoring navigation camera 102 can track the position of the auxiliary navigation camera 130 based on observing the camera navigation marker 132 that will be within the field of view FOV1 of the camera 102.

[0091] The positions of the auxiliary navigation markers 140 and 142 can then be tracked by the auxiliary navigation catheter 130, and the controller can calculate the positions of the markers 140 and 142 in the surgical coordinate space based on the position of the camera 130 in the surgical coordinate space. In this way, the controller can register and track the positions of the auxiliary markers in the surgical coordinate space without relying on kinematically calculated "pose" information, although the use of pose information is not excluded from the disclosed technology.

[0092] Figure 2An alternative robotic system 200 constructed according to the principles of the disclosed technology, according to some embodiments, is illustrated. A primary navigation camera 202 and a primary monitoring arm 204 can be arranged to view a primary navigation marker 210 in a surgical site 206, generally as described for similar components in robotic system 100. The primary difference in robotic system 200 is that a secondary navigation camera 230 is mounted on a dedicated secondary monitoring arm 234, rather than on a working robotic arm 220, which carries only the end effector 24, gripper 226, and surgical tool 228 at the distal end 220 of the arm. Mounting the secondary navigation camera 230 on a dedicated robotic arm 234 can be advantageous because the secondary camera 230 can be positioned to always view the secondary navigation markers 240 and 242, regardless of the positioning of the surgical tool 228 mounted on the working arm 220. The camera navigation marker 232 is shown mounted on the secondary navigation camera 230, but can be located elsewhere on the arm 234, as long as the marker's movement represents camera movement.

[0093] refer to Figure 3 、 Figure 4A and Figure 4B , shows the installation of auxiliary navigation cameras on tools and robot end effectors according to some embodiments. Figure 3 As shown, the end effector 300 includes a fixture tool 302 and a mounting flange 304. An auxiliary navigation camera 310 can be mounted on the free end of the fixture tool 302 opposite the attachment flange. The size of the auxiliary navigation camera 310 can be minimized, typically having a maximum dimension of no more than 10 cm, to reduce the possibility of interfering with the surgical procedure (e.g., interfering with the operation of one of the robotic arms or interfering with the surgical staff's view of the surgical field).

[0094] Figure 4A and Figure 4B An alternative auxiliary camera mounting embodiment is shown, comprising an end effector 400 and a flange 402 configured to attach to the free end of a robotic surgical arm. An auxiliary navigation camera 404 may be attached to the end effector 400 via a bracket 406. Figure 4A and Figure 4B The bracket 406 can be configured to be attached to a robotic arm in place of another tool, end effector, fixture, etc. Thus, according to the disclosed technology, auxiliary cameras can be placed in different arrangements on a multi-arm surgical robot. In other cases, the bracket 406 can be configured to be attached to a dedicated navigation or monitoring arm, which typically will not have other tools, end effectors, fixtures, etc., such as, for example Figure 2 shown.

[0095] In addition to the robotic system just described, the disclosed technology also provides a navigation method for performing robotic surgical procedures. In some embodiments, the method can employ a surgical robot having a robotic arm having a dedicated monitoring or navigation camera mounted thereon, wherein the distal end of the monitoring arm can be positioned at a conventional distance from the surgical site on the patient, the conventional distance being typically 1.5m to 2.5m. The surgical robot can additionally have a surgical arm configured with an end effector that can hold surgical tools to perform robotic surgery at the surgical site on the patient. In some embodiments, the surgical arm can also carry an auxiliary navigation camera, typically a small auxiliary navigation camera having a maximum diameter of 10 cm or less. Reference herein Figure 1 Such a surgical robot is shown and described.

[0096] A small auxiliary navigation catheter can be attached to a distal end or region of the surgical arm (e.g., to an end effector) and can be positioned to allow direct viewing of the surgical site. The method can further include implanting multiple navigation markers, for example, a relatively large navigation marker that can also be reused to perform the initial alignment step, and one, two, three, or more auxiliary navigation markers. For example, in spinal surgery, two, three, or more auxiliary markers can be used. Such multiple auxiliary cameras will typically be visible to one or more auxiliary cameras even when not visible to the primary monitoring camera. Even when not visible to the primary navigation camera, the auxiliary navigation markers can be visible to a smaller auxiliary navigation catheter that can be moved and positioned in the surgical site to view the auxiliary markers.

[0097] In some embodiments, during spinal surgery, auxiliary markers can typically be placed on the patient's vertebrae to be visible to an auxiliary navigation camera, typically within a surgical cavity that blocks the view of a primary navigation camera. Primary navigation markers, typically larger than the auxiliary markers, can be placed on adjacent vertebrae and extend out of the surgical cavity to be visible to a primary monitoring navigation camera, and one navigation marker can be placed on the auxiliary navigation camera, for example, attached to an end effector on a surgical arm.

[0098] In this approach, the primary monitoring navigation camera sees navigation markers on a secondary navigation camera placed on the surgical arm, and also sees primary "anatomical" navigation markers extending beyond the surgical site. The robotic controller regulates the movements of the monitoring and surgical arms, and can do so using the provided navigation information, typically without using the pose information provided by the initial registration step and subsequent kinematic tracking of the robotic arm. Optical tracking can continue while the primary monitoring navigation camera maintains simultaneous viewing of the primary and secondary navigation markers.

[0099] In some embodiments, in addition to the primary navigation camera located on the primary monitoring arm, the surgical robotic system also includes a dedicated auxiliary monitoring robotic arm with an auxiliary navigation camera mounted thereon. A working surgical arm, equipped with an end effector, can hold surgical tools for manipulation at the patient's surgical site. The auxiliary navigation camera on the dedicated auxiliary monitoring arm can be independently moved near the patient's surgical site to provide a vantage point for viewing the surgical tools and auxiliary markers, regardless of the positioning and manipulation of the surgical tools.

[0100] In some embodiments, the registration step can be performed at the beginning of the spinal surgical procedure.In some embodiments, navigation markers mounted to the vertebrae but configured to extend beyond the surgical site can include radiopaque components visible to CT or x-ray.

[0101] In some embodiments, a registration step can be performed to register the coordinate system of the robotic system with the patient's anatomy. If such a registration step is performed, then posture information can also be used for navigation during the surgical procedure. However, as discussed herein, those skilled in the art will understand that such posture information is not required. For example, the monitoring navigation camera can maintain simultaneous observation of navigation markers attached to the patient's anatomy and extending from the surgical site and navigation markers mounted to the micro-navigation camera mounted on the end effector, so that the micro-navigation camera can see anatomical elements adjacent to the patient's anatomy on which navigation markers extending from the patient's anatomy are mounted that are visible to the monitoring navigation camera.

[0102] Those skilled in the art will understand that variations of the described embodiments are possible while still remaining within the spirit of the disclosed technology. For example, the disclosed systems and methods can be deployed together with a surgical robot system having a monitoring arm and multiple surgical arms. In addition, the disclosed systems and methods can be deployed in surgical sites other than spinal surgery. Any robotic surgical site that would benefit from coordinated navigation, accurate navigation without the need for registration, and without interfering with the surgeon's workflow or line of sight would benefit from the application of the currently disclosed systems and methods. Although embodiments of the disclosed technology have been shown and described herein, it will be readily understood by those skilled in the art that such embodiments are provided by way of example only. Without departing from the scope of the present invention, those skilled in the art will now conceive of many variations, changes, and substitutions. It should be understood that various alternatives to the embodiments described herein may be adopted in practicing the present invention. It is intended that the appended claims define the scope of the technology and that the methods and structures within the scope of these claims and their equivalents are encompassed thereby.

Claims

1. A method for performing a robotic surgical procedure, the method comprising: Providing a surgical robot having at least a first robotic arm, a second robotic arm, a first camera or other sensor on the first robotic arm, a second camera or other sensor on the second robotic arm, and a controller configured to receive images from the first camera or other sensor and the second camera or other sensor and to kinematically position the first robotic arm and the second robotic arm; Place key landmarks at key locations on the patient's anatomy; scanning the patient's anatomy and the primary landmarks with the first camera or other sensor to generate a primary image, wherein the controller registers the positions of the primary landmarks within the surgical robot's coordinate system based on the primary image; placing one or more auxiliary markers at auxiliary locations on the patient's anatomy; as well as The one or more auxiliary markers are scanned with the second camera or other sensor to generate an auxiliary image, wherein the controller registers the auxiliary position relative to the primary position within the coordinate system of the surgical robot based on the auxiliary image. 2 . The method of claim 1 , wherein registering the auxiliary position relative to the primary position comprises determining the position of the auxiliary camera or other sensor relative to the primary camera or other sensor.

3. The method of claim 2, wherein determining the position of the secondary camera or other sensor relative to the primary camera or other sensor comprises scanning the secondary camera or other sensor with the primary camera or other sensor.

4. The method of claim 2, wherein determining the position of the secondary camera or other sensor relative to the primary camera or other sensor comprises the controller kinematically determining the positions of the first and second robotic arms.

5. The method of any one of claims 1 to 4, further comprising continuing to scan the primary marker with the first camera or other sensor during subsequent portions of the robotic surgical procedure to track the primary position in the coordinate system of the surgical robot.

6. The method of any one of claims 1 to 5, further comprising continuing to scan the auxiliary marker with the second camera or other sensor during a subsequent portion of the robotic surgical procedure to track the auxiliary position in the coordinate system of the surgical robot.

7. The method of claim 1, tracking the auxiliary position in the coordinate system of the surgical robot comprising tracking the position of the second camera or other sensor relative to the first camera or other sensor.

8. A method according to any one of claims 1 to 7, wherein the primary marker is larger than the auxiliary marker, and the distance between the primary camera or other sensor and the primary marker when scanning the primary marker is greater than the distance between the auxiliary camera or other sensor and the auxiliary marker when scanning the auxiliary marker.

9. A method according to any one of claims 1 to 8, wherein said at least said first and said robotic arms are mounted on a common chassis establishing said coordinate system.

10. The method of any one of claims 1 to 9, wherein the primary marker is fixed to a primary vertebra and the secondary marker is fixed to a secondary vertebra.

11. The method of any one of claims 1 to 10, wherein the surgical tool is operated by one or more of the robotic arms.

12. A method for performing a robotic surgical procedure, the method comprising: Providing a surgical robot having at least a first robotic arm, a second robotic arm, a first camera or other sensor on the first robotic arm, a second camera or other sensor on the second robotic arm, and a controller configured to receive images from the first camera or other sensor and the second camera or other sensor and kinematically position the first robotic arm and the second robotic arm in a surgical coordinate space; kinematically tracking the position of the first camera or other sensor in the surgical coordinate space; optically tracking the position of the second camera or other sensor in the surgical coordinate space with the first camera or other sensor; as well as optically tracking the position in the surgical coordinate space of one or more auxiliary markers affixed to auxiliary locations on the patient anatomy with the second camera or other sensor, Wherein the controller calculates the position of the one or more auxiliary markers in the surgical coordinate space based on the kinematically tracked position of the first camera or other sensor and the optically tracked position of the auxiliary marker relative to the second camera or other sensor.

13. The method of claim 12, wherein the auxiliary marker is in the field of view of the auxiliary camera or other sensor but not in the field of view of the first camera or other sensor.

14. The method of claim 12 or 13, further comprising tracking the position of a primary marker in the surgical coordinate space with the first camera or other sensor located at a first distance from a primary marker fixed at a primary location on the patient's anatomy.

15. The method of claim 14, wherein a first distance between the first camera or other sensor and the primary marker is greater than a second distance between the second camera and the secondary marker, and the primary marker is larger than the secondary marker.

16. The method of claim 15, wherein the first camera or other sensor is located at a distance of up to 1.5 m from the patient anatomy and the second camera or other sensor is positioned at a distance of 30 cm or less from the target anatomy.

17. The method according to claim 15 or 16, wherein the area of ​​the primary mark is greater than 10 cm 2 , and the area of ​​the auxiliary mark is less than 10cm 2 .

18. The method of any one of claims 14 to 17, further comprising continuing to scan the primary marker with the first camera or other sensor during subsequent portions of the robotic surgical procedure to track the primary position in the coordinate system of the surgical robot.

19. The method of claim 18, further comprising continuing to scan the auxiliary marker with the second camera or other sensor during subsequent portions of the robotic surgical procedure to track the auxiliary position in the coordinate system of the surgical robot.

20. A method according to any one of claims 12 to 19, wherein the at least first and robotic arms are mounted on a common chassis establishing the coordinate system.

21. The method of any one of claims 13 to 20, wherein the primary marker is fixed to a primary vertebra and the secondary marker is fixed to a secondary vertebra.

22. The method of any one of claims 13 to 21, further comprising performing surgery on at least one of the primary vertebra and the secondary vertebra using a surgical tool operated by one or more of the robotic arms.

23. A robotic surgical system comprising: a surgical robot comprising at least a first robotic arm, a second robotic arm, a first camera or other sensor on the first robotic arm, and a second camera or other sensor on the second robotic arm; and a controller configured to (a) receive images from the first camera or other sensor and the second camera or other sensor, (b) kinematically position the first robotic arm and the second robotic arm in a surgical coordinate space, (c) kinematically track the position of the first camera or other sensor in the surgical coordinate space, (d) optically track the position of the second camera or other sensor using the first camera or other sensor, and (e) calculate a position of an auxiliary marker in a field of view of the second camera or other sensor based on the position of the second camera or other sensor tracked by the first camera or other sensor.

24. The robotic surgical system of claim 23, further comprising a surgical tool disposed on the second robotic arm.

25. The robotic surgical system of claim 23 or 24, further comprising an optical marker attached proximate the second camera or other sensor, the optical marker configured to allow optical tracking of the second camera or other sensor by the first camera or other sensor.

26. A robotic surgical system according to any one of claims 23 to 25, wherein the first camera or other sensor is configured to be positioned at a distance of at least 1.5 m from the patient anatomy and the second camera or other sensor is configured to be positioned at a distance of 30 cm or less from the patient anatomy.

27. A robotic surgical system according to any one of claims 23 to 26, wherein the surgical robot comprises at least a third robotic arm carrying a surgical tool, and the first robotic arm comprises a monitoring robotic arm carrying only the first camera or other sensor.

28. The robotic surgical system of claim 27, wherein the third robotic arm also carries a third camera or other sensor having markers that allow optical tracking by the first camera or other sensor.

29. The robotic surgical system of any one of claims 23 to 28, wherein the first camera or other sensor is configured to track primary markers affixed to the patient anatomy.

30. The robotic surgical system of any one of claims 23 to 29, wherein at least the first and second robotic arms are mounted on a single chassis defining the surgical coordinate space.

31. The robotic surgical system of claim 30, wherein all robotic arms and the controller are mounted on the single chassis.

Images