Registration of multiple robotic arms using a single reference frame

Abstract

A registration method involves receiving image information corresponding to an anatomical element of a patient; receiving sensor information regarding a simultaneous pose of a patient reference frame, a first robot, and a second robot; determining, based on the image information and the sensor information, a correlation between a patient coordinate frame, a first coordinate frame of the first robot, and a second coordinate frame of the second robot; and controlling movement of the first robot and the second robot within a common work volume based on the correlation.

Description

Technical Field

[0001] This technology generally relates to robotic surgery, and more specifically to the registration of multiple robots or robotic arms for robotic surgery. Background Technology

[0002] Surgical navigation systems are used to track the position of one or more objects during surgery. Surgical robots are designed to hold one or more tools or devices during surgery and can operate autonomously (e.g., without any human input during the procedure), semi-autonomously (e.g., with some human input during the procedure), or non-autonomously (e.g., only as directed by human input). In some cases, using multiple robotic arms during surgery allows more procedures to be completed in a shorter time compared to using only one robotic arm. Summary of the Invention

[0003] Exemplary aspects of this disclosure include:

[0004] A registration method includes: receiving image information corresponding to anatomical elements of a patient; receiving sensor information about the posture of each of a patient reference frame, a first robot, and a second robot; determining a correlation between a patient coordinate system, a first coordinate system of the first robot, and a second coordinate system of the second robot based on the image information and the sensor information; and controlling the movement of the first robot and the second robot within a common working volume based on the correlation.

[0005] In any aspect of this article, the patient reference frame is firmly fixed to the patient's anatomical elements.

[0006] In any aspect of this paper, the sensor information includes information about at least one tracking tag positioned on the first robot.

[0007] In any aspect of this article, at least one of the tracking markers is a light-emitting diode.

[0008] In any aspect of this article, the postures of the first robot and the second robot are such that the first robot and the second robot come into contact with the patient reference frame.

[0009] Any aspect of this paper, wherein sensor information is received from a navigation camera.

[0010] Each of the aspects of this article is unique relative to the other coordinate systems, including the patient coordinate system, the first coordinate system, and the second coordinate system.

[0011] In any aspect of this paper, controlling the movement of the first and second robots within a common working volume based on correlation includes enabling the first and second robots to move in a coordinated manner to complete a surgical task.

[0012] In any aspect of this paper, determining the correlation between the patient coordinate system, the first coordinate system of the first robot, and the second coordinate system of the second robot based on image information and sensor information includes: determining a first correlation between the patient coordinate system and the navigation space; determining a second correlation between the first coordinate system and the navigation space; and determining a third correlation between the second coordinate system and the navigation space.

[0013] Any aspect of this article, wherein the sensor information includes information from at least two sensors.

[0014] Any aspect of this article, wherein the sensor information includes information about the posture of the first robot relative to the second robot.

[0015] A method for coordinating the control of multiple robots includes: receiving image information corresponding to anatomical elements of a patient; receiving first sensor information regarding the posture of a first robot and a patient reference frame; determining a first correlation based on the image information and the sensor information, the first correlation being between a patient coordinate system and a coordinate system of the first robot; receiving second sensor information from the first robot regarding the posture of a second robot; determining a second correlation based on the second sensor information and the first correlation, the second correlation being between the patient coordinate system and the coordinate system of the second robot; and controlling the first robot and the second robot based on the first correlation and the second correlation, wherein each of the patient coordinate system, the first coordinate system, and the second coordinate system is unique relative to the other coordinate systems.

[0016] In any aspect of this article, the second sensor information includes information about the physical connection between the first robot and the second robot.

[0017] Any aspect of this paper, wherein information from a second sensor is received from an imaging sensor of a first robot.

[0018] Any aspect of this article, wherein control includes coordinating the movement of the first and second robots to complete the surgical task.

[0019] A robot control system includes: a communication interface for communicating with a plurality of robots; at least one sensor; at least one processor; and at least one memory storing instructions for execution by the at least one processor. These instructions, when executed, are configured to cause the at least one processor to: receive sensor information about a patient reference frame, a first robot, and a second robot, the first robot having a coordinate system independent of the second robot's coordinate system; determine a correlation between the patient coordinate system, the coordinate system of the first robot, and the coordinate system of the second robot based on image information and sensor information corresponding to anatomical elements of the patient; and control the movement of the first robot and the second robot based on the correlation.

[0020] Any aspect of this article, wherein the sensor information includes first sensor information about a first robot and a patient reference frame, and second sensor information about a second robot and a patient reference frame.

[0021] In any aspect of this document, at least one memory stores additional instructions for execution by at least one processor, which, when executed, further cause at least one processor to: determine a first correlation between a patient coordinate system and a first coordinate system based on image information and first sensor information; and determine a second correlation between a patient coordinate system and a second coordinate system based on image information and second sensor information.

[0022] Any aspect of this article, wherein control includes coordinating the movement of the first and second robots to complete the surgical task.

[0023] In any aspect of this article, at least one sensor includes a navigation camera.

[0024] In any aspect of this document, at least one memory stores additional instructions for execution by at least one processor, which, when executed, further enable at least one processor to: control the movement of the first robot and the second robot within a common working volume based on correlation.

[0025] Details of one or more aspects of this disclosure are set forth in the following drawings and description. Other features, objectives, and advantages of the technology described in this disclosure will be apparent from the specification, drawings, and claims.

[0026] The phrases “at least one,” “one or more,” and “and / or” are open-ended expressions that possess both connective and disjoint qualities in operation. For example, the expressions “at least one of A, B, and C,” “at least one of A, B, or C,” “one or more of A, B, and C,” “one or more of A, B, or C,” and “A, B, and / or C” mean only A, only B, only C, A and B together, A and C together, B and C together, or A, B, and C together. When each of A, B, and C in the above expressions refers to an element such as X, Y, and Z, or such as X1-X… n Y1-Y m and Z1-Z o When referring to a single class of elements, the phrase means a single element selected from X, Y, and Z; a combination of elements selected from the same class (e.g., X1 and X2); or elements selected from two or more classes (e.g., Y1 and Z). o () combination.

[0027] The term "a / an" refers to one or more of the entities mentioned. Thus, the terms "a / an," "one or more," and "at least one" are used interchangeably herein. It should also be noted that the terms "comprising / including" and "having" are used interchangeably.

[0028] The foregoing is a simplified overview of this disclosure to provide an understanding of some aspects thereof. This summary is neither a broad nor an exhaustive overview of this disclosure and its various aspects, embodiments, and configurations. It is not intended to identify key or essential elements of this disclosure, nor to depict its scope, but rather to present selected concepts in a simplified form as an introduction to the more detailed description presented below. It should be understood that other aspects, embodiments, and configurations of this disclosure may utilize one or more of the features set forth above or described in detail below, individually or in combination.

[0029] Many additional features and advantages of the invention will become apparent to those skilled in the art upon consideration of the embodiments described below. Attached Figure Description

[0030] The accompanying drawings are incorporated in and form part of this specification to illustrate several examples of this disclosure. These drawings, together with the description, explain the principles of this disclosure. The drawings illustrate only preferred and alternative examples of how to carry out and use this disclosure, and should not be construed as limiting this disclosure to the examples shown and described only. Additional features and advantages will become apparent from the following more detailed description of various aspects, embodiments, and configurations of this disclosure, as illustrated by the figures referenced below.

[0031] Figure 1A block diagram of a system according to at least one embodiment of this disclosure;

[0032] Figure 2A An operating room with multiple robots according to at least one embodiment of the present disclosure is depicted;

[0033] Figure 2B An operating room with multiple robots according to at least one embodiment of the present disclosure is depicted;

[0034] Figure 3 This is a flowchart of a method according to at least one embodiment of the present disclosure;

[0035] Figure 4 Another flowchart of a method according to at least one embodiment of this disclosure;

[0036] Figure 5 Another flowchart of a method according to at least one embodiment of this disclosure;

[0037] Figure 6 This is another flowchart of a method according to at least one embodiment of the present disclosure; and

[0038] Figure 7 This is another flowchart of a method according to at least one embodiment of the present disclosure. Detailed Implementation

[0039] It should be understood that the various aspects disclosed herein can be combined in combinations different from those specifically given in the specification and drawings. It should also be understood that, depending on the example or embodiment, certain actions or events of any of the processes or methods described herein may be performed in different sequences, and / or may be added, combined, or omitted entirely (e.g., depending on different embodiments of this disclosure, performing the disclosed technology may not require all the described actions or events). Furthermore, for clarity, although some aspects of this disclosure are described as being performed by a single module or unit, it should be understood that the technology of this disclosure can be performed by a combination of units or modules associated with, for example, computing devices and / or medical devices.

[0040] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include non-transitory computer-readable media, which correspond to tangible media such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer).

[0041] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple A11, A12, A12X, A12Z, or A13 Bionic processors; or any other general-purpose microprocessor), application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuit systems. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other physical structures suitable for implementing the described techniques. Furthermore, this technique can be fully implemented in one or more circuit or logic elements.

[0042] Before explaining any embodiment of this disclosure in detail, it should be understood that this disclosure is not limited in its application to the construction details and component arrangements set forth in the following description or shown in the accompanying drawings. This disclosure can have other embodiments and can be practiced or carried out in various ways. Similarly, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “including / comprising” or “having” and variations thereof herein is intended to cover items listed thereafter and their equivalents, as well as additional items. Furthermore, this disclosure may use examples to illustrate one or more aspects thereof. Unless expressly stated otherwise, the use or listing of one or more examples (which may be indicated by “for example,” “by means of an example,” “e.g.,” “such as,” or similar language) is not intended to, and does not limit, the scope of this disclosure.

[0043] The use of multiple robotic arms during robotic surgery enables increasingly complex autonomous robotic surgeries and increasingly complex robot-assisted surgeries. Some surgeries and tasks require multiple simultaneous contacts with the patient, and these surgeries and tasks generally cannot be performed autonomously using a single robotic arm. Furthermore, while routine tasks are often preferred candidates for automation, routine tasks requiring multiple simultaneous contacts are generally not easy to automate when multiple robotic arms are not used simultaneously.

[0044] Two problems arise from the simultaneous use of multiple robotic arms: the feasibility and complexity of tracking multiple robotic arms using different reference frames and / or patient-mounted reference frames. When each arm requires a separate reference frame to associate the robot coordinate system with the patient's anatomy (sometimes referred to as patient space or patient coordinate system), the surgical area becomes increasingly crowded, and the time required for robot setup increases due to the need to register each robot to its appropriate reference frame. In cases where such a reference frame is patient-mounted, additional incisions are required, thus increasing trauma to the patient and the risk of future infections and / or complications.

[0045] In some embodiments of this disclosure, to reduce surgical area congestion and otherwise address these and other issues, a robotic arm can be used as a frame of reference for one or more additional robotic arms (e.g., in conjunction with navigation technology). The creation of this frame of reference can be accomplished by individually detecting any type of navigation marker on one or more arms within the arm (e.g., navigation markers on one or more segments of a robotic arm), or by creating a larger-scale, real-time varying frame of reference (e.g., navigation markers on one or more segments of multiple robotic arms) across several arms in the same system.

[0046] As discussed in this paper, some robotic systems may include a single common hub supporting multiple robotic arms. The use of such systems may not necessarily require multiple reference frames (e.g., one reference frame per arm) because once one of the robotic arms has been registered to a reference frame (to associate the robotic system's coordinate system with patient space), the common hub can be configured to control each of the multiple robotic arms based on that registration. For example, if the common hub is able to track or otherwise determine the position of each of the multiple robotic arms within the robotic system's coordinate system, a single registration from the robotic system's coordinate system to the patient's coordinate system may be required. In other words, a single common hub with multiple robotic arms can use a single robotic coordinate system, where the robotic system controls and guides the movement of two arms.

[0047] In other cases, multiple separate robotic systems can be used (each with, for example, a separate base and one or more robotic arms). For example, these robotic systems may be able to move independently relative to each other. In these examples, each robotic system may utilize an independent coordinate system, such that registering the coordinate system of the first robotic system to the patient's coordinate system may not be effective for associating the coordinate system of the second robotic system to the patient's coordinate system.

[0048] According to some embodiments of this disclosure, a first robot coordinate system may be associated with a navigation coordinate system and / or a patient coordinate system, and a second robot space may be associated with the first robot coordinate system. The second robot coordinate system may be associated with the first robot coordinate system based on a physical connection between robots corresponding to the first and second robot coordinate systems, or this association may be based on a sensed or otherwise detected relationship between the first and second robot coordinate systems.

[0049] In other embodiments, the first robot coordinate system may be associated with the navigation coordinate system and / or the patient coordinate system using a first reference system, and the second robot coordinate system may also be associated with the navigation coordinate system and / or the patient coordinate system using the first reference system.

[0050] The embodiments disclosed herein provide technical solutions to the problems of: (1) registering multiple robots and / or robot arms to a navigation coordinate system and / or a patient coordinate system; (2) using only a single reference system for such registration to avoid crowding of the surgical area with multiple reference systems; (3) safely operating multiple robots in the surgical environment; (4) avoiding collisions or other interferences between multiple robots operating in a common surgical space by registering the coordinate system of each robot to a common patient and / or navigation space; and / or (5) setting up the robotic system for surgical procedures as efficiently as possible.

[0051] First turn Figure 1 A block diagram of a system 100 according to at least one embodiment of the present disclosure is shown. System 100 may be used for: performing one or more aspects of one or more of the methods disclosed herein; for navigation purposes; for registration purposes; performing fully autonomous and / or robot-assisted surgery using multiple robots; or for any other useful purpose. System 100 includes a computing device 102, at least two robots 136, a navigation system 156, a database 160, and a cloud 164. Despite the foregoing description, systems according to other embodiments of the present disclosure may omit any one or more of the computing device 102, one or more of the at least two robots 136, the navigation system 156, the database 160, and / or the cloud 164. Additionally, systems according to other embodiments of the present disclosure may arrange one or more components of system 100 differently (e.g., one or more of the robots 136 and / or the navigation system 156 may include...). Figure 1 (One or more components shown as part of the computing device 102).

[0052] The computing device 102 includes at least one processor 104, at least one communication interface 108, at least one user interface 112, and at least one memory 116. In other embodiments of this disclosure, the computing device may omit one or both of the communication interface 108 and the user interface 112.

[0053] At least one processor 104 of computing device 102 may be any processor identified or described herein or any similar processor. At least one processor 104 may be configured to execute instructions stored in at least one memory 116, which may cause at least one processor 104 to perform one or more computational steps using or based on data received, for example, from robot 136, navigation system 156, database 160 and / or cloud 164.

[0054] The computing device 102 may also include at least one communication interface 108. The at least one communication interface 108 may be used to receive image data or other information from external sources (such as robot 136, navigation system 156, database 160, cloud 164, and / or portable storage media (e.g., USB drive, DVD, CD)), and / or to transmit instructions, images, or other information more generally from at least one processor 104 and / or the computing device 102 to external systems or devices (e.g., another computing device 102, robot 136, navigation system 156, database 160, cloud 164, and / or portable storage media (e.g., USB drive, DVD, CD)). The at least one communication interface 108 may include one or more wired interfaces (e.g., USB port, Ethernet port, FireWire port) and / or one or more wireless interfaces (e.g., configured to transmit information via one or more wireless communication protocols, such as 802.11a / b / g / n, Bluetooth, Bluetooth Low Energy, NFC, ZigBee, etc.). In some implementations, the at least one communication interface 108 may be used to enable the device 102 to communicate with one or more other processors 104 or computing devices 102, whether to reduce the time required to complete computationally intensive tasks or for any other reason.

[0055] At least one user interface 112 may be or include a keyboard, mouse, trackball, monitor, television, touchscreen, button, joystick, switch, lever, and / or any other means for receiving information from a user and / or for providing information to a user of computing device 102. At least one user interface 112 may be used, for example, to receive user selections or other user inputs for any step in conjunction with any of the methods described herein; to receive user selections or other user inputs regarding one or more configurable settings of computing device 102, robot 136, or another component of system 100; to receive user selections or other user inputs regarding how to store and / or transmit data received, modified, and / or generated by computing device 102 and / or where to store and / or transmit such data; and / or to display information (e.g., text, images) and / or play sounds to a user based on data received, modified, and / or generated by computing device 102. Although the system 100 includes the at least one user interface 112, the system 100 may automatically (e.g., without any input via the at least one user interface 112 or otherwise) perform one or more or all of the steps of any of the methods described herein.

[0056] Although the at least one user interface 112 is shown as part of the computing device 102, in some embodiments, the computing device 102 may utilize the user interface 112, which is housed separately from one or more other components of the computing device 102. In some embodiments, the user interface 112 may be located close to one or more other components of the computing device 102, while in other embodiments, the user interface 112 may be located away from one or more other components of the computing device 102.

[0057] At least one memory 116 may be or include RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible non-transitory memory used to store computer-readable data and / or instructions. At least one memory 116 may store information or data for performing any step of, for example, the methods 300, 400, 500, 600, and / or 700 described herein. At least one memory 116 may store, for example, information about one or more coordinate systems 120 (e.g., information about the robot coordinate system or space corresponding to each robot 136, information about the navigation coordinate system or space, information about the patient coordinate system or space); instructions 124 for execution by at least one processor 104, such as causing at least one processor 104 to perform one or more steps of methods 300, 400, 500, 600 and / or 700; and / or one or more algorithms 128 for any calculations required by the processor to perform to complete one or more steps of methods 300, 400, 500, 600 and / or 700 (e.g., mapping one coordinate system to another), or for any other calculations. In some embodiments, such predetermined coordinate system information 120, instructions 124 and / or algorithms 128 may be organized into one or more applications, modules, packages, layers or engines, and may cause at least one processor 104 to manipulate data stored in at least one memory 116 and / or received from or via another component of system 100.

[0058] Each robot in robot 136 can be any surgical robot or surgical robot system. Each robot 136 can be or includes, for example, Mazor X. TM Stealth robot guidance system. Each robot 136 may include a base 140 supporting a robotic arm 148. Each robot 136 may include one or more robotic arms 148 (e.g., some robots 136 may include two, three, four, or another number of robotic arms 148). In some embodiments, each robotic arm 148 may assist in surgical procedures (e.g., by holding the tool in a desired trajectory or posture and / or supporting the weight of the tool while another robot 136, surgeon, or other medical professional operates the tool; or by keeping the patient's skin taut while another robot 136, surgeon, or other medical professional makes an incision in the patient's skin; or by other means) and / or automate surgical procedures.

[0059] Each robotic arm 148 can have three, four, five, six or more degrees of freedom.

[0060] Robot 136 also includes one or more sensors 144. Sensor 144 may be an imaging sensor, such as a visible light camera, an infrared camera, or an ultrasonic probe. Sensor 144 may be a position sensor, a proximity sensor, a magnetometer, or an accelerometer. In some embodiments, sensor 144 may be a linear encoder, a rotary encoder, or an incremental encoder. Other types of sensors may also be used as sensor 144.

[0061] Data from one or more sensors 144 may be provided to the processor of robot 136, the processor 104 of computing device 102, and / or the navigation system 156. The data can be used to calculate the position of robot arm 148 in space relative to one or more coordinate systems (e.g., based on coordinate system information 120 stored in memory 116). The calculation may be based not only on data received from one or more sensors 144, but also on data or information (such as, for example, physical dimensions) about, for example, robot 136 or a portion thereof, or any other related object, which may be stored, for example, in memory 116 of computing device 102 or any other memory.

[0062] One or more tracking markers 152 may be securely attached or positioned on the robot 136, whether on the base 140, the robot arm 148, and / or elsewhere. As used herein, "securely attached" does not mean "permanently attached," and in fact, the tracking markers 152 can be detached from the robot arm 136. The tracking markers 152 may be light-emitting diodes (LEDs). The tracking markers 152 may be all identical, or one or more of the tracking markers 152 may be different from the other one or more tracking markers in the tracking markers 152. In some embodiments, one or more tracking markers 152 may be configured to emit light of a first wavelength, and the other one or more tracking markers 152 may be configured to emit light of a second wavelength different from the first wavelength. Furthermore, in some embodiments, one or more tracking markers 152 may be configured to reflect light of the first wavelength, while the other one or more tracking markers 152 may be configured to reflect light of a second wavelength different from the first wavelength. The emission wavelength and / or reflection wavelength of the light in the embodiments described above may be wavelengths within a specific spectrum (e.g., wavelengths corresponding to red light and wavelengths corresponding to blue light in the visible spectrum, or different wavelengths in the infrared spectrum) and wavelengths from different spectra (e.g., wavelengths in the visible spectrum and wavelengths in the infrared spectrum).

[0063] In some embodiments, one or more tracking marks in tracking mark 152 may be or include LEDs that generate pulses at a first frequency, and another one or more tracking marks in tracking mark 152 may be or include LEDs that generate pulses at a second frequency different from the first frequency. In some embodiments, tracking mark 152 may be or include reflective spheres, geometric patterns (such as, for example, QR codes), or other items or features that can be easily distinguished by sensors (such as sensor 144) and / or navigation systems (such as navigation system 156).

[0064] In some embodiments of this disclosure, one or more of the plurality of tracking marks 152 may be movably fixed to the robot arm 148 and may be further selectively movable relative to the robot arm 148. In such embodiments, one or more of the plurality of tracking marks 152 may be configured to move (or automatically move) from a first position on the robot arm 148 to a second position on the robot arm 148 as the robot arm 148 moves into or out of a position or group of positions. Such movement of one or more of the plurality of tracking marks 152 may be intended to facilitate the maintenance of the line of sight between each of the plurality of tracking marks 152 (or at least a subset of the plurality of tracking marks) and the sensor 144 / or navigation system 156. In such implementations, robot 136 (and / or another component of system 100) may be configured to track whether each of a plurality of tracking markers 152 is in its respective first or second position, and to provide such information to navigation system 156 (or to any other component of system 100) to establish the correlation between the robot coordinate system and the navigation coordinate system (or any other coordinate system) based on the position of the tracking markers 152 relative to the robot arm 148 as known to robot 136 (and / or another component of system 100) and further based on the position of the tracking markers 152 as detected by sensor 144 and / or navigation system 156.

[0065] The use of tracking markers 152 to determine the position of a robotic arm 148 in space is more fully described in U.S. Patent Application 63 / 036,130, entitled “Robotic Reference Frames for Navigation,” filed June 8, 2020. The entire disclosure of that U.S. Patent Application is incorporated herein by reference for its teaching and all disclosures.

[0066] The navigation system 156 of system 100 can provide navigation for the surgeon and / or two or more robots 136 during surgery. The navigation system 156 can be any navigation system now known or developed in the future, including, for example, Medtronic StealthStation.TM S8 Surgical Navigation System. Navigation system 156 may include cameras or one or more other sensors for detecting and / or tracking one or more reference markers, navigation trackers, or other objects in the operating room or other rooms where surgery is performed. In various embodiments, navigation system 156 may be used to track the position of the robotic arm 148 of each robot 136 (or more specifically, one or more tracking markers 152 attached to the robotic arm 148). Navigation system 156 may be used to track the position of one or more reference frames, markers, arrays, or other structures for detection by cameras or other sensors of navigation system 156. Navigation system 156 may be used, for example, to detect the position of a reference frame mounted on the patient and / or the position of one or more robotic arms 148, and to register or otherwise associate the patient coordinate system with the robot coordinate system at least based on the detected positions. Navigation system 156 may include a display for showing one or more images from an external source (e.g., computing device 102, database 160, cloud 164, or another source), or video streams from cameras or other sensors of navigation system 156. In some implementations, system 100 can operate without using navigation system 156.

[0067] Database 160 may store information relating one coordinate system to another (e.g., information relating one or more robot coordinate systems to a patient coordinate system and / or a navigation coordinate system). Database 160 may additionally or alternatively store: for example, information about or corresponding to one or more characteristics of tracking marker 152; one or more surgical plans (including, for example, image information about the anatomy of the patient at and / or near the surgical site, for use by the user of robot 136, navigation system 156, and / or computing device 102 or system 100); one or more useful images of a surgical procedure performed by or with the assistance of one or more other components of system 100; and / or any other useful information. Database 160 may be configured to provide any such information to computing device 102 or to any other device of system 100 or any other device outside system 100, whether directly or via cloud 164. In some implementations, database 160 may be or include part of a hospital image storage system, such as a Picture Archiving and Communication System (PACS), a Health Information System (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records including image data.

[0068] Cloud 164 can be or represents the Internet or any other wide area network. Computing device 102 can connect to cloud 164 via communication interface 108 using a wired connection, a wireless connection, or both. In some embodiments, computing device 102 can communicate with database 160 and / or external devices (e.g., computing devices) via cloud 164.

[0069] Now go to Figure 2A The embodiments of this disclosure can be used, for example, in conjunction with robot-assisted surgery or fully autonomous surgery involving two robots 136a, 136b. Although the two robots 136a, 136b can be positioned anywhere within the operating room 200, Figure 2A Two robots 136a, 136b are shown positioned on opposite sides of an operating table 204 on which the patient 208 lies. A reference frame 212 is securely attached to the patient 208, and more specifically to anatomical elements of the patient, such as bones (e.g., pelvis, vertebrae, skull). The reference frame 212 includes a plurality of reflective spheres 216, but in some embodiments, the reference frame 212 may include one or more other high-visibility markers that can be easily detected by a navigation system camera or other sensors, such as infrared emitting diodes. The reference frame 212 may be detected by a navigation system camera, an electromagnetic sensor, or other suitable sensors for navigation systems other than optical and electromagnetic navigation systems. Thus, the reference frame 212 enables the navigation system (e.g., navigation system 156) to determine the precise posture (e.g., precise position and orientation) of the reference frame 212 in the coordinate system of the navigation system. Using the detected information about the posture of reference frame 212 and the image information about the anatomical elements to which the reference frame is attached and / or the precise location of the attachment of the reference frame to the anatomical elements, computing devices (such as computing device 102) and / or navigation system 156 are able to register or otherwise associate a patient-centered coordinate system with the coordinate system of the navigation system, or vice versa.

[0070] like Figure 2A As shown, each of robots 136a and 136b includes a mobile base 140 and a robotic arm 148, and each robotic arm 148 includes an end effector 218. The mobile base 140 is selectively movable and, once positioned in the desired location, can be locked in place (whether using wheel locks or otherwise) to prevent movement during surgery. At any point during surgery (if necessary) and / or at the end of surgery, the mobile base 140 can be unlocked to facilitate the removal and / or repositioning of robots 136a and 136b. Although in Figure 2AThe robots 136a and 136b are shown as independent wheeled robots, but robots 136a and 136b may alternatively be mounted (directly or indirectly) to an operating table or another fixed (e.g., stationary) structure, including, for example, an operating room floor, ceiling, wall, and / or any structure fixedly mounted to either of the foregoing.

[0071] Multiple tracking markers 220 are positioned on the robot 136, including some tracking markers on the base 140 and other markers on the robot arm 148 (including one tracking marker 220 on each end effector 218). Tracking markers 220 may be the same as or similar to tracking markers 152. Figure 2A The positions of the tracking markers 220 on robots 136a and 136b are merely exemplary; in different embodiments, one or more tracking markers 220 may be located at different positions on either or both of robots 136a and 136b. Additionally, in different embodiments, more or fewer tracking markers 220 may be provided on robots 136a and 136b. For example, in some embodiments, each robot 136a and 136b may include only a single tracking marker 220, which may be, for example, on the end effector 218 of robot arm 148. In other embodiments, each robot 136a and 136b may include tracking markers 220 located only on robot arm 148 or only on base 140. Furthermore, in some embodiments, tracking markers 220 may be located only on the joints of robot arm 148, or only on a segment of robot arm 148 extending between its joints (or between end effector 218 and the joints of robot arm 148). Tracking marker 220 allows computing devices (e.g., computing device 102) and / or navigation systems (e.g., navigation system 156) to position robot 136 (or robot arm 148 or end effector 218) in navigation space (e.g., in navigation coordinate system) and / or in patient space (e.g., in patient coordinate system).

[0072] Furthermore, in some embodiments, the tracking marker 220 enables the computing device and / or navigation system to register or otherwise associate the coordinate system corresponding to the robot space of each robot 136a, 136b with one or both of the patient coordinate system (corresponding to the patient space) and / or the navigation coordinate system (corresponding to the navigation space). For both robots 136a and 136b, the registration or other association may occur simultaneously or sequentially. In some embodiments, the coordinate systems of each robot 136a, 136b and the patient coordinate system corresponding to the patient space of the patient 208 may be simultaneously registered or associated with each other, and / or simultaneously registered or associated with the navigation coordinate system.

[0073] In some implementations, the registration or other correlation of the coordinate systems of robots 136a and 136b with the patient space of patient 208 can enable, for example, a robot control system (which may be, for example, a computing device (such as computing device 102) or a navigation system (such as navigation system 156), or may include both a computing device and a navigation system) to simultaneously control both robots 136a and 136b to maintain one robot 136a (including its robotic arm 148) in a first working volume and the other robot 136b (including its robotic arm 148) in a second working volume separated from (but possibly adjacent to) the first working volume. For example, in cases where the first and second working volumes are adjacent to each other and / or have shared boundaries more complex than a simple plane, the registration of coordinate systems as discussed above can advantageously facilitate the precise definition of the first and second working volumes and thus help ensure successful separation of the robotic arms 148 of robots 136a and 136b and prevent any interference between them.

[0074] As an example, the robot control system described above can control a pair of robots 136a and 136b to operate in separate, non-adjacent work volumes. One robot 136a can be maintained within a work volume covering an upper vertebra (e.g., a thoracic vertebra), while the other robot 136b can be maintained within a work volume covering a lower vertebra (e.g., a lumbar vertebra), wherein the upper and lower vertebrae are not adjacent to each other. As another example, the robot control system can control the pair of robots 136a and 136b to operate in separate but adjacent work volumes. Thus, one robot 136a can be maintained within a work volume covering a first vertebra, while the other robot 136b can be maintained within a work volume covering a second vertebra adjacent to the first vertebra. Moreover, in some embodiments, the boundary between the work volumes of each robot 136 may not be defined by anatomical boundaries. For example, one robot 136a can be controlled within a work volume covering half of the vertebra, while the other robot 136b can be controlled within a work volume covering the other half of the same vertebra.

[0075] In other embodiments, the registration or other correlation of the coordinate systems of robots 136a and 136b with the patient space of patient 208 may enable, for example, a robot control system (which may be, for example, a computing device (such as computing device 102) or a navigation system (such as navigation system 156), or may include both a computing device and a navigation system) to simultaneously control both robots 136a and 136b within a common work volume to perform one or more individual surgical tasks and / or to perform surgical procedures without interfering with each other, although the robot arm 148 and / or end effector 218 alternately occupy the same space within the common work volume. In other words, instead of restricting one robot 136a, 136b to operate on one side of the patient 208 and the second robot 136a, 136b to operate on the other side of the patient 208, registering the coordinate systems of robots 136a, 136b to the coordinate system corresponding to the patient 208 allows for more efficient use of robot 136 than without registration (e.g., in the absence of artificially restricting the potential movement path of robot arm 148) (although this is not always the case, and as discussed above, embodiments of the invention can limit each robot 136a, 136b to a separate working volume). Registration also ensures that each robot 136a, 136b can interact precisely with a designated portion of the patient 208, thereby protecting patient safety and ensuring that any surgical task or procedure is performed correctly. In cases where registration includes registration with the navigation space or other relevant aspects, registration also enables the navigation system to track the precise position of each robot 136a, 136b relative to the patient 208, and / or provide information and / or commands for controlling the robots 136a, 136b to perform a given surgical task or procedure without interfering with each other and / or without harming the patient 208.

[0076] Robot 136a includes sensor 144, which in some embodiments can be used to detect the position of robot 136b and / or the position of robot arm 148 of robot 136b (e.g., by detecting tracking marker 220 on robot 136 and / or the position of robot arm 148 of robot 136b). Sensor 144 can be used, for example, to facilitate the registration or other correlation of robot coordinate system of robot 136b with robot coordinate system of robot 136a, or vice versa. Thus, once the first coordinate system of one of robots 136a, 136b has been registered to the patient coordinate system corresponding to patient 208 and / or the navigation coordinate system corresponding to a navigation system (such as navigation system 156), the second coordinate system of the other robot of robots 136a, 136b can be registered to or otherwise associated with the first coordinate system based on information collected by sensor 144.

[0077] While the present invention covers embodiments in which a first registration is performed between the first coordinate systems of the first robot 136a and a second registration is then performed between the first coordinate system of the first robot 136a and the second coordinate system of the second robot 136a, as discussed above, in other embodiments of the invention, each robot 136a, 136b is directly registered to the patient coordinate system corresponding to the patient 208, and / or directly registered to the navigation coordinate system corresponding to a navigation system (such as navigation system 156). In such embodiments, the sensor 144 on robot 136a may not be needed for registration purposes, or at least not for the purpose of registering the first coordinate system of the first robot 136a to the second coordinate system of the second robot 136b.

[0078] Figure 2B and Figure 2A Basically the same, except in Figure 2B In this configuration, two robots 136 are physically connected to each other by a rigid elongated member 224, which is secured to each robot 136a, 136b at a joint 228. The elongated member 224 may be, for example, a metal strip, or a strip or other member made of any rigid material. The elongated member 224 may include a ball at each end, and the joint 228 on each robot 136a, 136b may include a socket adapted to receive the ball and a locking mechanism configured to lock the ball into the socket at a given orientation so as to prevent relative movement of the members 224 once they are locked in place, and thus prevent relative movement of the robots 136a, 136b. By connecting robots 136a and 136b in this way, the coordinate system of one robot 136a or 136b can be registered or associated with the coordinate system of the other robot 136a or 136b based on, for example, dimensional information about component 224 (e.g., the length of component 224) and sensed or otherwise obtained information about the angle at which component 224 is fixed to joint 228 / robot 136. Such information can be sensed, for example, by sensors in joint 228, or measured using markings on joint 228 or using separate gauges or tools. In some embodiments, measurements along six degrees of freedom may be required to accurately register or associate the coordinate systems of the two connected robots 136a or 136b, but in other embodiments (e.g., depending on whether robots 136a and 136b are identical, the position of joint 228 on each robot 136a or 136b, etc.), measurements along fewer than six degrees of freedom may be necessary. For example, in some embodiments, measurements along five, four, three, or two degrees of freedom may be sufficient.

[0079] Figure 3A registration method 300 is depicted. Registration method 300 (and / or one or more steps thereof) may be executed, for example, by at least one processor or otherwise. The at least one processor may be the same as or similar to processor 104 of the computing device 102 described above. The at least one processor may be part of a robot (such as robot 136) or a navigation system (such as navigation system 156). Processors other than any processor described herein may also be used to execute method 300. At least one processor may execute method 300 by executing instructions stored in memory (such as instruction 124 in memory 116). These instructions may correspond to one or more steps of method 300 described below. These instructions may cause the processor to execute one or more algorithms, such as algorithm 128. For example, once each coordinate system has been located, one or more such algorithms 128 may be used to map one coordinate system to another.

[0080] Method 300 includes receiving image information corresponding to information about the patient's anatomical elements (step 304). The image information may be received, for example, as part of a preoperative plan. Alternatively or additionally, the image information may be received directly or indirectly from an imaging device such as a CT scanner, magnetic resonance imaging (MRI) scanner, optical coherence tomography (OCT) scanner, O-arm (including, for example, an O-arm 2D long-film scanner), C-arm, G-arm, another device utilizing X-ray-based imaging (e.g., a fluoroscope or other X-ray machine), or any other imaging device. The image information may be or include multiple two-dimensional (2D) images and / or one or more three-dimensional (3D) images. In some embodiments, the image information may be or include a 3D model of the anatomical elements, which may in turn be generated using one or more 2D or 3D images.

[0081] Method 300 further includes receiving sensor information about the reference frame, the posture of the first robot, and the posture of the second robot (step 308). The reference frame may be the same as or similar to reference frame 212, and the first robot and the second robot may be, for example, [missing information - likely related to sensor information]. Figure 1 Robot 136 and / or Figures 2A-2BThe robots 136a and 136b are identical or similar. In some embodiments, the poses of the reference frame, the first robot, and the second robot can be simultaneous poses (e.g., the poses of the reference frame, the first robot, and the second robot at the same moment). In other embodiments, the poses of the reference frame, the first robot, and the second robot can be the poses of each individual device at different moments. Sensor information can be any information used to determine the poses of the reference frame, the first robot, and the second robot at a single point in time. Thus, for example, sensor information can include image information received directly or indirectly from imaging devices such as ultrasound probes, CT scanners, magnetic resonance imaging (MRI) scanners, optical coherence tomography (OCT) scanners, O-arms (including, for example, O-arm 2D long film scanners), C-arms, G-arms, another device utilizing X-ray-based imaging (e.g., a fluoroscope or other X-ray machine), or any other imaging equipment. Sensor information may alternatively be or include information received directly or indirectly from an electromagnetic positioning or navigation system, which may include information about the detected positions of one or more magnets or other electromagnetic field sources from which the poses of the reference frame, the first robot, and the second robot can be determined.

[0082] In embodiments where sensor information includes image information, the image information may correspond to, for example, an X-ray image showing a reference frame, multiple tracking markers of a first robot (e.g., tracking marker 220), and multiple tracking markers of a second robot (e.g., tracking marker 220). The X-ray image may also show anatomical elements of the patient to which the reference frame is securely fixed.

[0083] Sensor information may also be, or include, information from one or more sensors of the first and second robots. For example, one or more sensors of the first and second robots may be used to determine (independent of any tracking marks thereon) the posture of their robot arm 148, which may be used in method 300. As another example, the navigation system may be used to detect any tracking marks attached to the robot's robot arm and / or other places on the robot, from which the posture of the robot and / or its robot arm may be determined. Therefore, sensor information may be received from navigation cameras or other navigation system sensors.

[0084] Furthermore, in some embodiments, the sensor information may include information about the position (or, alternatively, posture) of the second robot relative to the first robot, or vice versa (this information may be or includes, for example, information from which the position or posture of the second robot relative to the first robot can be determined). For example, in an embodiment where the first and second robots are physically connected by a rigid member (such as, for example, member 224), the sensor information may include information from sensors in each joint (e.g., each joint 228) connecting the rigid member to one of the robots, which allows the relative position / posture of the two robots to be determined based on known information about the position of the joints on the robots and known information about the length of the rigid member.

[0085] In some embodiments, the posture of each of the first and second robots can be a posture in which each robot is in contact with a reference frame or some other predetermined point. Such postures can be used, for example, to allow for the determination of the precise position of one or more points of each robot based on the determined position of one or more points of the reference frame based on the other predetermined points. In some embodiments, the robots may be in contact with each other at such predetermined points. The predetermined points may be stationary relative to the base of the other robot (e.g., a robot arm of one robot may contact the base of another robot) or movable relative to the base of another robot (e.g., a robot arm of one robot may contact the robot arm or end effector of another robot). A robot may, for example (using one or more sensors, such as sensor 144), "see" the predetermined points and automatically move its robot arm and / or end effector to the predetermined points. Once contact is made at the predetermined points, the position of one robot relative to the other robot is known. Furthermore, in some embodiments, the predetermined points may include multiple points that a robot arm may contact simultaneously. In particular, using multiple points facilitates the determination of the robot's position where the robot arm may contact each of the multiple points only in one orientation. Contact between the robot and a reference frame can be used instead of, for example, one or more tracking markers on the robot.

[0086] This disclosure covers embodiments of method 300 utilizing variations of step 308 (and the remaining steps of method 300). For example, in some embodiments, step 308 may include receiving sensor information about the pose of a reference frame and the poses of more than two robots, and / or receiving sensor information about the poses of the reference frame, the first robot, the second robot, and the anatomical element. In each case, the pose of the identified element may be a simultaneous pose (e.g., the pose of each element at the same time) or a sequential pose (e.g., the pose of each element at different times).

[0087] Method 300 further includes determining the correlation between a patient coordinate system corresponding to the patient, a first coordinate system corresponding to the first robot, and a second coordinate system corresponding to the second robot (step 312). The determination is based on image information and sensor information, but in some embodiments, the determination may be based solely on sensor information. Determining the correlation may include registering the first and second coordinate systems (for the first and second robots, respectively) to the patient coordinate system, or registering the patient coordinate system, the first coordinate system, and the second coordinate system to a navigation coordinate system or other coordinate systems. The determination may include using one or more algorithms (such as algorithm 128) to determine the mapping of any one or more of the patient coordinate system, the first coordinate system, and the second coordinate system to any other aforementioned coordinate system and / or to the navigation coordinate system or other coordinate systems. The correlation may enable the use of instructions referencing the patient coordinate system to control, for example, the first and second robots, or may enable the conversion of patient coordinate system-based instructions into instructions based on the first and / or second coordinate systems to cause the first and second robots to move within their respective coordinate systems to perform a task or procedure in the patient coordinate system. The identified correlations can be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The identified correlations can be transmitted via a network (such as cloud 164).

[0088] Method 300 further includes controlling the movement of the first robot and the second robot within a defined working volume based on correlation (step 316). The defined working volume may include separate working volumes for each robot, or a single shared working volume, as described elsewhere herein. When the defined working volume is a shared working volume, the shared working volume may be a volume comprising points reachable in space and occupies by both the first robot and the second robot (albeit not simultaneously). Thus, the shared working volume comprises a single working volume that is accessible in its entirety to both the first robot and the second robot, and wherein the first robot and the second robot can be safely controlled based on correlation without colliding with or otherwise interfering with each other. When the defined working volume includes separate working volumes, the separate working volumes may include a first working volume for the first robot and a second working volume for the second robot, wherein the first and second working volumes are mutually exclusive, thus also facilitating simultaneous operation of the first robot and the second robot while reducing the risk of collision or interference.

[0089] Control may include causing the first and second robots (and more specifically, the robotic arms of the first and second robots, or even more specifically, the end effectors of the first and second robots) to work in a coordinated manner to perform a given surgical task (e.g., making an incision in the patient's skin) or a given surgical procedure (e.g., spinal decompression). Control may be based, for example, on a preoperative plan and may utilize one or more instructions (such as instructions 124 stored in memory (e.g., memory 116)).

[0090] This disclosure covers embodiments of method 300 that include more or fewer steps than those described above, and / or one or more steps that differ from the steps described above. For example, in some embodiments, method 300 may further include receiving information about a first coordinate system and a second coordinate system, respectively, from, for example, a first robot and a second robot. In such embodiments, in addition to received sensor information and / or received image information, the determination step 312 may also be based on the received coordinate system information.

[0091] Each coordinate system discussed above in method 300 is unique relative to the others. In other words, the first robot has a first coordinate system with an origin different from the second coordinate system of the second robot, and the first and second coordinate systems of the first and second robots each have an origin different from the patient coordinate system. In addition to having different origins, in some embodiments, each coordinate system may be oriented differently such that none of the axes of the first coordinate system (e.g., none of the X, Y, or Z axes) is parallel to any of the axes of the second coordinate system and / or any of the axes of the patient coordinate system (or, if any two axes are parallel, then those axes are different (e.g., the X-axis of one coordinate system is parallel to the Z-axis of the other coordinate system)).

[0092] Now go to Figure 4The method 400 for determining the correlation between a patient coordinate system (e.g., corresponding to a patient), a first coordinate system (e.g., of a first robot), and a second coordinate system (e.g., of a second robot) may be performed, for example, by at least one processor or otherwise. The at least one processor may be the same as or similar to the processor 104 of the computing device 102 described above. The at least one processor may be part of a robot (such as robot 136) or a navigation system (such as navigation system 156). Processors other than any of the processors described herein may also be used to perform method 400. The at least one processor may perform method 400 by executing instructions stored in memory (such as instruction 124 in memory 116). These instructions may correspond to one or more steps of method 400 described below. These instructions may cause the processor to execute one or more algorithms, such as algorithm 128. For example, once each coordinate system has been located, one or more such algorithms 128 may be used to map one coordinate system to another.

[0093] Method 400 corresponds to step 312 of method 300 described above. In other words, method 400 constitutes a way to determine the correlation between the patient coordinate system, the first coordinate system of the first robot, and the second coordinate system of the second robot.

[0094] Method 400 includes determining a first correlation between the patient coordinate system and the navigation space (step 404). The patient coordinate system may be defined relative to points on, for example, patient anatomy structures selected by the surgeon or chosen autonomously, and / or by points on which a reference frame is securely anchored to the patient's anatomical elements. The patient coordinate system may or may not be defined with respect to the surgical procedure to be performed on the patient. In some embodiments, the patient coordinate system may be defined in the preoperative plan, while in other embodiments, the patient coordinate system may be defined independently of any preoperative plan.

[0095] The navigation space can be, for example, a space visible to sensors of a navigation system (such as navigation system 156). In some embodiments, the navigation space may be represented by or correspond to a navigation coordinate system, and step 404 may include determining a first correlation between the patient coordinate system and the navigation coordinate system.

[0096] Determining the first correlation may include using one or more algorithms (such as algorithm 128) to determine the mapping from the patient coordinate system to the navigation space. The first correlation enables the transformation of any coordinate in the patient coordinate system to any coordinate in the navigation space, or vice versa. The first correlation may be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The first correlation may be transmitted via a network (such as cloud 164).

[0097] Method 400 further includes determining a second correlation between the first coordinate system and the navigation space (step 408). The first coordinate system is a coordinate system used by a first robot, which may be, for example, robot 136. For example, the first robot may instruct its robotic arm to move its end effector to a given set of coordinates in the first coordinate system. The first coordinate system may be defined relative to a fixed point on the robot (e.g., a point on the robot base) and / or relative to a point on the robot's end effector when the robotic arm is in a predetermined position.

[0098] Determining the second correlation may include using one or more algorithms (such as algorithm 128) to determine the mapping from the first coordinate system to the navigation space. The second correlation may enable the conversion of any coordinate in the first coordinate system to any coordinate in the navigation space, or vice versa. For example, the second correlation may enable the control of a first robot using instructions referencing the navigation space (e.g., the navigation coordinate system), or may enable the conversion of instructions based on the navigation coordinate system to instructions based on the first coordinate system, so that the first robot can move within the first coordinate system to perform a task or procedure defined relative to the patient coordinate system. The second correlation may be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The second correlation may be transmitted via a network (such as cloud 164).

[0099] Method 400 further includes determining a third correlation between the second coordinate system and the navigation space (step 412). The second coordinate system is a coordinate system used by a second robot, which may be, for example, robot 136. For example, the second robot may instruct its robotic arm to move its end effector to a given set of coordinates in the second coordinate system. The second coordinate system may be defined relative to a fixed point on the robot (e.g., a point on the robot base) and / or relative to a point on the robot's end effector when the robotic arm is in a predetermined position.

[0100] Determining the third correlation may include using one or more algorithms (such as algorithm 128) to determine the mapping from the second coordinate system to the navigation space. The third correlation may enable the conversion of any coordinate in the second coordinate system to any coordinate in the navigation space, or vice versa. For example, the third correlation may enable the control of the second robot using instructions referencing the navigation space (e.g., the navigation coordinate system), or it may enable the conversion of navigation coordinate system-based instructions to second coordinate system-based instructions so that the second robot moves inward within the second coordinate system to perform a task or procedure defined relative to the patient coordinate system. The third correlation may be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The third correlation may be transmitted via a network (such as cloud 164).

[0101] Method 400 further includes determining the correlation of the combination based on a first correlation, a second correlation, and a third correlation (step 416). Determining the correlation of the combination may include using the first correlation between the patient coordinate system and the navigation space and the second correlation between the first coordinate system and the navigation space together to determine the mapping from the patient coordinate system to the first coordinate system, and / or vice versa. Determination may also include using the second correlation between the first coordinate system and the navigation space and the third correlation between the second coordinate system and the navigation space to determine the mapping from the first coordinate system to the second coordinate system, and / or vice versa. Determination may also include using the first correlation between the patient coordinate system and the navigation space and the third correlation between the second coordinate system and the navigation space together to determine the mapping from the patient coordinate system to the second coordinate system, and / or vice versa.

[0102] The correlation of the combination enables the transformation of any coordinate in the patient coordinate system, the first coordinate system, the second coordinate system, and / or the coordinate system corresponding to the navigation space into any other coordinate system among the aforementioned coordinate systems. The correlation of the combination also enables the control of the first and second robots using a common set of instructions prepared relative to the navigation space (e.g., the navigation coordinate system), and / or enables the transformation of instructions based on the navigation coordinate system or any other coordinate system into instructions based on the first and second coordinate systems, so that the first and second robots move appropriately, respectively, to perform a task or procedure defined relative to the patient coordinate system or the navigation coordinate system. The correlation of the combination can be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The correlation of the combination can be transmitted via a network (such as cloud 164).

[0103] This disclosure covers embodiments of method 400 that include more or fewer steps than those described above, and / or one or more steps that are different from those described above.

[0104] Turn now Figure 5For example, a method 500 for coordinating the control of multiple robots can be executed by at least one processor or otherwise performed. The at least one processor may be the same as or similar to processor 104 of the computing device 102 described above. The at least one processor may be part of a robot (such as robot 136) or a navigation system (such as navigation system 156). Processors other than any of the processors described herein may also be used to execute method 500. At least one processor may perform method 500 by executing instructions stored in memory (such as instruction 124 in memory 116). These instructions may correspond to one or more steps of method 500 described below. These instructions may cause the processor to execute one or more algorithms, such as algorithm 128. For example, once each coordinate system has been located, one or more such algorithms 128 may be used to map one coordinate system to another.

[0105] Method 500 includes receiving image information corresponding to information about the patient's anatomical elements (step 504). Step 504 may be the same as or similar to step 304 of method 300 described above.

[0106] Method 500 further includes receiving first sensor information regarding the patient reference frame and the posture of the first robot (step 508). The patient reference frame may be the same as or similar to reference frame 212, and the first robot may be, for example, [missing information - likely related to sensor information]. Figure 1 Robot 136 and / or Figures 2A-2B The first sensor information is identical or similar to robot 136a or robot 136b. The first sensor information can be any information used to determine the reference frame and the posture of the second robot at a single point in time. Thus, for example, the first sensor information may include image information received directly or indirectly from imaging devices such as CT scanners, magnetic resonance imaging (MRI) scanners, optical coherence tomography (OCT) scanners, O-arms (including, for example, O-arm 2D long-film scanners), C-arms, G-arms, another device utilizing X-ray-based imaging (e.g., a fluoroscope or other X-ray machine), or any other imaging equipment. Alternatively, the first sensor information may be or include information received directly or indirectly from an electromagnetic positioning or navigation system, which may include information about the detected positions of one or more magnets or other electromagnetic field sources from which the posture of the patient reference frame and the first robot can be determined.

[0107] In embodiments where the first sensor information includes image information, the image information may correspond to, for example, an X-ray image showing a patient reference frame and a plurality of tracking markers (e.g., tracking marker 220) of the first robot. The X-ray image may also show anatomical elements of the patient to which the patient reference frame is securely attached.

[0108] The first sensor information may also be or include information from one or more sensors of the first robot. For example, one or more sensors of the first robot may be used to determine (independent of any tracking marks on them) the posture of its robot arm. As another example, the navigation system may be used to detect any tracking marks attached to the robot arm of the first robot and / or other places on the first robot, from which the posture of the robot and / or the posture of its robot arm can be determined. Therefore, the first sensor information may be received from navigation cameras or other navigation system sensors.

[0109] Method 500 further includes determining a first correlation between the patient coordinate system and a first coordinate system of the first robot (step 512). The patient coordinate system may be the same as or similar to any patient coordinate system described herein, including, for example, step 404 of method 400. The first coordinate system may be the same as or similar to any first coordinate system described herein, including, for example, step 408 of method 400.

[0110] Determining the first correlation may include using one or more algorithms (such as algorithm 128) to determine the mapping from the patient coordinate system to the first coordinate system. The first correlation may enable the conversion of any coordinate in the patient coordinate system to any coordinate in the first coordinate system, or vice versa. For example, the first correlation may enable the control of a first robot using instructions referencing the patient coordinate system, or it may enable the conversion of patient-based instructions to first-based instructions so that the first robot can move within the first coordinate system to perform a task or procedure defined relative to the patient coordinate system. The first correlation may be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The first correlation may be transmitted via a network (such as cloud 164).

[0111] Method 500 further includes receiving second sensor information about the posture of the second robot from the first robot (step 516). The second robot may be, for example, Figure 1 Robot 136 and / or Figures 2A-2BThe second sensor information may be the same as or similar to robot 136a or robot 136b. The second sensor information may be any information used to determine the posture of the second robot relative to the posture of the first robot. Thus, for example, the second sensor information may include image information received directly or indirectly from sensors of the first robot (e.g., sensor 144). Such sensors may detect, for example, one or more tracking markers (e.g., tracking marker 220) on the second robot. In embodiments where the first robot has been physically connected to the second robot by component 224 or otherwise, the sensors may be or include one or more sensors configured to detect information about the physical connection from which the posture of the second robot relative to the first robot can be determined. Alternatively, the second sensor information may be or include information received directly or indirectly from an electromagnetic positioning or navigation system, which may include information about the detected location of one or more magnets or other electromagnetic field sources from which the posture of the second robot relative to the first robot can be determined.

[0112] Method 500 further includes determining a second correlation between the patient coordinate system and the second coordinate system of the second robot based on the first correlation and the second sensor information (step 520). The second coordinate system may be the same as or similar to any second coordinate system described herein, including, for example, step 412 of method 400.

[0113] Determining the second correlation may include using one or more algorithms (such as algorithm 128) to determine (again based on the first correlation and second sensor information) the mapping from the patient coordinate system to the second coordinate system. The first correlation may enable the conversion of any coordinate in the patient coordinate system to any coordinate in the first coordinate system, and the second sensor information may be used to determine the pose of the second robot relative to the first robot, thereby enabling the determination of an intermediate correlation between the first and second coordinate systems. The first correlation and the intermediate correlation may then be used to determine the second correlation. For example, once determined, the second correlation may enable the control of the second robot using instructions referencing the patient coordinate system, or it may enable the conversion of patient coordinate system-based instructions to second coordinate system-based instructions, thereby enabling the second robot to move within the second coordinate system to perform a task or procedure defined relative to the patient coordinate system. The second correlation may be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The second correlation may be transmitted via a network (such as cloud 164).

[0114] Method 500 further includes controlling the movement of the first and second robots based on a first correlation and a second correlation (step 524). Control may include causing the first and second robots (and more specifically, the robotic arms of the first and second robots, or even more specifically, the end effectors of the first and second robots) to work in a coordinated manner to perform a given surgical task (e.g., making an incision in the patient's skin) or a given surgical procedure (e.g., spinal decompression) defined with respect to a patient coordinate system. Control may be based, for example, on a preoperative plan and may utilize one or more instructions (such as instructions 124 stored in memory, such as memory 116).

[0115] This disclosure covers embodiments of method 500 that include more or fewer steps than those described above, and / or include one or more steps different from those described above. For example, in some embodiments, method 500 may further include determining one or more correlations between a patient coordinate system, a first coordinate system, and / or a second coordinate system on one side and a navigation coordinate system on the other side. Moreover, in some embodiments, method 500 may include determining the correlation between each possible pair of coordinate systems, and not just a first correlation between the patient coordinate system and a first coordinate system of a first robot and a second correlation between the patient coordinate system and a second coordinate system of a second robot.

[0116] Figure 6 The steps of method 600, which can be implemented, for example, by a robot control system according to embodiments of the present disclosure, are described. Method 600 is the same as or substantially similar to method 300 (e.g., step 604 is the same as or substantially similar to step 304, step 608 is the same as or substantially similar to step 308, step 612 is the same as or substantially similar to step 312, and step 616 is the same as or substantially similar to step 316). In particular, the received sensor information can be any sensor information sufficient (whether alone or in combination with image information) to establish a correlation between the patient coordinate system, the first coordinate system of the first robot, and the second coordinate system of the second robot, as described in step 612. In some embodiments, the sensor information received in step 608 can be or include first sensor information regarding the first robot and the patient reference system (e.g., sensor information sufficient, whether alone or in combination with image information, to establish a correlation between the first coordinate system and the patient coordinate system) and second sensor information regarding the second robot and the patient reference system (e.g., sensor information sufficient, whether alone or in combination with image information, to establish a correlation between the second coordinate system and the patient coordinate system).

[0117] Figure 7The steps of method 700 are described, which represents a way of determining the correlation between the patient coordinate system, the first coordinate system of the first robot, and the second coordinate system of the second robot during step 612 of method 600.

[0118] Method 700 includes determining a first correlation between the patient coordinate system and a first coordinate system of the first robot (step 704). Step 704 may be the same as or similar to step 512 of method 500. Step 704 may be based on, for example, image information (such as image information received in step 604 of method 600) and sensor information (such as sensor information received in step 608 of method 600). Where the sensor information includes first sensor information regarding the first robot and the patient reference system and second sensor information regarding the second robot and the patient reference system, step 704 may be based on the first sensor information.

[0119] Method 700 further includes determining a second correlation between the patient coordinate system and the second coordinate system (step 708). Step 708 may be the same as or similar to step 512 of method 500, wherein the second coordinate system is used instead of the first coordinate system. Step 708 may be based on, for example, image information (such as image information received in step 604 of method 600) and sensor information (such as sensor information received in step 608 of method 600). Where the sensor information includes first sensor information about a first robot and a patient reference system and second sensor information about a second robot and a patient reference system, step 704 may be based on the second sensor information.

[0120] Alternatively, step 708 may be the same as or similar to step 520 of method 500, wherein the second correlation is determined based on image information (such as image information received in step 604 of method 600) and / or on sensor information received from the first robot regarding the posture of the second robot, together with information regarding the first correlation determined in step 704.

[0121] Method 700 further includes determining the correlation of the combination based on a first correlation and a second correlation (step 712). Determining the correlation of the combination may include using the first correlation between the patient coordinate system and the first coordinate system and the second correlation between the patient coordinate system and the second coordinate system to determine the mapping from the first coordinate system to the second coordinate system, and / or vice versa. Determination may also include using the second correlation between the first coordinate system and the navigation space and the third correlation between the second coordinate system and the navigation space to determine the mapping from the first coordinate system to the second coordinate system, and / or vice versa.

[0122] The correlation of the combination enables the transformation of any coordinate in any of the patient coordinate system, the first coordinate system, and the second coordinate system into any other coordinate system among the aforementioned coordinate systems. The correlation of the combination also enables the control of the first and second robots using a common set of instructions prepared relative to patient space (e.g., the patient coordinate system), and / or enables the transformation of patient coordinate system-based instructions into instructions based on the first and second coordinate systems, so that the first and second robots move appropriately to perform a task or procedure defined relative to the patient coordinate system. The correlation of the combination can be stored, for example, in memory (such as memory 116), in a database (such as database 160), or elsewhere. The correlation of the combination can be transmitted via a network (such as cloud 164).

[0123] This disclosure covers embodiments of method 700 that include more or fewer steps than those described above, and / or one or more steps that are different from those described above.

[0124] Step 612 can be performed using method 400, method 700, any variation of the aforementioned methods, or another method. Similarly, step 312 can be performed using method 400, method 700, any variation of the aforementioned methods, or another method.

[0125] Although embodiments of this disclosure have been described with respect to the use of two separate robots, this disclosure also covers embodiments using three, four, five, six, seven, or more separate robots. In such embodiments, the methods described herein can be readily modified to allow the coordinate system of each robot to be associated with a patient coordinate system, a navigation coordinate system, and / or other robot coordinate systems.

[0126] As can be understood from the foregoing disclosure, this disclosure covers those with a greater than Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 The methods with fewer steps identified in the text (and the corresponding descriptions of methods 300, 400, 500, 600, and 700), and including all steps other than those listed below. Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7Methods involving steps other than those identified herein (and corresponding descriptions of methods 300, 400, 500, 600, and 700). This disclosure also covers methods including one or more steps from one method described herein and one or more steps from another method described herein. Any correlation described herein may be or includes registration or any other correlation. Any method described herein that includes determining one or more correlations between or among the patient coordinate system, the first coordinate system of the first robot, and the second coordinate system of the second robot may also include determining one or more correlations between any of the aforementioned coordinate systems and the navigation coordinate system. Similarly, any method described herein that includes determining one or more correlations between or among the patient coordinate system, the navigation coordinate system, the first coordinate system of the first robot, and the second coordinate system of the second robot may alternatively include determining only one or more correlations between or among the patient coordinate system, the first coordinate system, and the second coordinate system.

[0127] The foregoing is not intended to limit this disclosure to the one or more forms disclosed herein. In the foregoing specific embodiments, for the purpose of simplifying this disclosure, various features of this disclosure are grouped together in one or more aspects, embodiments, and / or configurations. Features of aspects, embodiments, and / or configurations of this disclosure may be combined in alternative aspects, embodiments, and / or configurations other than those discussed above. The approach of this disclosure should not be construed as reflecting an intention that the claims require more features than expressly recited in each claim. Rather, as reflected in the following claims, aspects of the invention lie in fewer than all the features of a single foregoing aspect, embodiment, and / or configuration. Therefore, the following claims are hereby incorporated into this specific embodiment, wherein each claim exists independently as a separate preferred embodiment of this disclosure.

[0128] Furthermore, although the description has included descriptions of one or more aspects, embodiments, and / or configurations, as well as certain variations and modifications, other variations, combinations, and modifications are also within the scope of this disclosure upon understanding it, for example, as may be within the skill and knowledge of one skilled in the art. It is intended to obtain the right to include alternative aspects, embodiments, and / or configurations to the permissible extent, including claimed alternatives, replacements, and / or equivalent structures, functions, scopes, or steps, regardless of whether such alternatives, replacements, and / or equivalent structures, functions, scopes, or steps are disclosed herein, and it is not intended for use with any patentable subject matter.

Claims

1. A robot control system comprising at least one processor configured to perform a registration method, the registration method comprising: Receive image information corresponding to the patient's anatomical elements; Receive first sensor information about the patient's reference frame and the posture of the first robot; Based on the image information and the first sensor information, a first correlation is determined between the patient's coordinate system and the first coordinate system of the first robot; Receive second sensor information about the posture of the second robot from the first robot; Based on the first correlation and the second sensor information, the intermediate correlation between the first coordinate system of the first robot and the second coordinate system of the second robot is determined; Based on the first correlation and the intermediate correlation, a second correlation is determined between the patient coordinate system and the second coordinate system of the second robot; as well as The movement of the first robot and the second robot within a common working volume is controlled based on the first correlation and the second correlation.

2. The robot control system of claim 1, wherein the patient reference frame is securely fixed to the anatomical element of the patient.

3. The robot control system of claim 1, wherein the first sensor information includes information about at least one tracking mark positioned on the first robot.

4. The robot control system according to claim 3, wherein the at least one tracking marker is a light-emitting diode.

5. The robot control system of claim 1, wherein the postures of the first robot and the second robot are such that the first robot and the second robot are in contact with the patient reference frame.

6. The robot control system of claim 1, wherein the first sensor information and the second sensor information are received from the navigation camera.

7. The robot control system of claim 1, wherein each of the patient coordinate system, the first coordinate system, and the second coordinate system is unique relative to the other coordinate systems.

8. The robot control system of claim 1, wherein controlling the movement of the first robot and the second robot within the common working volume based on the first correlation and the second correlation comprises causing the first robot and the second robot to move in a coordinated manner to complete a surgical task.

9. The robot control system according to claim 1, wherein the first sensor information and the second sensor information include information from at least two sensors.

10. The robot control system of claim 1, wherein the second sensor information includes information about the posture of the second robot relative to the first robot.

11. A robot control system comprising at least one processor configured to perform a method for coordinating the control of a plurality of robots, the method comprising: Receive image information corresponding to the patient's anatomical elements; Receive first sensor information about the patient's reference frame and the posture of the first robot; A first correlation is determined based on the image information and the first sensor information, wherein the first correlation is between the patient coordinate system and the first robot's first coordinate system. Receive second sensor information about the posture of the second robot from the first robot; An intermediate correlation is determined based on the second sensor information and the first correlation, wherein the intermediate correlation lies between the first coordinate system of the first robot and the second coordinate system of the second robot; Based on the first correlation and the intermediate correlation, a second correlation is determined, the second correlation being between the patient coordinate system and the second robot's second coordinate system. Control the first robot and the second robot based on the first correlation and the second correlation. Each of the patient coordinate system, the first coordinate system, and the second coordinate system is unique relative to the other coordinate systems.

12. The robot control system of claim 11, wherein the second sensor information includes information about the physical connection between the first robot and the second robot.

13. The robot control system of claim 11, wherein the second sensor information is received from the imaging sensor of the first robot.

14. The robot control system of claim 11, wherein the control includes coordinating the movement of the first robot and the second robot to complete the surgical task.

15. A robot control system, the robot control system comprising: A communication interface is provided for communicating with multiple robots, including a first robot and a second robot, wherein the first robot has a first coordinate system that is independent of the second robot's second coordinate system. Multiple tracking markers are positioned on the first robot and the second robot; At least one sensor is provided on the first robot and configured to detect the position of the second robot by detecting the position of a tracking marker on the second robot; At least one processor; as well as At least one memory, the at least one memory storing instructions for execution by the at least one processor, the instructions being configured, when executed, to cause the at least one processor to: Receive sensor information about a patient reference frame, the first robot, and the second robot, wherein the sensor information includes first sensor information and second sensor information, the first sensor information being about the first robot and the patient reference frame, and the second sensor information being received from the at least one sensor provided on the first robot; Based on image information corresponding to the anatomical elements of the patient and the first sensor information, a first correlation is determined between the patient coordinate system and the first coordinate system of the first robot; Based on the first correlation and the second sensor information, the intermediate correlation between the first coordinate system of the first robot and the second coordinate system of the second robot is determined. Based on the first correlation and the intermediate correlation, a second correlation is determined between the patient coordinate system and the second coordinate system of the second robot; as well as The movement of the first robot and the second robot is controlled based on the first correlation and the second correlation.

16. The robot control system of claim 15, wherein the control includes coordinating the movement of the first robot and the second robot to complete a surgical task.

17. The robot control system of claim 15, wherein the at least one sensor includes a navigation camera.

18. The robot control system of claim 15, wherein the at least one memory stores additional instructions for execution by the at least one processor, the additional instructions, when executed, further causing the at least one processor to: control the movement of the first robot and the second robot within a common working volume based on the first correlation and the second correlation.

Images