Pre-docking pose optimization

Abstract

The robotic medical system may recommend a pose for the robotic arm based on patient and medical procedure information and communicate the recommended pose to a user of the robotic medical system. The robotic medical system may include a robotic arm and a processor. The robotic medical system may be configured to acquire images of the patient, simulate the medical procedure to identify collisions, and select a pose based on the number of collisions. The robotic medical system may provide information indicative of the recommended pose.

Description

[Technical field]

[0001] The systems and methods disclosed herein are directed to configuring robotic medical systems, and more specifically, robotically controlled arms of robotic medical systems for medical procedures. [Background technology]

[0002] Robot-enabled medical systems can perform a variety of medical procedures, including both minimally invasive procedures such as laparoscopy, and non-invasive procedures such as endoscopic procedures (e.g., bronchoscopy, ureteroscopy, gastroscopy, etc.).

[0003] Such a robotic medical system may include a robotic arm configured to control the movement of a medical tool during a given medical procedure. To achieve a desired positioning of the medical tool, the robotic arm holding the medical tool can be placed in a specific pose during a set-up process of the robotic medical system or during remote operation. Summary of the Invention [Means for solving the problem]

[0004] Surgical procedures can benefit from precise port location and generally have a small margin of error. For example, port placement may need to be within about 1.5 centimeters of the optimal position in large patients and within about 1 centimeter in small patients. Achieving such precise port placement that satisfies the small margin of error can be difficult.

[0005] Additionally, different surgical procedures may involve different cannula and / or port locations. Although traditional port planning often involves the physician's inherent knowledge of the type of procedure to be performed, this planning may not take into account the entire patient area and possible collisions that may occur between the arm and external objects based on the port settings.

[0006] Robotic medical systems that include multiple robotic arms (e.g., four or more robotic arms) can provide surgeons with more flexibility to perform medical procedures. They can also enable the development of novel surgical procedures and techniques. However, such complex systems require the robotic arms to be carefully positioned during setup to avoid intraoperative collisions and still be able to reach the target anatomical structures during surgery. In particular, there is a need for a robotic medical system that can identify precise locations for port placement and corresponding optimal starting positions for the robotic arms.

[0007] As disclosed herein, a robotic medical system can be configured to select an initial robotic arm posture and corresponding port locations for a given patient and procedure. In some embodiments, the selected initial robotic arm posture and corresponding port locations are recommended or optimized initial robotic arm postures and corresponding port locations. Some embodiments disclosed herein combine the recommended port placement data with information about (i) the patient presentation, such as height, weight, description of abdominal (chest) size and shape, and surgical history (e.g., indicating visceral changes or scarring), and (ii) the surgical procedure, to generate a pre-docking posture of the robotic arm.

[0008] Some embodiments disclosed herein use a three-dimensional (3D) scanner (e.g., a scanning unit) communicatively connected to the robotic medical system to determine a recommended initial robotic arm pose (e.g., pre-docking pose) and corresponding port locations. The 3D scanner may be included with the robotic medical system (e.g., is a component of, forms part of, etc.). In some configurations, the 3D scanner may be a component of the robotic medical system located on the robotic arm, patient support platform (e.g., bed), or tower pendant of the robotic medical system, either as an integrated component or an attachment. In some other configurations, the 3D scanner may be part of an instrument (e.g., a medical tool such as a laparoscope) held by the robotic arm of the robotic medical system. Alternatively, the 3D scanner may be a separate accessory (e.g., a stand-alone scanner, a handheld scanner, etc.) communicatively connected to the robotic medical system and attached (e.g., mounted) to the robotic arm, patient support platform (e.g., bed), tower pendant of the robotic medical system, or the wall and / or ceiling of the operating room in which the robotic medical system is located. The 3D scanner can provide information such as patient view, patient characteristics, and / or position of attached accessories, which can be used by the robotic medical system (e.g., in combination with other information such as the target procedure and / or anatomical structure) to determine a recommended initial robotic arm pose (e.g., pre-docking pose) and corresponding port locations specifically tailored for the patient and / or medical procedure being performed.

[0009] As disclosed herein, a 3D scanner can be used to localize a patient relative to a robotic medical system. Although the robotic medical system can determine the position and / or orientation of the patient support platform and / or robotic arm of the robotic medical system (e.g., using sensors and / or encoders attached to the robotic medical system), it may be difficult for the robotic medical system to register other objects such as the patient and attached accessories, including but not limited to liver retractors, leg stirrups, arm boards, anesthesia accessories, etc. Thus, localizing the patient relative to the robotic medical system using a 3D scanner can provide the robotic medical system with information regarding the position of the patient and / or attached accessories.

[0010] As disclosed herein, data from the 3D scanner can be combined with other information / inputs, such as information about the target procedure and / or the target anatomy, to generate recommended initial robotic arm poses and corresponding port locations. For example, the scan data (e.g., imaging data, 3D imaging data, etc.) can include real-time patient information, such as patient size, patient characteristics (e.g., whether the patient is an amputee), and relevant objects (e.g., if there is a feeding tube exiting the patient, this can define a "keep out" zone). The robotic medical system can segment the patient into different parts (e.g., head, torso, legs), and using the location of markers (e.g., the patient's abdomen) and / or relevant organ locations for a particular surgical procedure, the robotic medical system (e.g., via an algorithm) can recommend port placements and starting poses that maximize the available workspace while avoiding or minimizing collisions, such as collisions between the patient and the robotic arms, between the robotic arms, and / or between instruments, during the procedure.

[0011] Thus, in one aspect, a robotic medical system includes a plurality of robotic arms, one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: (i) acquire imaging data of a patient; (ii) acquire information related to a medical procedure of the patient; (iii) simulate the medical procedure from a start state to identify, for each start state of the plurality of start states, a number of collisions that occur with an arm of the plurality of robotic arms, where the simulating uses the imaging data and the information related to the medical procedure; (iv) select a first start state of the plurality of start states, where the first start state has a minimum number of collisions, the first start state identifying a respective start posture of each arm of the plurality of robotic arms; and (v) provide posture information for the first start state to an operator of the robotic medical system.

[0012] In another aspect, a method for determining a starting state of a robotic system including a plurality of robotic arms includes: (i) acquiring imaging data of a patient; (ii) acquiring information related to a medical procedure of the patient; (iii) simulating the medical procedure from the starting states to identify, for each starting state of the plurality of starting states, a number of collisions that occur with an arm of the plurality of robotic arms, where the simulating uses the imaging data and the information related to the medical procedure; (iv) selecting a first starting state of the plurality of starting states, where the first starting state has a minimum number of collisions, the first starting state identifying a respective starting pose of each arm of the plurality of robotic arms; and (v) providing pose information for the first starting state to an operator of the robotic medical system.

[0013] The systems, methods, and devices of the present disclosure each have several innovative aspects, no single one of which is solely responsible for the desirable attributes disclosed herein. [Brief description of the drawings]

[0014] The disclosed aspects are hereinafter described in conjunction with the accompanying drawings, which illustrate, but do not limit, the disclosed aspects, and in which like designations refer to like elements and in which: [Figure 1] 1 illustrates one embodiment of a cart-based robotic system positioned for a diagnostic and / or therapeutic bronchoscopic procedure, according to some embodiments. [Diagram 2] 2 depicts further aspects of the robotic system of FIG. 1, according to some embodiments. [Diagram 3] 2 illustrates an embodiment of the robotic system of FIG. 1 positioned for ureteroscopy, according to some embodiments. [Figure 4] 2 illustrates an embodiment of the robotic system of FIG. 1 positioned for a vascular procedure, according to some embodiments. [Diagram 5] 1 illustrates one embodiment of a table-based robotic system positioned for a bronchoscopic procedure, according to some embodiments. [Figure 6] 1 illustrates an example system configured to accommodate a robotic arm, according to some implementations. [Figure 7] 1 illustrates an alternative embodiment of a table-based robotic system, according to some embodiments. [Figure 8] 8 illustrates an end view of the table-based robotic system of FIG. 7, in accordance with some embodiments. [Figure 9] FIG. 1 illustrates an end view of a table-based robotic system with an attached robotic arm, according to some embodiments. [Figure 10] 1 illustrates an example instrument driver, according to some embodiments. [Figure 11] 1 illustrates an exemplary medical instrument having paired instrument drivers, according to some embodiments. [Figure 12] 13 illustrates an alternative design for the instrument driver and instrument, where the axis of the drive unit is parallel to the axis of the elongated shaft of the instrument, according to some embodiments. [Figure 13] 1 illustrates an instrument having an instrument-based insertion architecture, according to some embodiments. [Figure 14] 1 illustrates an example controller, according to some embodiments. [Figure 15] 1 illustrates an exemplary robotic system, according to some embodiments. [Figure 16] 1 illustrates another view of an exemplary robotic system, according to some embodiments. [Figure 17A] 1 illustrates different views of an exemplary robotic arm, according to some embodiments. [Figure 17B] 1 illustrates different views of an exemplary robotic arm, according to some embodiments. [Figure 17C] 1 illustrates different views of an exemplary robotic arm, according to some embodiments. [Figure 18A] 1 illustrates exemplary cannula placements for different surgical procedures, according to some embodiments. [Figure 18B] 1 illustrates exemplary cannula placements for different surgical procedures, according to some embodiments. [Figure 18C] 1 illustrates exemplary cannula placements for different surgical procedures, according to some embodiments. [Figure 19] 1 illustrates an example workflow for generating attitude and port information, according to some embodiments. [Figure 20A] 1 illustrates an exemplary 3D scanning and registration process for a robotic medical system, according to some embodiments. [Figure 20B] 1 illustrates an exemplary 3D scanning and registration process for a robotic medical system, according to some embodiments. [Figure 21A] 1 illustrates an exemplary process for identifying a target anatomical structure location, according to some embodiments. [Figure 21B] 1 illustrates an exemplary process for identifying a target anatomical structure location, according to some embodiments. [Figure 22A] 1 illustrates an exemplary set of port locations for a medical procedure, according to some embodiments. [Figure 22B] 22B illustrates example arm postures for the set of port locations of FIG. 22A, according to some embodiments. [Figure 22C] 22B illustrates example arm postures for the set of port locations of FIG. 22A, according to some embodiments. [Figure 22D] 22B illustrates example arm postures for the set of port locations of FIG. 22A, according to some embodiments. [Figure 22E] 1 illustrates another exemplary set of port locations for a medical procedure, according to some embodiments. [Figure 22F] 22F illustrates example arm postures for the set of port locations of FIG. 22E, according to some embodiments. [Figure 22G] 22F illustrates example arm postures for the set of port locations of FIG. 22E, according to some embodiments. [Fig. 22H] 22F illustrates example arm postures for the set of port locations of FIG. 22E, according to some embodiments. [Figure 23A] 1 is a flowchart illustrating an example method for selecting a robot arm pose, in accordance with some embodiments. [Figure 23B] 1 is a flowchart illustrating an example method for selecting a robot arm pose, in accordance with some embodiments. [Figure 24] FIG. 2 is a schematic diagram illustrating electronic components of a medical system, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] 1. Overview. Aspects of the present disclosure may be integrated into a robot-enabled medical system capable of performing a variety of medical procedures, including both minimally invasive, such as laparoscopy, and non-invasive, such as endoscopic procedures. Among endoscopic procedures, the system may be capable of performing bronchoscopy, ureteroscopy, gastroscopy, and the like.

[0016] In addition to performing a wide range of procedures, the system may provide additional benefits such as enhanced imaging and guidance to assist the physician. Furthermore, the system may provide the physician with the ability to perform procedures with improved ease of use, such that a single user may control one or more of the instruments of the system.

[0017] Various embodiments are described below, for purposes of illustration, in conjunction with the drawings, and are intended to be illustrative, and not to be limiting, and are not to be construed as limiting the present invention.

[0018] A. Robotic System - Cart. A robot-enabled medical system may be configured in a variety of ways depending on the particular procedure. FIG. 1 illustrates one embodiment of a cart-based robot-enabled system 10 arranged for diagnostic and / or therapeutic bronchoscopy procedures. During bronchoscopy, the system 10 may include a cart 11 having one or more robotic arms 12 for delivering medical instruments, such as a steerable endoscope 13, which may be a procedure-specific bronchoscope for bronchoscopy, to a natural orifice access point (i.e., the mouth of a patient positioned on a table in this example) for delivering diagnostic and / or therapeutic tools. As shown, the cart 11 may be positioned adjacent to the patient's upper torso to provide access to the access point. Similarly, the robotic arms 12 may be actuated to position a bronchoscope relative to the access point. The arrangement of FIG. 1 may also be utilized when performing gastro-intestinal (GI) procedures using a gastroscope. FIG. 2 depicts an example embodiment of the cart in more detail.

[0019] With continued reference to FIG. 1, once the cart 11 is properly positioned, the robotic arm 12 may insert the steerable endoscope 13 into the patient robotically, manually, or a combination thereof. As shown, the steerable endoscope 13 may include at least two nested parts, such as an inner leader portion and an outer sheath portion, each portion coupled to a separate instrument driver from a set of instrument drivers 28, each instrument driver coupled to the distal end of an individual robotic arm. This linear arrangement of the instrument drivers 28, which facilitates coaxial alignment of the leader portion with the sheath portion, creates a "virtual rail" 29 that can be repositioned in space by manipulating one or more robotic arms 12 to different angles and / or positions. The virtual rails described herein are depicted in the figures using dashed lines (as in FIGS. 3 and 4), and thus the dashed lines do not depict any physical structure of the system. Translation of the instrument driver 28 along the virtual rail 29 nests the inner leader portion relative to the outer sheath portion, or advances or retracts the endoscope 13 from the patient. The angle of the virtual rail 29 may be adjusted, translated, or pivoted based on clinical application or physician preference. For example, in bronchoscopy, the angle and position of the virtual rail 29 shown represents a compromise between providing the physician access to the endoscope 13 and minimizing friction that results from bending the endoscope 13 into the patient's mouth.

[0020] The endoscope 13 can be directed downstream of the patient's trachea and lungs after insertion using precise commands from the robotic system until the target destination or surgical site is reached. To enhance navigation through the patient's pulmonary network and / or reach a desired target, the endoscope 13 can be manipulated to telescope the inner leader portion from the outer sheath portion and provide enhanced articulation and a larger bend radius. The use of a separate instrument driver 28 also allows the leader and sheath portions to be driven independently of one another.

[0021] For example, the endoscope 13 may be oriented to deliver a biopsy needle to a target, such as a lesion or nodule in the patient's lung. The needle may be deployed downstream of a working channel that runs the length of the endoscope to obtain a tissue sample that is analyzed by a pathologist. Depending on the results of the pathology, additional tools may be deployed downstream of the working channel of the endoscope for additional biopsies. After identifying a nodule as malignant, the endoscope 13 may deliver tools endoscopically to account for potential cancerous tissue. In some cases, diagnostic and therapeutic treatments may be delivered in separate procedures. In other cases, diagnostic and therapeutic treatments may be delivered during the same procedure.

[0022] The system 10 may also include a movable tower 30 that may be connected to the cart 11 via a support cable and provide support for control, electronics, fluidics, optics, sensors, and / or power to the cart 11.

[0023] To support the robotic system described above, the tower 30 may include computer-based control system components that store computer program instructions in a non-transitory computer-readable storage medium, such as, for example, a persistent magnetic storage drive, a solid-state drive, or the like. Execution of these instructions, whether execution occurs in the tower 30 or in the cart 11, may control the entire system or subsystems thereof. For example, when executed by a processor in the computer system, the instructions may cause the robotic system components to actuate associated carriages and arm mounts, operate a robotic arm, and control a medical instrument. For example, in response to receiving control signals, motors in the joints of the robotic arm may position the arm in a particular pose.

[0024] The tower 30 may also include pumps, flow meters, valve controls, and / or fluid access to provide controlled irrigation and aspiration capabilities to a system that may be deployed through the endoscope 13. These components may also be controlled using the computer system of the tower 30. In some embodiments, irrigation and aspiration capabilities may be provided directly to the endoscope 13 via separate cables.

[0025] The tower 30 may also include support equipment for sensors deployed throughout the robotic system 10. For example, the tower 30 may include optoelectronic equipment for detecting, receiving, and processing data received from optical sensors or cameras anywhere in the robotic system 10. In combination with a control system, such optoelectronic equipment may be used to generate real-time images for display in any number of consoles deployed throughout the system, including in the tower 30. Similarly, the tower 30 may also include electronic subsystems for receiving signals from and processing received electromagnetic (EM) sensors deployed. The tower 30 may also be used to house and position EM field generators for detection by EM sensors in or on the medical instrument.

[0026] The tower 30 may also include a console 31 in addition to other consoles available to the rest of the system, for example a console mounted on top of a cart. The console 31 may include a user interface and a display screen, such as a touch screen, for the operator physician. The consoles in the system 10 are generally designed to provide both pre-operative and real-time information of the procedure, such as robotic control and navigation and localization information for the endoscope 13. If the console 31 is not the only console available to the physician, it may be used by a second operator, such as a nurse, to monitor the patient's health or vitals and the operation of the system 10, as well as to provide procedure-specific data, such as navigation and localization information. In other embodiments, the console 31 is housed in a body separate from the tower 30.

[0027] FIG. 2 provides a detailed illustration of one embodiment of a cart 11 from the cart-based robot-enabled system shown in FIG. 1. The cart 11 generally includes an elongated support structure 14 (often referred to as a "column"), a cart base 15, and a console 16 at the top of the column 14. The column 14 may include one or more carriages, such as a carriage 17 (alternatively an "arm support") for supporting the deployment of one or more robotic arms 12 (three are shown in FIG. 2). The carriage 17 may include individually configurable arm mounts that rotate along orthogonal axes to adjust the base of the robotic arms 12 for better positioning relative to the patient. The carriage 17 also includes a carriage interface 19 that allows the carriage 17 to translate vertically along the column 14.

[0028] The carriage interface 19 is connected to the column 14 through slots, such as slots 20 positioned on either side of the column 14 to guide the vertical translation of the carriage 17. The slots 20 include vertical translation interfaces for positioning and holding the carriage 17 at various vertical heights relative to the cart base 15. The vertical translation of the carriage 17 allows the cart 11 to adjust the reach of the robotic arm 12 to meet various table heights, patient sizes, and physician preferences. Similarly, an individually configurable arm mount on the carriage 17 allows the robotic arm base 21 of the robotic arm 12 to be angled in various configurations.

[0029] Column 14 may contain mechanisms therein, such as gears and motors, designed to use a vertically aligned leadscrew to mechanically translate carriage 17 in response to control signals generated in response to user input, such as input from console 16.

[0030] The robotic arm 12 may generally include a robotic arm base 21 and an end effector 22 separated by a series of linkages 23 connected by a series of joints 24, each joint including an independent actuator, each actuator including an independently controllable motor. Each independently controllable joint represents an independent degree of freedom available to the robotic arm 12. Each of the robotic arms 12 may have seven joints, thus providing seven degrees of freedom. A large number of joints provides a large number of degrees of freedom, allowing for "redundant" degrees of freedom. Redundant degrees of freedom allow the robotic arm 12 to position its respective end effector 22 at a specific position, orientation, and trajectory in space using different linkage positions and joint angles. This allows the system to position and orient the medical instrument from a desired point in space, while allowing the physician to move the arm joints to a clinically advantageous position away from the patient to create greater access while avoiding arm collisions.

[0031] The cart base 15 balances the weight of the column 14, carriage 17, and robotic arm 12 on the floor. Thus, the cart base 15 houses the heavier components such as electronics, motors, power supplies, and components that allow either movement and / or immobilization of the cart 11. For example, the cart base 15 includes casters 25 in the form of rollable wheels that allow the cart 11 to be easily moved around the room before a procedure. After reaching the proper position, the casters 25 can be immobilized using wheel locks to hold the cart 11 in place during a procedure.

[0032] A console 16 positioned at the vertical end of column 14 allows for both a user interface and a display screen (or dual-purpose device such as, for example, a touch screen 26) to receive user input and provide both pre-operative and intra-operative data to the physician user. Potential pre-operative data on the touch screen 26 may include pre-operative planning, navigation and mapping data derived from a pre-operative computed tomography (CT scan), and / or notes from a pre-operative patient interview. Intra-operative data on the display may also include vital patient statistics such as respiration, heart rate, and / or pulse, along with optical information provided from tools, sensor information from sensors, and coordinate information.

[0033] 3 illustrates one embodiment of the robot-enabled system 10 positioned for ureteroscopy. In a ureteroscopy procedure, the cart 11 can be positioned to deliver a ureteroscope 32, a procedure-specific endoscope designed to follow the patient's urethra and ureters, to the patient's lower abdominal region. As shown, the cart 11 can be aligned at the table leg to allow the robotic arm 12 to position the ureteroscope 32 for direct linear access to the patient's urethra. From the table leg, the robotic arm 12 can insert the ureteroscope 32 along a virtual rail 33 directly through the urethra and into the patient's lower abdomen.

[0034] After insertion into the urethra using control techniques similar to those in bronchoscopy, the ureteroscope 32 may be navigated to the bladder, ureters, and / or kidneys for diagnostic and / or therapeutic applications. For example, the ureteroscope 32 may be directed to the ureter and kidney to fragment formed kidney stones using a laser or ultrasonic lithotripsy device deployed downstream of the working channel of the ureteroscope 32. After lithotripsy is complete, the resulting stone fragments may be removed using a basket deployed downstream of the ureteroscope 32.

[0035] FIG. 4 illustrates one embodiment of a robot-enabled system 10 similarly positioned for a vascular procedure. In a vascular procedure, the system 10 may be configured such that the cart 11 may deliver a medical instrument 34, such as a steerable catheter, to an access point in the femoral artery in the patient's leg. As in a ureteroscopy procedure, the cart 11 may be positioned toward the patient's leg and lower abdomen to allow the robotic arm 12 to provide a virtual rail 35 with direct linear access to the femoral artery access point in the patient's thigh / hip region. After insertion into the artery, the medical instrument 34 may be oriented and inserted by translating the instrument driver 28. Alternatively, the cart may be positioned around the patient's upper abdomen to reach alternative vascular access points, such as the carotid and brachial arteries near the shoulder and wrist.

[0036] B. Robot system-table. An embodiment of a robot-enabled medical system may also incorporate a patient table. Incorporating a table may reduce the amount of capital equipment in the operating room by eliminating carts, thereby allowing better access to the patient. FIG. 5 illustrates an embodiment of such a robot-enabled system deployed for a bronchoscopy procedure. The system 36 includes a support structure or column 37 for supporting a platform 38 (shown as a "table" or "bed") above the overall floor. Much like a cart-based system, the end effector of the robotic arm 39 of the system 36 includes an instrument driver 42 that is designed to manipulate an elongated medical instrument, such as the bronchoscope 40 of FIG. 5, through or along a virtual rail 41 formed from the linear alignment of the instrument driver 42. In practice, a C-arm for providing fluoroscopic imaging may be positioned above the upper abdominal region of the patient by placing emitters and detectors around the table 38.

[0037] As shown, the column 37 may include one or more carriages 43, shown in the system 36 as ring-shaped, that may be the base for one or more robotic arms 39. The carriages 43 may translate along a vertical column interface 44 that extends the length of the column 37 to provide different viewing positions from which the robotic arms 39 may be positioned to reach the patient. The carriages 43 may rotate about the column 37 using mechanical motors positioned within the column 37, allowing the robotic arms 39 to have access to multiple sides of the table 38, such as, for example, both sides of the patient.

[0038] The system 36 may also include a tower (not shown) that divides the functions of the system 36 between the table and the tower to reduce the form factor and bulk of the table. As with the previously disclosed embodiments, the tower may provide various support functions to the table, such as processing, computing, and control capabilities, power, fluidics, and / or optical and sensor processing. The tower may also be movable to be positioned away from the patient to improve physician access and keep the operating room clutter-free. The tower may also include a master controller or console that provides both a user interface for user input, such as a keyboard and / or pendant, and a display screen (or touch screen) for pre-operative and intra-operative information, such as real-time imaging, navigation, and tracking information. In some embodiments, the tower may also include a holder for a gas tank used for venting.

[0039] In some embodiments, the table base houses and stores the robot arm when not in use. Figure 6 illustrates a system 47 for housing the robot arm in one embodiment of a table-based system. In the system 47, the carriage 48 can be translated vertically into the base 49 to store the robot arm 50, arm mount 51, and carriage 48 therein. The base cover 52 can be translated and retracted open to allow the carriage 48, arm mount 51, and arm 50 to deploy about the column 53, and the base cover 52 can be closed to house and protect them when not in use.

[0040] 7 and 8 illustrate isometric and end views of one embodiment of a table-based surgical robotic system 100. The surgical robotic system 100 includes one or more adjustable arm supports 105 that can be configured to support one or more robotic arms relative to a table 101 (see, e.g., FIG. 9). In the illustrated embodiment, only one adjustable arm support 105 is shown, but additional arm supports can be provided on the opposite side of the table 101. The adjustable arm support 105 can be configured to move relative to the table 101 such that the position of the adjustable arm support 105 and / or any robotic arm attached thereto can be adjusted and / or changed relative to the table 101. For example, the adjustable arm support 105 can be adjusted with one or more degrees of freedom relative to the table 101. The adjustable arm support 105 provides the system 100 with great versatility, including the ability to easily accommodate one or more adjustable arm supports 105 and any robotic arms attached thereto under the table 101. The adjustable arm support 105 can be raised from a stowed position to a position below the top surface of the table 101. In other embodiments, the adjustable arm support 105 can be raised from a stowed position to a position above the top surface of the table 101.

[0041] The adjustable arm support 105 can provide several degrees of freedom, including lift, lateral translation, tilt, etc. In the illustrated embodiment of Figures 7 and 8, the arm support 105 is configured with four degrees of freedom, which are illustrated by arrows in Figure 7. The first degree of freedom allows adjustment of the adjustable arm support 105 in the z-direction ("Z-lift"). For example, the adjustable arm support 105 can include a carriage 109 configured to move up and down along or relative to the column 102 that supports the table 101. The second degree of freedom can allow the adjustable arm support 105 to tilt. For example, the adjustable arm support 105 can include a rotational joint, which can allow the adjustable arm support 105 to align with the bed in the Trendelenburg position. The third degree of freedom allows the adjustable arm support 105 to "pivot up," which can be used to adjust the distance between the side of the table 101 and the adjustable arm support 105. A fourth degree of freedom may allow translation of the adjustable arm support 105 along the longitudinal length of the table.

[0042] The surgical robotic system 100 of Figures 7 and 8 can include a table supported by a column 102 mounted to a base 103. The base 103 and column 102 support the table 101 against a support surface. A bed axis 131 and a support axis 133 are shown in Figure 8.

[0043] An adjustable arm support 105 can be mounted to the column 102. In other embodiments, the arm support 105 can be mounted to the table 101 or the base 103. The adjustable arm support 105 can include a carriage 109, a bar or rail connector 111, and a bar or rail 107. In some embodiments, one or more robot arms mounted to the rail 107 can translate and move relative to each other.

[0044] 9 illustrates an end view of a surgical robotic system 140A with two adjustable arm supports 105A, 105B mounted on either side of a table 101. A first robotic arm 142A is attached to a bar or rail 107A of the first adjustable arm support 105B. The first robotic arm 142A includes a base 144A attached to the rail 107A. The distal end of the first robotic arm 142A includes an instrument drive mechanism 146A that can be attached to one or more robotic medical instruments or tools. Similarly, the second robotic arm 142B includes a base 144B attached to the rail 107B. The distal end of the second robotic arm 142B includes an instrument drive mechanism 146B. The instrument drive mechanism 146B can be configured to attach to one or more robotic medical instruments or tools.

[0045] In some embodiments, one or more of the robotic arms 142A, 142B include arms with seven or more degrees of freedom. In some embodiments, one or more of the robotic arms 142A, 142B include eight degrees of freedom, including an insertion axis (one degree of freedom including insertion), wrist (three degrees of freedom including wrist pitch, yaw and roll), elbow (one degree of freedom including elbow pitch), shoulder (two degrees of freedom including shoulder pitch and yaw), and base 144A, 144B (one degree of freedom including translation). In some embodiments, the insertion degree of freedom can be provided by the robotic arms 142A, 142B, while in other embodiments, the instrument itself provides the insertion via an instrument-based insertion architecture.

[0046] C. Instrument drivers and interfaces. The end effector of the robotic arm of the system includes (i) an instrument driver (alternatively referred to as an "instrument drive mechanism" or "instrument device manipulator") incorporating electromechanical means for actuating the medical instrument, and (ii) a removable or detachable medical instrument, which may not include electromechanical components such as motors. This dichotomy may be caused by the need to sterilize medical instruments used in medical procedures and the inability to adequately sterilize expensive capital equipment due to the complex mechanical assembly and sensitive electronics of medical instruments. Thus, medical instruments may be designed to be detached, removed, and replaced from the instrument driver (and thus the system) by the physician or physician's staff for individual sterilization or disposal. In contrast, the instrument driver does not need to be replaced or sterilized and may be draped for protection.

[0047] FIG. 10 illustrates an exemplary instrument driver. An instrument driver 162 positioned at the distal end of the robotic arm includes one or more drive units 163 arranged on parallel axes to provide a controlled torque to a medical instrument via a drive shaft 164. Each drive unit 163 includes a separate drive shaft 164 for interacting with the instrument, a gearhead 165 for converting motor shaft rotation to a desired torque, a motor 166 for generating the drive torque, an encoder 167 for measuring the speed of the motor shaft and providing feedback to the control circuitry, and a control circuit 168 for receiving control signals and operating the drive unit. Each drive unit 163 is controlled and motorized independently of the others, and the instrument driver 162 may provide multiple (four shown in FIG. 11) independent drive outputs to the medical instrument. In operation, the control circuit 168 will receive the control signals and send motor signals to the motors 166, compare the resulting motor speed measured by the encoders 167 to a desired speed, and modulate the motor signals to generate the desired torque.

[0048] For procedures requiring a sterile environment, the robotic system may incorporate a drive joint, such as a sterile adapter connected to a sterile drape, located between the instrument driver and the medical instrument. The primary purpose of the sterile adapter is to transfer angular motion from the drive shaft of the instrument driver to the drive input of the instrument while maintaining physical separation between the drive shaft and the drive input, and thus sterility. Thus, an exemplary sterile adapter may include a set of rotational inputs and outputs intended to mate with the drive shaft of the instrument driver and a drive input for the instrument. The sterile drape, which is connected to the sterile adapter, is composed of a thin flexible material, such as a clear or translucent plastic, and is designed to cover the instrument driver, the robot arm, and capital equipment, such as a cart (in a cart-based system) or a table (in a table-based system). The use of the drape allows the capital equipment to be positioned in close proximity to the patient while still being located in an area that does not require sterility (i.e., the non-sterile field). On the other side of the sterile drape, the medical instrument may interact with the patient in an area that requires sterility (i.e., the sterile field).

[0049] D. Medical equipment. 11 illustrates an exemplary medical instrument with a mated instrument driver. Similar to other instruments designed for use with a robotic system, the medical instrument 170 includes an elongated shaft 171 (or elongated body) and an instrument base 172. The instrument base 172, also referred to as an "instrument handle" due to its design intended for manual interaction by a physician, may generally include a rotary drive input 173, e.g., a receptacle, pulley, or spool, designed to mate with a drive output 174 that passes through a drive interface on an instrument driver 175 at a distal end of a robotic arm 176. When physically connected, latched, and / or coupled, the mating drive input 173 of the instrument base 172 may share an axis of rotation with the drive output 174 in the instrument driver 175 to enable the transfer of torque from the drive output 174 to the drive input 173. In some embodiments, the drive output 174 may include a spline designed to mate with a receptacle on the drive input 173.

[0050] The elongated shaft 171 is designed to be delivered through either an anatomical opening or lumen, such as in an endoscope, or a minimally invasive incision, such as in a laparoscopy. The elongated shaft 171 can be either flexible (e.g., having properties similar to an endoscope) or rigid (e.g., having properties similar to a laparoscope), or can include a customized combination of both flexible and rigid portions. If designed for laparoscopy, the distal end of the rigid elongated shaft can be connected to an end effector extending from an articulating wrist formed from a clevis having at least one degree of freedom, and to a surgical tool or medical instrument, such as a grasper or scissors, that can be actuated based on a force from a tendon as the drive input rotates in response to a torque received from the drive output 174 of the instrument driver 175. If designed for endoscopy, the distal end of the flexible elongated shaft can include a steerable or controllable bend that can be articulated and bent based on a torque received from the drive output 174 of the instrument driver 175.

[0051] Torque from the instrument driver 175 is transferred downstream of the elongate shaft 171 using tendons along the shaft 171. These individual tendons, such as pull wires, may be individually secured to individual drive inputs 173 in the instrument handle 172. From the handle 172, the tendons pass along the elongate shaft 171 through one or more pull lumens and are secured to a distal portion of the elongate shaft 171 or are secured to a wrist at the distal portion of the elongate shaft. During a surgical procedure, such as a laparoscopic, endoscopic, or hybrid procedure, these tendons may be coupled to a distally mounted end effector, such as a wrist, grasper, or scissors. In such an arrangement, torque exerted on the drive input 173 will transmit tension to the tendons to actuate the end effector in some manner. In some embodiments, the tendons may rotate a joint about an axis to move the end effector in one direction or another during a surgical procedure. Alternatively, the tendon may be connected to one or more jaws of a grasper at the distal end of the elongate shaft 171, which closes under tension from the tendon.

[0052] In endoscopy, the tendons may be coupled to a bending or articulating section positioned along (e.g., at the distal end) of the elongate shaft 171 via adhesive, control rings, or other mechanical fixation. When fixedly attached to the distal end of the bending section, torque exerted on the drive input 173 is transferred to the tendons, causing the softer bending section (sometimes referred to as the articulating section or area) to bend or articulate. Along the non-bending section, it may be convenient to helical or spiral the individual pull lumens that direct the individual tendons along (or inwardly) the wall of the endoscope shaft to counterbalance the radial forces resulting from tension in the pull wires. The angle of the helix and / or spacing between them may be altered or designed for a particular purpose, with narrower helices exhibiting poor shaft compression under load forces, while fewer helices provide better shaft compression under load forces, but also limited bending. At the other end of the spectrum, orienting the pull lumen parallel to the longitudinal axis of elongate shaft 171 may allow for controlled articulation at desired bends or articulations.

[0053] In endoscopy, the elongated shaft 171 houses several components that aid in robotic procedures. The elongated shaft may include a working channel for deployment of surgical tools (or medical instruments), irrigation, and / or aspiration to the surgical area at the distal end of the shaft 171. The elongated shaft 171 may also house wires and / or optical fibers that carry signals to / from an optical assembly at the distal tip, which may include an optical camera. The shaft 171 may also house optical fibers for carrying light from a proximally located light source, such as a light emitting diode, to the distal end of the shaft.

[0054] At the distal end of the instrument 170, the distal tip may include a working channel opening for delivering tools to the surgical site for diagnosis and / or treatment, irrigation, and aspiration. The distal tip may also include a port for a camera, such as a fiberscope or digital camera, to capture images of the internal anatomical space. In this regard, the distal tip may also include a port for a light source to illuminate the anatomical space when the camera is in use. In the example of FIG. 11, the drive shaft axis, and therefore the drive input axis, is orthogonal to the axis of the elongated shaft.

[0055] FIG. 12 illustrates an alternative design of the instrument driver and instrument, in which the axis of the drive unit is parallel to the axis of the elongated shaft of the instrument. As shown, a circular instrument driver 180 includes four drive units with their drive outputs 181 aligned in parallel at the end of a robotic arm 182. The drive units and their respective drive outputs 181 are housed in a rotating assembly 183 of the instrument driver 180, which is driven by one of the drive units in that assembly 183. In response to torque provided by the rotating drive units, the rotating assembly 183 rotates along a circular bearing connecting the rotating assembly 183 to a non-rotating portion 184 of the instrument driver. In other embodiments, the rotating assembly 183 is integrated into the non-rotatable portion 184 and thus responds to a separate drive unit that is not parallel to the other drive units. The rotating mechanism 183 enables the instrument driver 180 to rotate the drive units and their respective drive outputs 181 as a single unit about the instrument driver axis 185.

[0056] Similar to the previously disclosed embodiments, the instrument 186 may include an elongated shaft portion 188 and an instrument base 187 (shown with a transparent exterior skin for purposes of discussion) that includes a number of drive inputs 189 (such as receptacles, pulleys, and spools) configured to receive the drive output 181 in the instrument driver 180. Unlike the previously disclosed embodiments, the instrument shaft 188 extends from the center of the instrument base 187 with its axis being substantially parallel to the axis of the drive input 189, rather than orthogonal as seen in the design of FIG.

[0057] When coupled to the rotation assembly 183 of the instrument driver 180, a medical instrument 186, including an instrument base 187 and an instrument shaft 188, rotates with the rotation assembly 183 about the instrument driver axis 185. Because the instrument shaft 188 is positioned in the center of the instrument base 187, the instrument shaft 188 is coaxial with the instrument driver axis 185 when attached. Thus, rotation of the rotation assembly 183 causes the instrument shaft 188 to rotate about its own longitudinal axis.

[0058] 13 illustrates an instrument having an instrument-based insertion architecture, according to some embodiments. The instrument 200 can be coupled to any of the instrument drivers discussed above. The instrument 200 includes an elongate shaft 202, an end effector 212 connected to the elongate shaft 202, and a handle 220 coupled to the elongate shaft 202. The elongate shaft 202 includes a tubular member having a proximal portion 204 and a distal portion 206. Manipulation of one or more cables 230a (e.g., via the instrument driver) results in actuation of the end effector 212.

[0059] The instrument handle 220, sometimes referred to as the instrument base, may generally include a mounting interface 222 having one or more mechanical inputs 224, e.g., receptacles, pulleys, or spools, designed to intermate with one or more torque couplers on the mounting surface of the instrument driver.

[0060] In some embodiments, instrument 200 includes a series of pulleys or cables that allow elongate shaft 202 to translate relative to handle 220. In other words, instrument 200 itself includes an instrument-based insertion architecture that accommodates the insertion of instruments, thereby minimizing reliance on a robotic arm to effect the insertion of instrument 200. In other embodiments, the robotic arm may be primarily responsible for the insertion of the instruments.

[0061] E. Controller. Any of the robotic systems described herein can include an input device or controller for manipulating an instrument attached to the robotic arm. In some embodiments, the instrument and controller can be coupled (e.g., communicatively, electronically, wirelessly, and / or mechanically) such that manipulation of the controller causes a corresponding manipulation of the instrument via, for example, master-slave control.

[0062] 14 is a perspective view of one embodiment of the controller 242. In the illustrated embodiment, the controller 242 is configured to enable operation of two medical instruments and includes two handles 244. Each of the handles 244 is connected to a gimbal 246. Each gimbal 246 is connected to a positioning platform 248.

[0063] 14, each positioning platform 248 includes a SCARA arm (selective compliance assembly robot arm) 258 coupled to a column 254 by a prismatic joint 256. The prismatic joint 256 is configured to translate along the column 254 (e.g., along a rail 257) to allow each of the handles 244 to translate in the z-direction, providing a first degree of freedom. The SCARA arm 258 is configured to allow movement of the handles 244 in the xy-plane, providing an additional two degrees of freedom.

[0064] F. Navigation and Control. Traditional endoscopy may involve the use of fluoroscopy (e.g., as may be delivered through a C-arm) and other forms of radiation-based imaging modalities to provide intraluminal guidance to the operator-physician. In contrast, the robotic system contemplated by the present disclosure may provide non-radiation-based navigation and localization means to reduce the physician's exposure to radiation and reduce the amount of equipment in the operating room. As used herein, the term "localization" may refer to determining and / or monitoring the position of an object in a reference coordinate system. Techniques such as pre-operative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to achieve a radiation-free surgical environment. In other cases where radiation-based imaging modalities are still used, pre-operative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to improve information available only through radiation-based imaging modalities.

[0065] As discussed above, the robotic systems discussed herein may be designed to incorporate one or a combination of two or more of the above techniques. The computer-based control system of the tower, bed, and / or cart-based robotic system may store, for example, in a non-transitory computer-readable storage medium, such as a persistent magnetic storage drive, solid-state drive, etc., computer program instructions that, when executed, cause the system to receive and analyze sensor data and user commands, generate control signals throughout the system, and display navigational and localization data, such as instrument position within a global coordinate system, anatomical maps, etc.

[0066] 2. Pose optimization for robotic systems. FIG. 15 illustrates a robotic medical system 300 according to some embodiments. In some embodiments, the robotic medical system 300 is a robotic surgery system. In the example of FIG. 15, the robotic medical system 300 includes a patient support platform 302 (e.g., a patient platform, a table, a bed, etc.). Two ends along the length of the patient support platform 302 are referred to as "head" and "foot", respectively. Two sides of the patient support platform 302 are referred to as "left" and "right", respectively. The patient support platform 302 includes a support 304 (e.g., a rigid frame) for the patient support platform 302.

[0067] The robotic medical system 300 also includes a base 306 for supporting the robotic medical system 300. The base 306 includes wheels 308 (casters) that allow the robotic medical system 300 to be easily movable or repositionable in a physical environment. In some embodiments, the wheels 308 are omitted from the robotic medical system 300 or are retractable so that the base 306 rests directly on the ground or floor. In some embodiments, the wheels 308 are replaced by feet.

[0068] The robotic medical system 300 includes one or more robotic arms 310. The robotic arms 310 can be configured to perform the robotic medical procedures described above with reference to Figures 1-14. Although Figure 15 shows five robotic arms 310 (e.g., arms 310-1-310-5), it should be understood that the robotic medical system 300 can include any number of robotic arms, including less than five, or six or more.

[0069] According to some embodiments, the robotic medical system 300 also includes one or more supports 320 (e.g., a bar or an adjustable bar) that support the robotic arms 310. Each of the robotic arms 310 is supported on and movably coupled to the arm support 320 by a respective base joint of the robotic arm. In some embodiments, the arm support 320 can provide several degrees of freedom, including lift, lateral translation, tilt, etc., as described above. In some embodiments, each of the robotic arms 310 and / or adjustable arm supports 320 are also referred to as a respective kinematic chain.

[0070] 15 shows three robotic arms 310-1, 310-2, and 310-3 supported by arm support 320 within the field of view of the figure. The remaining two robotic arms 310-4 and 310-5 are supported by another bar located across the other length of the patient support platform 302.

[0071] In some embodiments, the adjustable arm support 320 may be configured to provide a home position for one or more of the robotic arms 310 for a robotic medical procedure. The robotic arm 310 may be positioned relative to the patient support platform 302 by translating the robotic arm 310 along the length of its underlying bar 320 and / or by adjusting the position and / or orientation of the robotic arm 310 via one or more joints and / or links (see, e.g., FIG. 17A). In some embodiments, the arm support 320 attitude may be changed via manual, remote, and / or power-assisted movement.

[0072] In some embodiments, the adjustable arm support 320 can be translated along the length of the patient support platform 302. In some embodiments, translation of the arm support 320 along the length of the patient support platform 302 causes one or more of the robotic arms 310 supported by the arm support 320 to translate simultaneously with or relative to the arm support. In some embodiments, the arm support 320 can translate one or more of the robotic arms while remaining stationary relative to the base 306 of the robotic medical system 300.

[0073] 15, the adjustable arm support 320 is located along the length of the patient support platform 302. In some embodiments, the adjustable arm support 320 extends across a partial length or the entire length of the patient support platform 302 and / or across a partial width or the entire width of the patient support platform 302.

[0074] According to some embodiments, during a robotic medical procedure, one or more of the robotic arms 310 can also be configured to hold an instrument 312 (e.g., a robotically controlled medical instrument or tool, such as an endoscope and / or any other instrument that may be used during surgery (e.g., a sensor, a lighting instrument, a cutting instrument, etc.)) and / or can be coupled to one or more accessories, including one or more cannulas.

[0075] FIG. 16 illustrates another view of the robotic medical system 300 of FIG. 15 according to some embodiments. In this example, the robotic medical system 300 includes six robotic arms 310-1, 310-2, 310-3, 310-4, 310-5, and 310-6. The patient platform 302 is supported by columns 314 extending between the base 306 and the patient platform 302. In some embodiments, the patient platform 302 includes a tilt mechanism 316. The tilt mechanism 316 can be positioned between the columns 314 and the patient platform 302 to allow the patient platform 302 to pivot, rotate, or tilt relative to the columns 314. The tilt mechanism 316 can be configured to allow lateral and / or longitudinal tilt of the patient platform 302. In some embodiments, the tilt mechanism 316 allows simultaneous lateral and longitudinal tilt of the patient platform 302.

[0076] FIG. 16 shows the patient platform 302 in a non-tilted state or position. In some embodiments, the non-tilted state or position is the default position of the patient platform 302. In some embodiments, the default position of the patient platform 302 is a substantially horizontal position as shown in FIG. 16. As illustrated, in the non-tilted state, the patient platform 302 can be positioned horizontally or parallel to the surface (e.g., the ground or floor) supporting the robotic medical system 300. In some embodiments, the term "non-tilted" refers to a state in which the angle between the default position and the current position is less than a threshold angle (e.g., less than 5 degrees, or less than an angle that would cause the patient to shift on the patient platform, etc.). In some embodiments, the term "non-tilted" refers to a state in which the patient platform is substantially perpendicular to the direction of gravity, regardless of the angle formed with respect to gravity by the surface supporting the robotic medical system.

[0077] 16 , in the illustrated example of the robotic medical system 300, the patient platform 302 includes a support 304. In some embodiments, the support 304 includes a rigid support structure or frame and can support one or more surfaces, pads, or cushions 322. The upper surface of the patient platform 302 can include a support surface 324. A patient can be placed on the support surface 324 during a medical procedure.

[0078] 16 shows the robotic arm 310 and adjustable arm support 320 in an exemplary deployed configuration, where the robotic arm 310 reaches above the patient platform 302. In some embodiments, the configuration of the robotic medical system 300 allows for storage of different components below the patient platform 302, allowing the robotic arm 310 and arm support 320 to occupy space below the patient platform 302. Thus, in some embodiments, the tilt mechanism 316 has a low profile and / or low volume to increase the space available for storage below.

[0079] FIG. 16 also illustrates an exemplary x, y, and z coordinate system that may be used to describe certain features of the embodiments disclosed herein. It is understood that this coordinate system is provided for purposes of illustration and explanation only, and other coordinate systems may be used. In the illustrated example, the x-direction or x-axis extends laterally across the patient platform 302 when the patient platform 302 is in a non-tilted state. In some configurations, the x-direction extends across the patient platform 302 from one lateral side (e.g., right side) to the other lateral side (e.g., left side) when the patient platform 302 is in a non-tilted state. The y-direction or y-axis extends longitudinally along the patient platform 302 when the patient platform 302 is in a non-tilted state. That is, the y-direction extends along the patient platform 302 from one longitudinal end (e.g., head end) to the other longitudinal end (e.g., foot end) when the patient platform 302 is in a non-tilted state. In the non-tilted state, the patient platform 302 may be in or parallel to an xy plane, which may be parallel to a floor or ground. In the illustrated example, the z direction or z axis extends vertically along the column 314. In some embodiments, the tilt mechanism 316 is configured to laterally tilt the patient platform 302 by rotating the patient platform 302 about a lateral tilt axis parallel to the y axis. The tilt mechanism 316 may further be configured to longitudinally tilt the patient platform 302 by rotating the patient platform 302 about a longitudinal tilt axis parallel to the x axis.

[0080] A. Robotic arm. 17A-17C illustrate different views of the robot arm 210, according to some embodiments. FIG. 17A illustrates the robot arm 310 includes multiple links 402 (e.g., a linkage). The links 402 are connected by one or more joints 404. Each of the joints 404 includes one or more degrees of freedom (DoF). In FIG. 17A, the joints 404 include a first joint 404-1 (e.g., a base joint or an A0 joint) located at or near a base 406 of the robot arm 310. In some embodiments, the base joint 404-1 includes a prismatic joint that allows the robot arm 310 to translate along the arm support 320 (e.g., along the y-axis). The joint 404 also includes a second joint 404-2. In some embodiments, the second joint 404-2 rotates relative to the base joint 404-1. The joint 404 also includes a third joint 404-3 connected to one end of the link 402-2. In some embodiments, the joint 404-3 includes multiple DoFs to facilitate both tilt and rotation of the link 402-2 tilt relative to the joint 404-3.

[0081] FIG. 17A also shows a fourth joint 404-4 connected to the other end of link 402-2. In some embodiments, joint 404-4 includes an elbow joint connecting link 402-2 and link 402-3. Joint 404 further includes a pair of joints 404-5 (e.g., wrist roll joint) and 404-6 (e.g., wrist pitch joint) located at a distal portion of robot arm 310. A proximal end of robot arm 310 may be connected to base 406, and a distal end of robot arm 310 may be connected to advanced device manipulator (ADM) 408 (e.g., tool driver, instrument driver, robot end effector, etc.). ADM 408 may be configured to control the positioning and manipulation of medical instrument 312 (e.g., tool, scope, etc.).

[0082] The robotic arm 310 may also include a cannula sensor 410 for detecting the presence of a cannula or the proximity of a cannula to the robotic arm 310. In some embodiments, when the cannula sensor 410 detects the presence of a cannula (e.g., via one or more processors of the robotic medical system 300), the robotic arm 310 is placed in a docked state (e.g., a docked position). In some embodiments, when the robotic arm 310 is in the docked position, the robotic arm 310 may perform null space motion to maintain the position and / or orientation of the cannula, as discussed in more detail below. Conversely, when a cannula is not detected by the cannula sensor 410, the robotic arm 310 is placed in an undocked state (e.g., an undocked position).

[0083] In some embodiments, as illustrated in FIG. 17A, the robotic arm 310 includes an input or button 412 (e.g., a donut-shaped button, or other type of control, etc.) that can be used to put the robotic arm 310 into admittance mode (e.g., by pressing the button 412). The admittance mode is also referred to as an admittance scheme or admittance control. In the admittance mode, the robotic system 310 measures forces and / or torques (e.g., applied to the robotic arm 310) and outputs a corresponding velocity and / or position. In some embodiments, the robotic arm 310 can be manually manipulated by a user in the admittance mode (e.g., during a set-up procedure, or during a procedure, etc.). In some examples, by using admittance control, an operator does not need to overcome all of the inertia in the robotic medical system 300 to move the robotic arm 310. For example, under admittance control, when an operator applies a force to the arm, the robotic medical system 300 can assist the operator in moving the robotic arm 310 by measuring the force and driving one or more motors associated with the robotic arm 310, thereby resulting in a desired velocity and / or position of the robotic arm 310.

[0084] In some embodiments, the link 402 is removably coupled to the medical instrument 312 (e.g., to facilitate attachment and detachment of the medical instrument 312 from the robotic arm 310). The joints 404 provide the robotic arm 310 with multiple degrees of freedom (DoF) that facilitate control of the medical instrument 312 via the ADM 408. In embodiments including multiple robotic arms, each robotic arm can hold its own respective medical tool and pivot the medical tool about a remote center of motion.

[0085] FIG. 17B illustrates a front view of the robot arm 310. FIG. 17C illustrates a perspective view of the robot arm 310. In some embodiments, the robot arm 310 includes a second input or button 414 (e.g., a push button) different from the button 412 of FIG. 17A to put the robot arm 310 into impedance mode (e.g., by pressing the button 414 once or continuously). In this example, the button 414 is located between the joint 404-5 and the joint 404-6. The impedance mode is also referred to as impedance scheme or impedance control. In the impedance mode, the robotic medical system 300 measures displacement (e.g., change in position and velocity) and outputs forces and / or torques to facilitate manual movement of the robot arm. In some embodiments, the robot arm 310 can be manually operated by a user in the impedance mode (e.g., during a set-up procedure). In some embodiments, under impedance mode, operator movement of a portion of the robotic arm 310 causes movement of one or more joints and / or links of the entire robotic arm 310.

[0086] In some embodiments, for admittance control, force sensors or load cells can measure the force an operator is applying to the robotic arm 310 and move the robotic arm 310 in a way that feels lighter. Under admittance control, motors in the controller can help accelerate the mass, thus hiding the perceived inertia of the robotic arm 310, so admittance control can feel lighter than impedance control. In contrast, with impedance control, according to some embodiments, the user bears most, if not all, of the mass acceleration.

[0087] In some situations, depending on the position of the robotic arm 310 relative to the operator, it may be inconvenient to reach for button 412 and / or button 414 to activate a manual operation mode (e.g., admittance mode and / or impedance mode). Thus, under these circumstances, it may be convenient for the operator to trigger the manual operation mode other than by a button.

[0088] In some embodiments, the robotic arm 310 includes a single button (e.g., button 412 or 414) that can be used to put the robotic arm 310 into admittance mode and / or impedance mode (e.g., by using different presses, such as a long press, a short press, press and hold, etc.). In some embodiments, the robotic arm 310 is put into impedance mode by a user pressing an arm linkage (e.g., link 402) and / or a joint (e.g., joint 404) and overcoming a force threshold. In some embodiments, the admittance mode and impedance mode are common in that they both allow a user to command movement by gripping the robotic arm 310 and interfacing directly with it.

[0089] In some embodiments, the robotic arm 310 includes an input control for activating the arm following mode. For example, in some embodiments, the robotic arm 310 includes designated touch points located on the links 402 or joints 404 (e.g., the outer shell of the links 402 or the buttons 416) of the robotic arm. User interaction (e.g., user touch, contact, etc.) with the designated touch points activates the arm following mode. In some embodiments, the robotic arm 310 includes multiple touch points. User interaction with any of the touch points (e.g., one or more) activates the arm following mode.

[0090] During a medical procedure, it may be desirable to maintain a remote center of motion (RCM) of the ADM 408 of the robotic arm 310 and / or the instrument 312 coupled thereto in a static attitude (e.g., position and / or orientation). The RCM may refer to a point in space where the motion of a cannula or other access port through which the medical instrument 312 is inserted is constrained. In some embodiments, the medical instrument 312 includes an end effector that is inserted through an incision or natural orifice in a patient while maintaining the RCM. In some embodiments, the medical instrument 312 includes an end effector that is in a retracted state during a setup process of the robotic medical system.

[0091] In some situations, the robotic medical system 300 can be configured to move one or more links 402 of the robotic arm 310 in a "null space" to avoid collisions with nearby objects (e.g., other robotic arms) while the ADM 408 and / or RCM of the robotic arm 310 are maintained in their respective poses (e.g., positions and / or orientations). The null space can be considered as a set of joint states that the robotic arm 310 can move into that do not result in movement of the ADM 408 and / or RCM, thereby maintaining the position and / or orientation of the medical instrument 312 (e.g., within a patient). In some embodiments, the robotic arm 310 can have multiple positions and / or configurations available for each pose of the ADM 408.

[0092] In order for the robotic arm 310 to move the instrument to a desired pose in space, in certain embodiments, the robotic arm 310 may have at least six DoFs, i.e., three DoFs for translation (e.g., X position, Y position, and Z position) and three DoFs for rotation (e.g., yaw, pitch, and roll). In some embodiments, each joint 404 provides the robotic arm 310 with a single DoF, and thus the robotic arm 310 may have at least six joints to achieve degrees of freedom of movement to position the ADM 408 at any pose in space. To further maintain the ADM 408 and / or remote center or movement of the robotic arm 310 at a desired pose, the robotic arm 310 may further have at least one additional "redundant joint." Thus, in certain embodiments, the system includes a robotic arm 310 with at least seven joints 404, providing the robotic arm 310 with at least seven DoFs. In some embodiments, the robotic arm 310 includes a subset of the joints 404 each having two or more degrees of freedom, thereby achieving additional DoF for null space motion, however, depending on the embodiment, the robotic arm 310 may have a greater or lesser number of DoF.

[0093] 7, the arm support 320 (e.g., an adjustable arm support) can provide several degrees of freedom including lift, lateral translation, tilt, etc. Thus, depending on the embodiment, the robotic medical system can have many more robotically controlled degrees of freedom beyond those in the robotic arm 310 to provide zero space movement and collision avoidance. In each of these embodiments, the end effector (and any tools or instruments coupled thereto) of one or more robotic arms, as well as remote centers along the axes of the tools, can advantageously maintain their orientation and / or position within the patient.

[0094] A robotic arm 310 with at least one redundant DoF has at least one more DoF than the minimum number of DoFs to perform a given task. For example, the robotic arm 310 may have at least seven DoFs, and according to some embodiments, one of the joints 404 of the robotic arm 310 may be considered a redundant joint. The one or more redundant joints may enable the robotic arm 310 to move in null space to maintain the attitude of the ADM 408 and the position of the RCM, and to avoid collisions with other robotic arms or objects.

[0095] In some embodiments, the robotic medical system 300 is configured to perform collision avoidance, for example, to avoid collisions between adjacent robotic arms 310, by utilizing movement of one or more redundant joints in the null space. For example, when a robotic arm 310 collides with or approaches (e.g., within a defined distance of) another robotic arm 310, one or more processors of the robotic medical system 300 can be configured to detect the collision or impending collision (e.g., via kinematics). Thus, the robotic medical system 300 can control one or both of the robotic arms 310 to adjust their respective joints in the null space to avoid the collision or impending collision. In embodiments including at least a pair of robotic arms, the base of one of the robotic arms and its end effector can remain in its pose while the link or joint between them moves in the null space to avoid collision with the adjacent robotic arm.

[0096] B. Exemplary port placements for different surgical procedures. As explained above, robotic arm pose and port placement relative to the target anatomy are critical to the success of a medical procedure, which typically requires precise port location with little tolerance for error.

[0097] 18A-18C illustrate exemplary cannula placements for different surgical procedures, according to some embodiments. FIG. 18A illustrates an exemplary surgical environment 500 including placement of four cannulas 502-1, 502-2, 502-3, and 502-4 at several anatomical locations within a patient 504 to provide access to at least a portion of the patient's anatomy 506 (e.g., abdomen). The cannulas 502 can pass through port locations 508 within the patient (or within the patient's anatomy 506). As used herein, a port location (e.g., port, entry port, entry point, port area, port area, or port location, etc.) refers to a location on the patient's body through which a medical tool / instrument (e.g., held by a robotic arm) can be inserted and motion constrained. In some embodiments, the port location corresponds to an incision point (or incision area) made through the patient's skin to facilitate a medical operation or procedure. In some embodiments, the port location corresponds to a natural orifice such as the patient's mouth (e.g., for a bronchoscopy procedure). In the example of Figure 18A, the illustrated placement of the cannula 502 may allow access to the right upper quadrant of the patient's abdomen.

[0098] In some embodiments, a camera is docked to one of the cannulas 502 (e.g., cannula 502-3) to provide a view of the patient's anatomy (e.g., the right upper quadrant in FIG. 18A ). Medical instruments may be docked to one or more of the cannulas 502 (e.g., 502-1, 502-2, and / or 502-4). Each of the cannulas 502 provides access to the patient's anatomy 506 from a unique position and / or orientation, allowing flexibility in the way the medical procedure is performed. The angles at which medical tools may access the patient's anatomy 506 from the cannulas 502 are illustrated by the arrows extending from each cannula 502. As an example, during an early stage of a medical procedure, a physician may use medical tools docked to cannulas 502-1 and 502-2, and at a later stage, may use medical tools docked to cannulas 502-3 and 502-4. By using medical tools docked to different cannulas 502, a physician may be able to access the patient's anatomy 506 from different angles, allowing more options for direct access to various portions of the patient's anatomy 506.

[0099] In some embodiments, as illustrated in FIG. 18A, cannulas 502-1 and 502-4 provide access to the patient's anatomical structure 506 from substantially opposite sides, allowing a physician to access both sides of the anatomical structure 506 by simply selecting a medical tool to be docked to cannulas 502-1 and 502-4.

[0100] FIG. 18B illustrates an exemplary surgical environment 510 that includes five cannulas 502-5 through 502-9 positioned at anatomical locations on a patient 505 to provide access to the upper left quadrant of the patient's abdomen.

[0101] Figure 18C illustrates an exemplary surgical environment 520 including five cannulas 502-10 through 502-14 at anatomical locations on a patient 504 to provide access to multiple quadrants of the abdomen of the patient 504. The arrangement of cannulas 502 in Figure 18C may be provided for performing a colectomy as an exemplary surgical procedure.

[0102] C. Example workflow for optimal robot arm pose 19 illustrates an example workflow 600 for generating optimal port locations and robotic arm poses for a medical surgery or procedure, according to some embodiments. In some embodiments, workflow 600 is implemented as a set of instructions executed by one or more processors (e.g., processor 380, FIG. 24 ) of a robotic medical system to recommend port placements and initiate poses to avoid collisions during surgery (e.g., between a patient and a robotic arm, between two or more robotic arms, between a robotic arm and a surgical tool, between two or more surgical tools, etc.).

[0103] The workflow 600 includes placing 602 a patient on a table (e.g., patient support platform 302, bed, etc.) of a robotic medical system according to some embodiments. In some embodiments, the patient is positioned on the table in a position where he or she would be during a medical operation or procedure.

[0104] The workflow 600 includes acquiring 604 (e.g., creating, generating, receiving, determining, collecting, etc.) a 3D scan (e.g., 3D data, imaging data, scan data, etc.) including a view of a patient, according to some embodiments. FIGS. 20A and 20B illustrate an exemplary 3D scanning and registration process of a patient to a robotic medical system, according to some embodiments. FIG. 20A shows a side view 700 illustrating a patient 702 positioned on a patient support platform 302 of the robotic medical system 300. In this example, a 3D scanner 704 is attached to the robotic arm 310 of the robotic medical system 300. FIG. 20B illustrates a top view 710 including a portion of the patient 702 (e.g., an anatomical structure such as the abdomen) and the location of the robotic arms 310-1 to 310-6 and adjustable arm supports 320-1 and 320-2 of the robotic medical system 300.

[0105] In some embodiments, the 3D scanner 704 is included with (e.g., is a component of, forms a part of, etc.) the robotic medical system 300. For example, the 3D scanner 704 can be a component of the robotic medical system 300 that is integrated into or attached to the robotic medical system 300. For example, the 3D scanner 704 can be located (e.g., positioned) on any part of the robotic medical system 300, including the robotic arm 310, the patient support platform 302 (e.g., bed), the adjustable arm support 320, the tower pendant, and / or the patient view.

[0106] In some embodiments, the 3D scanner 704 is part of an instrument (e.g., a medical tool such as a laparoscope) held by the robotic arm 310 of the robotic medical system 300. In some embodiments, the 3D scanner 704 is a separate accessory (e.g., a stand-alone scanner, a handheld scanner, etc.) that is communicatively connected to the robotic medical system 300 and attached (e.g., mounted) to any part of the robotic medical system 300, including the robotic arm 310, the patient support platform 302 (e.g., a bed), the adjustable arm support 320, the tower pendant, and / or the patient view. In some embodiments, the 3D scanner 704 is mounted to a wall and / or ceiling of the operating room in which the robotic medical system 300 is located.

[0107] In some embodiments, the 3D scanner 704 performs 3D scanning using time-of-flight (e.g., laser imaging, detection, and ranging (LIDAR)), dot pattern recognition, stereo photography (e.g., using stereo cameras), and / or other established scanning technologies. Note that these scanning technologies are examples and are not intended to be limiting.

[0108] The 3D scanner 704 creates (e.g., generates, acquires, etc.) data (e.g., scan data, 3D scan data, etc.) including a view of the patient 702. In some embodiments, the 3D scan data includes visual data (e.g., imaging data) of at least a portion of the patient 702 and the environment (e.g., space and / or other objects) surrounding the patient 702. The scan data can capture size (e.g., dimensions), shape, and / or depth information of at least a portion of the patient 702 and one or more objects (e.g., patient support platform, medical tools, etc.) surrounding the patient 702. In some embodiments, the scan data includes dimensional information (e.g., distance, angle, etc. information) that can be used to determine relative separation (e.g., spatial separation, angular separation) between the patient 702 and other objects.

[0109] The workflow 600 includes locating (606) the patient on a table (e.g., the patient support platform 302) based on the position of the robotic arm 310, according to some embodiments. For example, the 3D scan data can include views of the patient and parts of the robotic medical system, such as the robotic arm, the patient support platform, and / or the adjustable arm support (or portions thereof). Because the robotic medical system 300 can determine the positions and / or orientations of the robotic arm, the patient support platform, and / or the adjustable arm support (e.g., through data acquired by sensors and / or encoders located throughout the robotic medical system), the robotic medical system 300 can locate the patient (e.g., determine one or more positions of the patient) relative to components of the robotic medical system 300 (e.g., relative to the robotic arm 310).

[0110] In some embodiments, after locating the patient based on the position of the robotic arm 310, the robotic medical system proceeds to place the patient on a coordinate system of the robotic medical system 300 and use the coordinate system to determine coordinates corresponding to different parts of the patient (e.g., head, toes, navel, etc.). For example, in some embodiments, the robotic medical system 300 includes a coordinate system (e.g., a robot coordinate system, a coordinate frame, a system frame, etc.), and the respective positions of the patient support platform 302, the robotic arm 310, and / or the adjustable arm support 320 can be expressed as coordinates (e.g., x-, y-, and z-coordinates) on the coordinate system. The robotic medical system 300 can use scan data acquired by the 3D scanner 704 (e.g., imaging data including views of the patient and a portion of the robotic arm 310) to "match" the patient frame with the system frame (e.g., the robotic arm frame) and "register" the patient to the coordinate system.

[0111] The workflow 600 includes registering (608) (e.g., determining) anatomical structures (e.g., external and / or internal anatomical structures) in the 3D scan by the robotic medical system 300, according to some embodiments. In some embodiments, the robotic medical system 300 compares (e.g., matches) the 3D scan data (e.g., automatically or manually based on user input) with previously generalized imaging data (e.g., heuristic data, previous imaging data, imaging data from a database, etc.), such as MRI imaging data and / or CT scan data. The robotic medical system 300 identifies one or more anatomical structures according to the comparison (e.g., the robotic medical system 300 identifies one or more anatomical structures according to a determination that an anatomical structure depicted in the 3D scan data matches an anatomical structure in the database of imaging data).

[0112] In some embodiments, the robotic medical system 300 acquires pre-operative imaging data corresponding to the patient (e.g., MRI imaging data, CT scan data, etc.). The robotic medical system 300 compares the 3D scan data to the pre-operative imaging data (e.g., automatically or manually based on user input, side-by-side comparison, overlaying the two sets of data, etc.) and identifies (e.g., registers) one or more anatomical structures according to the comparison.

[0113] In some embodiments, the robotic medical system 300 registers (e.g., identifies, determines, etc.) one or more anatomical markers (e.g., anatomical landmarks, anatomical structures) of the patient from the 3D scan data (e.g., automatically or manually based on user input). For example, the one or more anatomical markers can include an external anatomical marker, such as the navel, or an internal anatomical marker (e.g., the xiphoid process or anterior superior iliac spine (ASIS)), which can be inferred based on the 3D scan data. In some configurations, the robotic medical system 300 utilizes image processing software or pattern recognition software to identify the location of the one or more external anatomical markers. In some configurations, the robotic medical system 300 determines the location of the one or more internal anatomical markers from the location of the one or more external anatomical markers (e.g., the internal anatomical marker is located below a position having a certain distance relative to the two or more external anatomical markers).

[0114] In some embodiments, the robotic medical system registers (e.g., automatically or manually based on user input) one or more anatomical markers based on fiducials (e.g., anatomical side markers) placed on the patient prior to collecting the 3D scan. In some embodiments, the position of the one or more fiducials is deemed to correspond to the position of the one or more anatomical markers. In some embodiments, the position of the one or more anatomical markers is determined based on the position of the one or more fiducials. For example, the position of the one or more anatomical markers is deemed to correspond to a predetermined distance below the one or more fiducials. In another example, the position of the anatomical marker is deemed to correspond to a position determined based on two or more fiducials (e.g., having an equal distance to two or more fiducials).

[0115] The workflow 600, according to some embodiments, includes receiving user input (610) from a user of the robotic medical system 300, such as a surgeon, a nurse, or a surgeon assistant. The input can include an identification of a medical procedure to be performed on a patient, a target anatomical structure (or target anatomical structures), and / or a number of robotic arms (and / or a selection of robotic arms) to be used for the procedure. FIG. 21A illustrates user input of a target anatomical structure or procedure, according to some embodiments. As shown in FIG. 21A, an operator may provide user input through a user interface. In some embodiments, the robotic medical system includes a touch-sensitive display (e.g., a touch screen), and the operator may provide user input through the touch-sensitive display. Alternatively, the operator may provide user input through any other input device (e.g., a mouse, a keyboard, one or more buttons or switches, etc.). FIG. 21B illustrates the robotic medical system 300 estimating a location of a target anatomical structure (e.g., a stomach). In some embodiments, the robotic medical system 300 estimates the location of the target anatomical structure based on the techniques described with respect to operation 608 of the workflow 600 and / or in combination with user input in operation 610.

[0116] The workflow 600, according to some embodiments, includes combining (612) the 3D scan data (e.g., from operation 604) with the registered anatomical structures (e.g., obtained from operation 608) and the intended (or selected) procedure and / or target anatomical structures (e.g., obtained from operation 610) to identify candidate patient-specific robotic arm poses and port placements for the medical procedure. In some embodiments, the combining includes simulating the medical procedure based on the combined data (e.g., mapping the robotic arm trajectory during the medical procedure).

[0117] The workflow 600 includes generating (614) one or more recommended (e.g., optimized) port locations and robotic arm poses specific to the patient and procedure (medical operation). For example, the robotic medical system identifies a number of locations that are within a first predetermined distance from the target anatomical structure (e.g., the target organ), excludes locations adjacent to one or more exclusion regions (also called "exclusion zones"), such as areas where other important anatomical structures such as organs are located, areas where other tools or other accessories are located, identifies (e.g., from a memory of the robotic medical system that stores the number and orientations of ports) the number of ports associated with the intended (or selected) procedure and their orientations relative to the target anatomical structure, and selects locations that are within a predetermined distance from the target anatomical structure in the identified orientation. Additionally, the robotic medical system identifies a starting pose of the robotic arms involved in the procedure. The robotic medical system simulates the procedure and the movement of the robotic arms during the procedure to identify any collisions that may occur (e.g., between the robotic arms and / or between the robotic arms and the patient, other equipment, or people).

[0118] In some embodiments, the robotic medical system identifies one or more potential starting poses (e.g., each corresponding to a recommended set of port placements) and simulates the procedure and movement for each potential starting pose. For example, the robotic medical system may select multiple potential starting poses from a set of predetermined potential starting poses. In another example, the robotic medical system may generate multiple potential starting poses by varying joint angles of one or more joints (e.g., generating multiple potential starting poses corresponding to specific joint angles of specific joints). In some embodiments, the robotic medical system selects the potential starting pose that results in the least number of collisions (e.g., 0) as the recommended starting pose. For example, the robotic medical system may record the number of collisions for each starting pose among the multiple potential starting poses (e.g., based on a simulation) and select the starting pose among the multiple potential starting poses with the least number of collisions. In another example, the robotic medical system may select the starting poses that have less than a predetermined number of collisions. In some embodiments, port locations separated by at least a second predetermined distance (e.g., to provide sufficient distance between tools) are selected. In some embodiments, to optimize port locations, a value function is used (in the optimization process) based on the distance of each port location to the target anatomical structure, the distance of each port location to other port locations, the distance of each port location to one or more exclusion regions, and / or the orientation provided by each port location relative to the target anatomical structure.

[0119] The workflow 600 includes displaying (616) (or causing to be displayed) the optimized port locations and robotic arm postures, according to some embodiments. FIG. 22A illustrates a top view 800 of a patient 702 including optimal port locations 802 displayed (e.g., projected) on the abdomen of the patient 702. FIG. 22B illustrates the optimized port locations and robotic arm postures displayed on a user interface 820 of the robotic medical system 300. In some embodiments, the user interface 820 is located on a surgeon viewer (e.g., tower viewer, tower pendant, etc.) or any display device communicatively connected to the robotic medical system 300. In some embodiments, the robotic medical system 300 displays the recommended arm postures and port locations using augmented-reality (AR) glasses communicatively connected to the robotic medical system 300. For example, the AR glasses can be worn by a user of the robotic medical system 300 during the setup phase for a medical procedure.

[0120] In some embodiments, the robotic medical system 300 causes the recommended robotic arm poses and port locations to be printed onto a medium. For example, in some embodiments, the recommended port locations are printed onto a sterile temporary tattoo that can be applied onto the patient. In some embodiments, the printed medium includes sterile paper, film, and / or fabric onto which a patient view including the recommended port locations is printed and attached (e.g., taped) to the patient.

[0121] As mentioned above, FIG. 22A illustrates a set of recommended port locations 802 for a medical procedure, according to some embodiments. In the example of FIG. 22A, the recommended port locations 802 include four port locations 802-1, 802-2, 802-3, and 802-4 in the region of the patient's stomach. FIG. 22B illustrates an example set of initial (e.g., pre-docking) postures of the robotic arm 804 corresponding to the port locations 802. As shown in FIG. 22B, a subset of all available robotic arms may be used for the corresponding port locations (e.g., robotic arms 804-1, 804-2, 804-3, and 804-4). For example, in a system having up to six robotic arms, four robotic arms are used. In some embodiments, each robotic arm has multiple joints (e.g., as discussed above with reference to FIGS. 17A-17C), and identifying the initial arm postures includes identifying an initial position for each joint of the multiple joints.

[0122] 22C and 22D illustrate exemplary arm posture movements during a procedure, according to some embodiments, corresponding to a set of port locations 802 and the initial arm postures shown in FIG. 22B. FIG. 22C shows the user interface 820 with the robot arm 804-4 moving from the initial position 805-a to the new position 805-b and the robot arm 804-3 moving from the initial position 807-a to the new position 807-b. FIG. 22D shows the user interface 820 with the robot arm 804-1 moving from the initial position 809-a to the new position 809-b and the robot arm 804-2 moving from the initial position 811-a to the new position 811-b. FIG. 22D further shows a collision between the robot arms 804-1 and 804-2 due to the movements. According to some embodiments, the movements shown in Figures 22C and 22D correspond to a simulation (e.g., a simulation performed by system 279) of the movements of the robotic arm 804 during a medical procedure based on the set of port locations 802 and the initial arm postures shown in Figure 22B. In some embodiments, the robotic medical system refrains from selecting the port locations 802 and the initial postures shown in Figure 22B as the recommended port locations and recommended initial postures due to identifying a collision (e.g., unless another combination of port locations and initial postures has fewer collisions).

[0123] FIG 22E illustrates another set of port locations 812 for a medical procedure, according to some embodiments. In the example of FIG 22E, the recommended port locations 812 include four port locations 812-1, 812-2, 812-3, and 812-4 in the region of the patient's stomach. In some embodiments, the port locations 812 correspond to viable alternatives to the port locations 802 for a given medical procedure. FIG 22F illustrates an example set of initial (e.g., pre-docking) poses of the robotic arm 804 that correspond to the port locations 812.

[0124] 22G and 22H illustrate example arm posture movements during a procedure, according to some embodiments, corresponding to the set of port locations 812 and the initial arm postures shown in FIG. 22F. FIG. 22G shows the user interface 820 with the robot arm 804-4 moving from the initial position 813-a to the new position 813-b and the robot arm 804-3 moving from the initial position 815-a to the new position 815-b. FIG. 22H shows the user interface 820 with the robot arm 804-1 moving from the initial position 819-a to the new position 819-b and the robot arm 804-2 moving from the initial position 817-a to the new position 817-b. According to some embodiments, the movements shown in FIG. 22G and 22H correspond to a simulation (e.g., a simulation performed by the system 279) of the movement of the robot arm 804 during a medical procedure based on the set of port locations 812 and the initial arm postures shown in FIG. 22F. According to some embodiments, the port location 812 and initial orientation shown in FIG. 22F may be selected as a recommended port location and orientation that is collision-free (or has less collision than the alternative shown in FIG. 22C and FIG. 22D) following the movements shown in FIG. 22G and FIG. 22H.

[0125] 22A-22H, in some embodiments, one or more additional robotic arms that do not correspond to the port placement are used during the procedure (e.g., to hold lights or instruments that do not require corresponding ports). In embodiments where additional robotic arms are present, the robotic medical system takes those arms into account when determining the initial pose and simulating collisions during a medical operation or procedure.

[0126] 23A and 23B are flow charts illustrating a method 900 for determining a starting state of a robotic medical system, according to some embodiments. The method 900 is implemented in a robotic medical system (e.g., medical system 36 or robotic medical system 300) having one or more processors (e.g., processor 280), a memory (e.g., memory 282), and multiple robotic arms. In some embodiments, the memory (e.g., memory 282) stores instructions for execution by the one or more processors (e.g., processor 280). In some embodiments, at least a subset of the robotic arms are coupled to respective medical instruments.

[0127] The robotic medical system acquires imaging data of the patient (902). For example, the robotic medical system acquires imaging data as described above with respect to operation 604. In some embodiments, the imaging data includes a 3D surface scan of the patient. In some embodiments, the robotic medical system includes an imaging device (e.g., 3D scanner 704). In some embodiments, the imaging device is not part of the robotic medical system but is in communication with the robotic medical system. In some embodiments, the imaging device incorporates at least one of time-of-flight or dot pattern recognition. In some embodiments, the robotic medical system uses the imaging data to identify one or more bony landmarks of the patient (e.g., the rib cage or hip bones).

[0128] In some embodiments, the patient imaging data includes images of the patient after the patient has been prepared for the medical procedure (904). In some embodiments, the patient imaging data includes images of the patient with a body cavity insufflated. In some embodiments, the system determines (e.g., in real time) one or more physical characteristics of the patient (e.g., size and / or other characteristics such as unique anatomy). In some embodiments, the patient imaging data includes images of the patient after being positioned in position (e.g., positioned on the patient support platform 302) for the medical procedure. In some embodiments, the imaging data includes a portion of a robotic arm and is used to locate the patient relative to the robotic arm (e.g., as described above with respect to operation 606).

[0129] The robotic medical system obtains (906) information regarding the patient's medical procedure. For example, the robotic medical system obtains information regarding the medical procedure as described above with respect to operation 610. As another example, a doctor or assistant informs the system of the type of medical procedure being performed (e.g., enters details of the medical procedure into the console 31). In some embodiments, the information regarding the medical procedure includes the type of procedure being performed. In some embodiments, the information regarding the medical procedure includes information regarding additional equipment or tools (e.g., stirrups, liver retractors, arm boards, and / or anesthesia accessories) used during the medical procedure.

[0130] The robotic medical system simulates the medical procedure from the start state to identify, for each start state of the plurality of start states, a number of collisions that occur with the arms of the plurality of robotic arms (908). The simulating uses the imaging data and information about the medical procedure. For example, the simulating includes simulating robot arm movements as described with respect to FIGS. 22B-22H. In some embodiments, the medical procedure involves several successive steps, and at least a subset of the steps have corresponding robot arm position requirements. In some embodiments, the simulating includes simulating robot arm movements between required robot arm positions. In some embodiments, the simulating includes simulating alternative movements between required robot arm positions to identify which alternative results in the least collisions. In some embodiments, the simulating is further based on the type of robotic medical system (e.g., how many arms the system has and how they are mounted). In some embodiments, the simulating is further based on the type of robotic arm (e.g., how many joints the robot has and how many degrees of freedom are available). For example, in some embodiments, the robotic arm support has four degrees of freedom, the robotic arm has five degrees of freedom (e.g., roll and twist), and the medical instrument attached to the robotic arm has four degrees of freedom (e.g., roll, pitch, yaw, and insertion). In some embodiments, simulating includes determining the number of robotic arms of the system that will be used in the medical procedure (e.g., determining whether a subset of the robotic arms will be unused and can be stored during the medical procedure).

[0131] In some embodiments, simulating includes (i) identifying a first set of port configurations, (ii) determining a starting pose based on the first set of port configurations, and (iii) mapping a trajectory of the robotic arm during the medical procedure as the robotic arm moves from the starting pose to positions required for different steps of the procedure. In some embodiments, simulating includes (i) repeating the identifying, determining, and mapping for multiple sets of port configurations and multiple starting poses, and (ii) selecting the set of port configurations and starting poses that results in the least number of collisions.

[0132] In some embodiments, the robotic medical system acquires (910) an anatomical target for the medical procedure, and the simulating further uses the anatomical target, for example, the robotic medical system acquires the anatomical target as described above with respect to operation 610. In some embodiments, a user inputs the medical procedure and the target anatomical structure.

[0133] In some embodiments, the robotic medical system identifies anatomical targets using the patient's imaging data (912). For example, the robotic medical system identifies anatomical targets as described above with respect to operation 608. In some embodiments, the robotic medical system estimates the location of one or more important anatomical structures, such as relevant organs, based on the imaging data. In some embodiments, the anatomical targets include external anatomical markers, such as the navel, or internal anatomical markers (e.g., the xiphoid process or anterior superior iliac spine), which can be inferred based on the imaging data. In some embodiments, the system compares the patient's imaging data to generalized imaging data (e.g., heuristic data, previous imaging data, imaging data from a database, etc.) and identifies analysis targets according to the comparison. In some embodiments, the system estimates the location of internal organs based on the anatomical targets.

[0134] In some embodiments, the robotic medical system acquires (914) imaging data of the periphery of the robotic medical system, and simulating further uses the imaging data of the periphery. For example, the robotic medical system determines one or more boundary conditions (e.g., spatial limits) of the robotic arm. In some embodiments, the data includes visual data (e.g., imaging data) of at least a portion of the patient and the environment (e.g., space and / or objects) surrounding the patient. In some embodiments, the periphery includes non-robotic components such as bedside accessories that are fixed to the system (e.g., liver retractors and anesthesia equipment that may be fixed to the bed and draped over the patient's head). In some embodiments, the periphery includes movable and removable (optional) portions of the patient support platform, such as movable or removable leg supports. In some embodiments, based on the visual data of the periphery, the robotic medical system identifies one or more boundary conditions or "keep-out" zones for the robotic arm (e.g., to prevent collisions between the robotic arm and objects in the periphery). In some embodiments, the robotic medical system uses one or more machine learning techniques to identify non-robotic components and / or movable and removable parts of the patient support platform (e.g., object recognition algorithms).

[0135] In some embodiments, the robotic medical system obtains (918) position information of one or more objects (or entities) in the vicinity of the robotic medical system, and simulating further uses the position information. For example, the position information captures size (e.g., dimensions), shape, and / or depth information of at least a portion of the patient and one or more objects surrounding the patient (e.g., lighting fixtures, oxygen tanks, carts, displays, and trays). The position information includes dimensional information (e.g., distance, angle, etc. information) that can be used to determine relative separation (e.g., spatial separation, angular separation) between the patient and other objects. In some embodiments, the objects include accessories (e.g., attached to a bedside or robotic arm, etc.), such as stirrups, liver retractors, arm boards, anesthesia accessories, etc.

[0136] In some embodiments, the robotic medical system maps (920) a trajectory of each of the multiple robotic arms during a medical procedure and identifies collisions based on the trajectory. For example, the robotic medical system maps the trajectory to identify collisions between a patient and a robotic arm, between two robotic arms, between two instruments, between a robotic arm and an instrument, and / or between a robotic arm and another nearby object.

[0137] The robotic medical system selects (922) a first starting state of the multiple starting states, the first starting state having a minimum number of collisions. The first starting state specifies a respective starting pose for each arm of the multiple robotic arms. For example, the robotic medical system generates optimized pose information and port placements as described above with respect to operation 614. In some embodiments, selecting the first starting state includes specifying how many of the robotic arms will be used for the medical procedure (e.g., by simulating the use of different numbers of robotic arms to perform the medical procedure). In some embodiments, the robotic medical system selects the first starting state according to requiring a minimum number of robotic arms.

[0138] In some embodiments, each arm of the plurality of robotic arms has a plurality of joints, and the first starting state includes positioning information for each joint of the plurality of joints for each arm of the plurality of robotic arms (924). In some embodiments, the first starting state includes positioning information for each robotic arm and each robotic arm support (e.g., arm support 105 or 320). In some embodiments, the plurality of joints includes a subset of the joints that lock in position during the medical procedure, and the positioning information includes locked positions for the subset of joints. For example, the robotic arm may include (i) a set-up arm (e.g., a proximal joint of the arm, such as joints 404-1 through 404-6) that is used to position the operative end of the arm, and (ii) a distal end of the operative end of the arm. In some embodiments, the set-up joint remains in a locked position (specified in the positioning information) during the medical procedure.

[0139] In some embodiments, the medical procedure requires a medical instrument (926), and the pose information includes information regarding which of the multiple robotic arms the medical instrument should be attached to. In some embodiments, the medical instrument has different size options, and the pose information includes information regarding a recommended size option for the medical instrument. In some embodiments, the robotic medical system identifies an assigned task (e.g., holding a cannula, light, camera, etc.) for each robotic arm. For example, the medical procedure requires three different medical instruments, and the system identifies which arm will hold each instrument (e.g., to minimize the number of collisions during the procedure).

[0140] In some embodiments, the robotic medical system selects (928) a recommended port location based on the simulating (e.g., as illustrated in FIGS. 22A and 22E). The recommended port location is a location (e.g., position) on the patient's body where a medical tool / instrument will be inserted and motion will be constrained. In some embodiments, the port location corresponds to an incision point (or incision area) that will be made through the patient's skin to facilitate the medical procedure. In some embodiments, the port location corresponds to a natural orifice such as the patient's mouth (e.g., for a bronchoscopy procedure).

[0141] The robotic medical system provides pose information for the first starting state to an operator of the robotic medical system (930). For example, the robotic medical system provides the pose information as described above with respect to operation 616. In some embodiments, the system informs the user of the optimal pre-docking pose for one or more arms and provides the user (e.g., a hospital assistant) with an option to move the one or more arms to their respective optimal poses. In some embodiments, the robotic medical system displays one or more graphical representations indicating the pose information on a display (e.g., touch screen 26) or provides electrical signals to a display to display the pose information (e.g., displayed in VR or AR).

[0142] In some embodiments, the robotic medical system moves at least one arm of the multiple robotic arms to a respective start position (932). In some embodiments, the arm is moved from a draping position to the start position. In some embodiments, the robotic medical system automatically drives to the first start state. In some embodiments, the robotic medical system automatically docks to the cannula via an integrated sensing mechanism. In some embodiments, a user manually completes the docking after moving the multiple robotic arms to their respective start positions.

[0143] In some embodiments, the arms are moved from a stowed position (e.g., under a table for a medical procedure) to a start position. In some embodiments, the robotic arms are initially in a stowed position under a tabletop of a robotic medical system (e.g., as illustrated in FIG. 6). In some embodiments, the robotic arms are initially in a stowed position adjacent to a patient support platform (e.g., as illustrated in FIG. 16). In some embodiments, the arms are deployed to an initial drape position, thereby allowing them to be draped to maintain a sterile boundary. In some embodiments, the system moves the robotic arms from the drape position to their respective start positions (e.g., based on the 3D scan data and surgical procedure information).

[0144] In some embodiments, a robotic medical system includes (i) a plurality of robotic arms; (ii) one or more processors; and (iii) a memory that stores instructions that, when executed by the one or more processors, cause the one or more processors to: (a) acquire imaging data of a patient; (b) acquire information related to a medical procedure of the patient; (c) simulate the medical procedure from a start state to identify, for each start state of the plurality of start states, a number of collisions that occur with an arm of the plurality of robotic arms, where the simulating uses the imaging data and the information related to the medical procedure; (d) select a first start state of the plurality of start states, where the first start state has a minimum number of collisions, the first start state identifying a respective start posture of each arm of the plurality of robotic arms; and (e) provide posture information for the first start state to an operator of the robotic medical system.

[0145] In some embodiments, the robotic medical system includes an adjustable bar to which at least a subset of the arms are coupled. In some embodiments, each robotic arm is coupled to an instrument for performing a medical procedure on a patient. In some embodiments, the robotic medical system includes multiple robotic arms (e.g., four, five, or six arms) to provide increased flexibility to the surgeon (e.g., requiring careful setup and positioning to avoid intraoperative collisions while still being able to reach the target anatomical structures).

[0146] In some embodiments, the robotic medical system displays one or more graphical representations indicating the posture information on a display or provides electrical signals to a display to display the posture information (e.g., displayed in VR or AR).

[0147] In some embodiments, the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to acquire an anatomical target for the medical procedure. The simulating further uses the anatomical target. In some embodiments, the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to identify the anatomical target using imaging data of the patient. In some embodiments, the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to acquire imaging data of a periphery of the robotic medical system, and the simulating further uses the imaging data of the periphery. In some embodiments, the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to acquire position information of one or more objects in a vicinity of the robotic medical system, and the simulating further uses the position information.

[0148] In some embodiments, the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to select a recommended port location based on the simulating. In some embodiments, each arm of the plurality of robotic arms has a plurality of joints, and the first start state includes, for each arm of the plurality of robotic arms, positioning information for each joint of the plurality of joints. In some embodiments, the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to move at least one arm of the plurality of robotic arms to a respective start posture.

[0149] 3. Implementation Systems and Terminology. 24 is a schematic diagram illustrating electronic components of a system 279 (e.g., a robotic medical system) according to some embodiments. In some embodiments, the system 279 is an example of one of the systems 10, 36, 47, 100, and 140A. The system 279 includes one or more processors 380 in communication with a computer-readable storage medium 382 of memory (e.g., computer memory devices such as random access memory, read-only memory, static random access memory, and non-volatile memory, or other storage devices such as hard drives, optical disks, magnetic tape recordings, or any combination thereof) that stores instructions for performing any of the methods described herein (e.g., the operations described with respect to FIGS. 19, 23A, and 23B).

[0150] The one or more processors 380 are also in communication (via a system bus or any suitable electrical circuitry) with an input / output controller 384. The input / output controller 384 receives sensor data from one or more sensors 388-1, 388-2, etc., and relays the sensor data to the one or more processors 380. The input / output controller 384 also receives instructions and / or data from the one or more processors 380 and relays the instructions and / or data to one or more actuators, such as first motors 387-1 and 387-2. In some embodiments, the input / output controller 384 is coupled to one or more actuator controllers 386 and provides instructions and / or data to at least a subset of the one or more actuator controllers 386, which in turn provide control signals to selected actuators. In some embodiments, the one or more actuator controllers 386 are integrated with the input / output controller 384, which provides control signals directly to the one or more actuators 387 (without a separate actuator controller). While FIG. 24 shows that there is one actuator controller 386 (e.g., one actuator controller for the entire mobile medical platform), in other embodiments, additional actuator controllers are used (e.g., one actuator controller for each actuator, etc.). In some embodiments, the one or more processors 380 are in communication with one or more displays 381 for displaying information (e.g., recommended robot arm poses and port locations) as described herein.

[0151] It should be noted that, as used herein, the terms "couple," "coupled," "coupled," or other variations of the word coupled, may indicate either an indirect connection or a direct connection. For example, when a first component is "coupled" to a second component, the first component may be either indirectly connected to the second component through another component, or directly connected to the second component.

[0152] The term "computer-readable medium" refers to any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM), or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that computer-readable media may be tangible and non-transitory. As used herein, the term "code" may refer to software, instructions, code, or data that is executable by a computing device or processor.

[0153] The methods disclosed herein include one or more steps or acts for achieving the described method. Method steps and / or acts may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or acts is required for the proper operation of the described method, the order and / or use of specific steps and / or acts may be modified without departing from the scope of the claims.

[0154] As used herein, the term "plurality" refers to two or more. For example, a plurality of components refers to two or more components. The term "determining" encompasses a wide variety of acts, and thus "determining" can include calculating, computing, processing, deriving, investigating, looking up (e.g., consulting a table, database, or another data structure), ascertaining, and the like. "Determining" can also include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. "Determining" can also include resolving, selecting, electing, establishing, and the like.

[0155] The phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" describes both "based only on" and "based at least on."

[0156] The foregoing description of the disclosed implementations is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these implementations will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other implementations without departing from the scope of the present invention. For example, those skilled in the art will understand that many corresponding alternative and equivalent structural details may be employed, such as similar manners of fastening, mounting, coupling, or engaging tool components, equivalent mechanisms for producing specific actuation motions, and equivalent mechanisms for delivering electrical energy. Thus, the present invention is not limited to the implementations shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0157] [Embodiment] (1) A robotic medical system, comprising: A plurality of robotic arms; one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: acquiring imaging data of a patient; obtaining information regarding a medical procedure for the patient; simulating the medical procedure from a plurality of starting states to identify, for each starting state among the plurality of starting states, a number of collisions that occur with an arm among the plurality of robotic arms, where the imaging data and the information about the medical procedure are used; selecting a first start state of the plurality of start states, the first start state having a minimum number of collisions, the first start state specifying a respective start pose of each arm of the plurality of robotic arms; and providing posture information about the first starting state to an operator of the robotic medical system. (2) The robotic medical system of embodiment 1, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to acquire an anatomical target for the medical procedure, and wherein the simulating further uses the anatomical target. (3) The robotic medical system of embodiment 2, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to identify the anatomical target using the imaging data of the patient. (4) The simulating step comprises: mapping a trajectory of each arm of the plurality of robotic arms during the medical procedure; A robotic medical system according to any one of embodiments 1 to 3, further comprising identifying a collision based on the trajectory. (5) A robotic medical system described in any of embodiments 1 to 4, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to select a recommended port location based on the simulating.

[0158] (6) A robotic medical system according to any one of embodiments 1 to 5, wherein each of the plurality of robot arms has a plurality of joints, and the first starting state includes, for each of the plurality of robot arms, positioning information for each of the plurality of joints. (7) A robotic medical system described in any of embodiments 1 to 6, wherein the memory further includes instructions which, when executed by the one or more processors, cause the one or more processors to acquire imaging data of the peripheral area of ​​the robotic medical system, and wherein the simulating further uses the imaging data of the peripheral area. (8) A robotic medical system described in any one of embodiments 1 to 7, wherein the memory further includes instructions which, when executed by the one or more processors, cause the one or more processors to obtain positional information of one or more objects in the vicinity of the robotic medical system, and wherein the simulating further uses the positional information. (9) A robotic medical system described in any one of embodiments 1 to 8, wherein the medical procedure requires a medical instrument, and the posture information includes information regarding which arm of the plurality of robot arms the medical instrument should be attached to. (10) A robotic medical system described in any of embodiments 1 to 9, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to move at least one arm of the plurality of robot arms to the respective starting posture.

[0159] (11) A robotic medical system described in any one of embodiments 1 to 10, wherein the imaging data of the patient includes images of the patient after the patient has been prepared for the medical procedure. (12) A method for determining a starting state of a robotic system including a plurality of robotic arms, comprising: acquiring imaging data of a patient; Obtaining information regarding a patient's medical treatment; simulating the medical procedure from a plurality of starting states to identify, for each starting state among the plurality of starting states, a number of collisions that occur with an arm among the plurality of robotic arms, where the imaging data and the information about the medical procedure are used; selecting a first start state of the plurality of start states, the first start state having a minimum number of collisions, the first start state specifying a respective start pose of each arm of the plurality of robotic arms; and providing pose information for the first starting state to an operator of the robotic system. (13) The method of claim 12, further comprising obtaining an anatomical target for the medical procedure, and wherein the simulating further uses the anatomical target. (14) The simulating step comprises: mapping a trajectory of each arm of the plurality of robotic arms during the medical procedure; 14. The method of claim 12 or 13, comprising identifying a collision based on the trajectory. (15) The method of any one of claims 12 to 14, further comprising selecting a recommended port location based on the simulating.

[0160] (16) A method according to any one of embodiments 12 to 15, further comprising acquiring imaging data of the peripheral area of ​​the robot system, and wherein the simulating further uses the imaging data of the peripheral area. (17) A method according to any one of embodiments 12 to 16, further comprising obtaining positional information of one or more objects in the vicinity of the robot system, and wherein the simulating further uses the positional information. (18) A method according to any one of embodiments 12 to 17, wherein the medical procedure requires a medical instrument and the posture information includes information regarding which arm of the plurality of robotic arms the medical instrument should be attached to. (19) A method according to any one of embodiments 12 to 18, further comprising moving at least one of the plurality of robot arms to the respective starting posture. (20) A method according to any one of embodiments 12 to 19, wherein the imaging data of the patient includes images of the patient after the patient has been prepared for the medical procedure.

Claims

1. A robotic medical system, comprising: A plurality of robotic arms; one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: acquiring imaging data of a patient; obtaining information regarding a medical procedure for the patient; simulating the medical procedure from a plurality of starting states to identify, for each starting state among the plurality of starting states, a number of collisions that occur with an arm among the plurality of robotic arms, where the imaging data and the information about the medical procedure are used; selecting a first start state of the plurality of start states, the first start state having a minimum number of collisions, the first start state specifying a respective start pose of each arm of the plurality of robotic arms; providing pose information about the first starting state to an operator of the robotic medical system.

2. 2. The robotic medical system of claim 1, wherein the memory further comprises instructions that, when executed by the one or more processors, cause the one or more processors to acquire an anatomical target for the medical procedure and the simulating further uses the anatomical target.

3. 3. The robotic medical system of claim 2, wherein the memory further comprises instructions that, when executed by the one or more processors, cause the one or more processors to identify the anatomical target using the imaging data of the patient.

4. The simulating step comprises: mapping a trajectory of each arm of the plurality of robotic arms during the medical procedure; and identifying a collision based on the trajectory.

5. 2. The robotic medical system of claim 1, wherein the memory further comprises instructions that, when executed by the one or more processors, cause the one or more processors to select a recommended port location based on the simulating.

6. 2. The robotic medical system of claim 1, wherein each arm of the plurality of robotic arms has a plurality of joints, and the first starting state includes, for each arm of the plurality of robotic arms, positioning information for each joint of the plurality of joints.

7. 2. The robotic medical system of claim 1, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to acquire imaging data of a periphery of the robotic medical system, and wherein the simulating further uses the imaging data of the periphery.

8. 2. The robotic medical system of claim 1, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to obtain position information of one or more objects in a vicinity of the robotic medical system, and wherein the simulating further uses the position information.

9. The robotic medical system of claim 1 , wherein the medical procedure requires a medical instrument and the pose information includes information regarding which of the multiple robotic arms the medical instrument should be attached to.

10. 2. The robotic medical system of claim 1, wherein the memory further comprises instructions that, when executed by the one or more processors, cause the one or more processors to move at least one arm of the plurality of robotic arms to the respective start posture.

11. The robotic medical system of claim 1 , wherein the imaging data of the patient includes images of the patient after the patient has been prepared for the medical procedure.

12. 1. A method for determining a starting state of a robotic system including multiple robotic arms, comprising: acquiring imaging data of a patient; Obtaining information regarding a patient's medical treatment; simulating the medical procedure from a plurality of starting states to identify, for each starting state among the plurality of starting states, a number of collisions that occur with an arm among the plurality of robotic arms, where the imaging data and the information about the medical procedure are used; selecting a first start state of the plurality of start states, the first start state having a minimum number of collisions, the first start state specifying a respective start pose of each arm of the plurality of robotic arms; and providing pose information for the first starting state to an operator of the robotic system.

13. The method of claim 12 , further comprising obtaining an anatomical target for the medical procedure, and wherein the simulating further uses the anatomical target.

14. The simulating step comprises: mapping a trajectory of each arm of the plurality of robotic arms during the medical procedure; and identifying a collision based on the trajectory.

15. The method of claim 12 , further comprising selecting a recommended port location based on the simulating.

16. The method of claim 12 , further comprising obtaining imaging data of a periphery of the robotic system, and wherein the simulating further uses the imaging data of the periphery.

17. The method of claim 12 , further comprising obtaining position information of one or more objects in a vicinity of the robotic system, and wherein the simulating further uses the position information.

18. The method of claim 12 , wherein the medical procedure requires a medical instrument and the pose information includes information regarding which of the plurality of robotic arms the medical instrument should be attached to.

19. The method of claim 12 , further comprising moving at least one arm of the plurality of robotic arms to the respective start pose.

20. The method of claim 12 , wherein the imaging data of the patient includes images of the patient after the patient has been prepared for the medical procedure.

Images