Virtual reality surgery device

Abstract

The present invention relates to a system for surgery, which includes a central body, a visualization system operatively connected to the central body, a video rendering system, a head-mounted display for displaying information from the video rendering system, a sensor system, and a robotic device operatively connected to the central body. The visualization system includes at least one camera and a scanning system and / or a tilting system. The sensor system tracks the spatial position and / or orientation of the head-mounted display relative to a reference point. The scanning system and / or the tilting system are configured to adjust the field of view of the camera in response to information from the sensor system about changes in at least one of the spatial position and orientation of the head-mounted display relative to the reference point.

Description

[0001] This application is a divisional application of a Chinese patent application with a filing date of May 5, 2015, national application number 201580023222.8 (PCT application number PCT / US2015 / 029247), and name “Virtual Reality Surgical Device”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 61 / 988,498, filed on May 5, 2014, entitled “Method for Natural Human-Like Motion and Human Interface in Surgical Robotics,” and Provisional Patent Application No. 62 / 136,883, filed on March 23, 2015, entitled “Virtual Reality Surgical Device,” under 35 U.S. C. 119(e), the entire contents of which are incorporated herein by reference. Technical Field

[0004] The present application relates generally to minimally invasive surgery and virtual reality systems. Background Art

[0005] Since the field of minimally invasive surgery began in the early 1990s, it has experienced rapid growth. Although minimally invasive surgery has greatly increased the amount of treatment for patients, this increase is premised on the surgeon's ability to operate accurately and smoothly. During laparoscopy, the surgeon must insert laparoscopic instruments through a small incision in the patient's abdominal wall. Since laparoscopic instruments cannot move left and right without damaging the abdominal wall, the natural insertion of the tool through the abdominal wall limits the movement of the laparoscopic instrument. Standard laparoscopic instruments are limited to four axes of motion. These four axes of motion are the movement of the instrument in and out of the trocar (axis 1), the rotation of the instrument in the trocar (axis 2), and the angular movement of the trocar in two planes while maintaining the pivot point of the entrance of the trocar into the abdominal cavity (axis 3 and 4). In the past two decades, most minimally invasive surgeries have been performed with only these four degrees of freedom of motion.

[0006] Existing robotic surgical devices attempt to address some of these issues. Some existing robotic surgical devices utilize additional degrees of freedom at the end of the instrument to replicate non-robotic laparoscopic surgery. However, even with many costly changes to the surgical procedure, existing robotic surgical devices do not provide increased patient treatment volume in most surgical procedures in which they are employed. In addition, existing robotic devices create an increased separation between the surgeon and the surgical end effector. This increased separation can cause injuries due to the surgeon's misunderstanding of the motions and forces applied by the robotic device. Because many of the degrees of freedom of existing robotic devices are unfamiliar to the operator, the surgeon must perform extensive training on a robotic simulator before operating on a patient in order to minimize the possibility of causing unintentional injuries.

[0007] To control existing robotic devices, the surgeon sits on an operating table and uses his hands and feet to control the manipulators. In addition, the robotic camera is kept in a semi-fixed position and is moved by the surgeon's combined hand and foot movements. These semi-fixed cameras with limited viewing areas lead to difficulties in viewing the operating area.

[0008] Other robotic devices have two robotic manipulators inserted through a single incision. These devices reduce the number of incisions required to a single incision, typically in the umbilical point. However, existing single-incision robotic devices have significant obstruction defects caused by their actuator design. Existing single-incision robotic devices include servo motors, encoders, gear boxes, and all other actuation devices within the body robot. This decision to include motors and gear boxes within the patient's body results in a larger robot with limited performance. This larger robot must be inserted through a large incision, thereby increasing the risk of hernia formation, infection risk, pain, and general morbidity. The incision size required for some existing devices is between 1.5 inches and 2 inches - similar to the incision size of open surgery. In addition, the size of these devices may not be significantly reduced due to the inclusion of motors, gears, etc. within the body device. This increased incision size results in significantly increased damage to the patient and greatly reduces the practicality of existing devices.

[0009] Existing single-incision devices also have limited degrees of freedom. Some of these degrees of freedom are non-intuitive to humans, such as extension of the arms during the surgical procedure. These degrees of freedom require a user interface where the surgeon must make non-intuitive learning movements similar to the movements of existing multi-incision devices.

[0010] Humanoid Robot

[0011] Several people have previously proposed the idea of ​​a surgical robot designed to replicate the degrees of freedom of a human arm. However, all existing designs include very complex gearboxes and gear trains that are disposed within the robotic arm. Due to these gearboxes and gear trains, existing humanoid arms are difficult to manufacture, large in size, and slow in speed. In addition, no previous inventors of humanoid robots have described a human-machine interface designed to fully exploit the advantages of humanoid robots. Without an appropriate human-machine interface, a humanoid arm offers little advantage over alternative robotic designs. Summary of the invention

[0012] In one embodiment, the present invention includes a system for surgery, the system for surgery comprising: a central body; a visualization system operatively connected to the central body, the visualization system including at least one camera; at least one of a panning system and a tilting system; a video rendering system for generating images based on information from at least one camera; a head-mounted display that displays images received from the camera; a sensor system that tracks at least one of a spatial position of the head-mounted display relative to a reference point and a spatial orientation of the head-mounted display relative to the reference point, wherein at least one of the panning system and the tilting system is configured to adjust the field of view of the camera in response to information from the sensor system about changes in at least one of the spatial position and spatial orientation of the head-mounted display relative to the reference point; and a robotic device operatively connected to the central body. In the surgical system, the video rendering system is also configured to adjust the field of view of the produced image based on information from the sensor system. The surgical system may also include a second camera. In a surgical system including two cameras, the image generated by the video rendering system may also include a stereoscopic image based on information from the first and second cameras.

[0013] The surgical system may also include multiple cameras. In a surgical system including multiple cameras, the video rendering system may also generate images based on software that interweaves signal information from the multiple cameras.

[0014] The surgical system may further include at least one sensor that measures at least one of a position and an orientation of the camera. The surgical system includes two cameras, and in the insertion configuration, a cross-sectional dimension of the visualization system in a plane perpendicular to the insertion axis is smaller than a center distance between the first camera and the second camera along the insertion axis.

[0015] The robotic device of the surgical system may further include a first rotation actuator for rotating a portion of the robotic device relative to another portion of the robotic device, and a first articulation actuator for changing an angle between the portion of the robotic device and another portion of the robotic device operatively coupled to the first rotation actuator. The robotic device may further include a position actuator for changing a position of the robotic device relative to the central body, such that the robotic device may be used on a first side or a second side of the central body.

[0016] In other embodiments, the robotic device of the surgical system may further include a first rotational actuator for rotating a portion of the robotic device relative to another portion of the robotic device, a first articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the first rotational actuator, a second rotational actuator for rotating a portion of the robotic device relative to another portion of the robotic device operatively coupled to the first articulation actuator, a second articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the second rotational actuator, a third rotational actuator for rotating a portion of the robotic device relative to another portion of the robotic device operatively coupled to the second articulation actuator, a third articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the third rotational actuator, and a surgical end effector operatively coupled to the third articulation actuator.

[0017] In further other embodiments, the surgical system may further include a second robotic device, the second robotic device including a fourth rotary actuator for rotating a portion of the robotic device relative to another portion of the robotic device, and a fourth articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the fourth rotary actuator, a fifth rotary actuator for rotating a portion of the robotic device relative to another portion of the robotic device operatively coupled to the fourth articulation actuator, a fifth articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the fifth rotary actuator, a sixth rotary actuator for rotating a portion of the robotic device relative to another portion of the robotic device operatively coupled to the fifth articulation actuator, for changing an angle between a portion of the robotic device and another portion of the robotic device, a sixth articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the sixth rotary actuator, and a surgical end effector operatively coupled to the sixth articulation actuator.

[0018] In another aspect, the present invention includes a robotic surgical device including a first cable-driven rotary actuator for rotating a portion of the robotic device relative to another portion of the robotic device, a first cable-driven articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the first cable-driven rotary actuator, a second cable-driven rotary actuator for rotating a portion of the robotic device relative to another portion of the robotic device operatively coupled to the first cable-driven articulation actuator, a second cable-driven articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device operatively coupled to the second cable-driven rotary actuator, a third cable-driven rotary actuator for rotating a portion of the robotic device relative to another portion of the robotic device operatively coupled to the second cable-driven articulation actuator, the second cable-driven articulation actuator for changing an angle between a portion of the robotic device and another portion of the robotic device, a third cable-driven articulation actuator for changing an angle between a portion of the robotic device and another cable-driven portion of the robotic device operatively coupled to the third rotary actuator, and a surgical end effector operatively coupled to the third cable-driven articulation actuator. In some embodiments, the robotic device is disposed partially within the body of a patient. In other embodiments, the robotic device is disposed entirely within the patient's body.

[0019] In still another aspect, the present invention includes a robotic actuator, the robotic actuator comprising: a first body, the first body including a proximal connection component that connects the robotic actuator to a proximal system and a first bearing surface; a second body, the second body including a distal connection component that connects the robotic actuator to a distal system and a second bearing surface that forms a bearing with the first bearing surface, wherein the bearing limits the movement of the first body relative to the movement of the second body in at least one degree of freedom; a pulley or capstan operatively connected to the first body or the second body; an actuator cable configured to actuate the pulley or capstan, and at least one contoured surface defined by the robotic actuator and forming a contoured path to allow multiple additional cables to pass through the path from the system connected to the proximal connection component to the system connected to the distal connection component, wherein the shape and position of the path are such that the length of the additional cables remains substantially constant for substantially the entire range of motion using the robotic actuator.

[0020] The present invention also includes a robotic actuator, comprising: a first body including a proximal connection component; a second body including a distal connection component; a bearing system that limits the movement of the first body relative to the second body in all degrees of freedom except rotation about an axis perpendicular to the distal-proximal axis of the robotic actuator; a pulley or capstan operatively connected to the first body or the second body; an actuator cable configured to actuate the pulley or capstan, and at least one contoured surface defined by the robotic actuator and forming a contoured path to allow multiple additional cables to pass through the path from the system connected to the proximal connection component to the system connected to the distal connection component, wherein the shape and position of the path are such that the length of the additional cables remains substantially constant for substantially the entire range of motion using the robotic actuator.

[0021] The present invention also includes a robotic actuator, comprising: a first body including a proximal connection component; a second body including a distal connection component; a bearing system that limits the movement of the first body relative to the second body in all degrees of freedom except rotation about a distal-proximal axis of the robotic actuator; a pulley or capstan operatively coupled to the first body or the second body; an actuator cable configured to actuate the pulley or capstan, and a hole defined by the robotic actuator, the hole having an inner diameter at least three times the diameter of the actuator cable, the hole being configured so that an additional cable can pass through the hole from a system coupled to the proximal connection component to a system coupled to the distal connection component, wherein the shape and position of the hole is such that the length of the additional cable remains substantially constant for substantially the entire range of motion using the robotic actuator.

[0022] Another aspect of the present invention includes a surgical grasper, the surgical grasper including a main grasper body; a first grasper jaw operatively coupled to the grasper body; a second grasper jaw operatively coupled to the grasper body; an actuation cable, and a linkage mechanism that couples at least one of the first grasper jaw and the second grasper jaw to the actuation cable, wherein the linkage provides nonlinear motion of a distal portion of the first grasper jaw or the second grasper jaw in response to motion of the actuation cable. The surgical grasper may also include a strain gauge fixed to at least one of the main grasper body, the first grasper jaw, the second grasper jaw, the actuation cable, and the linkage mechanism, wherein strain measured by the strain gauge may be used to calculate a force between a distal portion of the first grasper jaw and a distal portion of the second grasper jaw. In another embodiment, the surgical grasper may also include at least one spring operatively coupled to at least one of the first grasper jaw and the second grasper jaw. In another embodiment, the surgical grasper may further include at least one of a software control loop and a hardware control loop for controlling at least one of the force of the grasper jaws and the position of the grasper jaws. In another embodiment, the surgical grasper may also include at least one servo motor operatively coupled to the actuation cable. In another embodiment, the surgical grasper may further include at least one position sensor, where the position sensor measures the position of at least one of the first grasper jaw and the second grasper jaw.

[0023] The surgical grasper may also include a device for providing tactile feedback, where the calculated force is used to determine the tactile feedback force. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Note that the coded items remain consistent throughout the drawings. Items numbered with the same reference numeral are the same item or identical copies of an item. Items numbered with different reference numerals are parts with different designs, or are accidental identical parts used for different purposes.

[0025] Figure 1A is a front perspective view of one embodiment of a dual-arm robotic surgical device configured for use.

[0026] Figure 1B is a rear perspective view of one embodiment of a dual-arm robotic surgical device configured for use.

[0027] Figure 2A is an exploded perspective view of the right arm of a robotic surgical device in one embodiment.

[0028] Figure 2B is an illustration of one embodiment of a dual-arm robotic surgical device with a first actuator oriented in a first position.

[0029] Figure 2Cis an illustration of one embodiment of a dual-arm robotic surgical device with a first actuator oriented in a second position.

[0030] Figure 3A is a perspective view of one embodiment of a dual-arm robotic surgical device configured for insertion through a trocar.

[0031] Figure 3B is a perspective view of a dual-arm robotic surgical device with a trocar fully inserted, according to one embodiment.

[0032] Figure 3C is an illustration of a trocar having an inner sleeve in one embodiment.

[0033] Figure 3D is a diagram showing one embodiment of a patent lumen trocar.

[0034] Figure 4 is a perspective view showing an alternative embodiment of the insert body.

[0035] Figure 5A is a perspective view of a trocar cannula with a camera according to one embodiment.

[0036] Figure 5B is a perspective view of a trocar cannula during insertion through a trocar, according to one embodiment.

[0037] Figure 5C is a perspective view of a trocar cannula with a camera in place during use according to one embodiment.

[0038] Fig. 6A is a front view of a camera configured for use according to one embodiment.

[0039] Figure 6B is a front view of a camera configured for insertion into the abdomen according to one embodiment.

[0040] Fig. 7A is a perspective view of a central connecting member according to one embodiment.

[0041] Figure 7B is an exploded view of a centrally connected main view according to one embodiment.

[0042] Figure 7C is an exploded view of a rear view of a center connection according to one embodiment.

[0043] Fig. 8A is a perspective view of a hinge actuator according to one embodiment.

[0044] Figure 8B is an exploded view of a hinge actuator according to one embodiment.

[0045] Fig.9A is an illustration of one embodiment of a cross section of a hinge actuator actuated to 0 degrees.

[0046] Fig. 9B is an illustration of one embodiment of a cross section of a hinge actuator actuated to 30 degrees.

[0047] Fig. 10A is a side view of a first side of a rotary actuator according to one embodiment.

[0048] Fig. 10B is a side view of a second side of a rotary actuator according to one embodiment.

[0049] Fig. 10C is an exploded view of a rotary actuator according to one embodiment.

[0050] Fig. 10D is a cross-sectional view of a rotary actuator according to one embodiment.

[0051] Fig.11A is a perspective view of a gripper according to one embodiment.

[0052] Fig. 11B is an exploded view of a gripper according to one embodiment.

[0053] Fig. 11C is a cross-section of a gripper when closed according to one embodiment.

[0054] Fig.11D is a cross section of a gripper when opened according to one embodiment.

[0055] Fig. 12A is a diagram showing an open gripper according to one embodiment.

[0056] Fig. 12B is a diagram illustrating a closing gripper according to one embodiment.

[0057] Fig.13 is a perspective view of an extension segment according to one embodiment.

[0058] Fig.14A is a perspective view of a four-arm robotic surgery system according to one embodiment.

[0059] Fig. 14B is a stereoscopic view of a four-arm robotic surgical trocar according to one embodiment.

[0060] Fig.15 is a diagram showing placement of a MEMS sensor on a user wearing a virtual reality head mounted view device according to one embodiment.

[0061] Fig.16 is a block diagram of one embodiment of a virtual reality robotic system.

[0062] Fig.17 is a front view of an apparatus with separate robot and camera systems according to one embodiment. DETAILED DESCRIPTION

[0063] Although the current system is designed for use by a surgeon in the abdominal cavity, alternative uses of the device are possible. For example, the user may be a physician assistant, a nurse, a surgical assistant, or any other surgical personnel. In addition, the device may be arranged in any part of the patient's body, and future embodiments may be designed to be very small to allow use in smaller areas of the patient's body. Smaller and larger devices may be manufactured for use in areas such as the sinuses, colon, stomach, or any other area in the human body. Micromachining using MEMS or other devices may allow the device to be positioned in a very small area such as a human blood vessel. Alternatively, the system may be used to obtain excellent acuity and visualization, even during an opening process in which the device is partially or completely positioned outside the human body.

[0064] In other embodiments, the device is used for non-surgical or non-medical tasks, such as micro-machining assembly of parts, bomb defusing, or any other task requiring fine motor skills. Alternative embodiments of the device can be made with human-sized or even giant arms, allowing humans to perform tasks that they are too small, too weak, or unable to perform. Obviously, in these embodiments, the user may not necessarily be a surgeon.

[0065] The following definitions are used in the detailed description and claims:

[0066] Surgeon: User of the device

[0067] Abdominal cavity: Any closed or semi-closed area into which a device is inserted

[0068] Abdominal wall: The wall that partially or completely encloses the abdominal cavity.

[0069] Trocar: A tube used to insert a device through the abdominal wall.

[0070] Far end: Closer to the end effector of the robot arm

[0071] Proximal: farther from the end effector of the robot arm

[0072] Overall device design

[0073] Figure 1AA stereoscopic view of one embodiment of our device arranged in the abdominal cavity of a patient is shown. The device includes a conduit 100 connected to a central body 101. The central body is arranged in the abdomen of the patient. The conduit includes a hollow tube that traverses the abdominal wall, thereby directing power, signals, control cables, flushing agents, vacuum and any other systems from systems outside the patient's body to the inside of the patient's body. In some embodiments, the conduit 100 includes multiple lumens to separate various systems and cables and provide independent fluid channels. In other embodiments, the conduit includes multiple interlocking segments so that the conduit is flexible while all control cables are slack and become rigid when tension is applied to the control cables in the conduit. This design will operate in a similar manner to a camping tent support pole with interlocking segments connected by cables. When the cables are tensioned and the segments move together, they form a rigid column. In another embodiment, the conduit is a rigid tube that is bent into a nonlinear shape.

[0074] Figure 1A Also shown are a right robotic arm 103 and a left robotic arm 104 attached to the central body 101. Each of these robotic arms includes a plurality of actuators connected to form a humanoid robotic arm. The actuators of each robotic arm are assembled to form a robotic shoulder 105, a robotic elbow 106, a robotic wrist 107, and a surgical end effector 109.

[0075] Figure 1B A camera assembly 102 is shown attached to a central body 101. The camera body is positioned so that it is generally centrally located between and slightly above the robot shoulders 105. The camera body is positioned so that the following ratios hold true.

[0076]

[0077] Although this ratio is approximately equal to 2 for the average person, it is understood that the ratio may vary from person to person. In one embodiment, the device can be manufactured to accurately match the surgeon's ratio, while in another embodiment the general ratio remains approximately the same as that of the average surgeon. In another embodiment, the ratio can be adjusted during use or before insertion into the patient. Alternatively, the ratio can be intentionally increased to reduce the overall vertical height of the device during use. This reduction is used to increase the working area within the patient's abdominal cavity. For maximum versatility of the initial device, one embodiment is designed with a ratio of approximately 4, including a human-like feel for increased versatility of the device. It is assumed that a device with a ratio between 1 and 4 will maintain a sufficiently human-like view for the surgeon.

[0078] According to the above ratios, the camera assembly is positioned in a position that gives a natural, human-like view of the robotic arm. In an alternative embodiment of the device having only one human-like robotic arm, the camera is still positioned so that it maintains a human-like perspective on the arm. In yet another alternative embodiment, the camera moves forward so that the plane formed by the camera assembly 102 and the two robotic shoulders 105 is perpendicular to the plane formed by the central body 101 and the two robotic shoulders. Alternatively, the camera zoom can give the user the impression that the camera has been placed more forward, or an actuator can give the camera body the ability to be advanced during surgery.

[0079] In other embodiments, none of the right robotic arm 103, the left robotic arm 104, and the camera assembly 102 are attached to the central body 101. In these embodiments, the separate components of the system are inserted separately into the patient's abdominal cavity. The components can be inserted through a single trocar or through a number of trocars. The components can be assembled within the abdominal cavity. Alternatively, the components can remain separate, yet be positioned so that the humanoid robot works in sync with natural humanoid visualization. One such embodiment includes a camera system that is inserted separately and supported by its own conduit 234, such as Fig.17 shown.

[0080] Robotic arm design

[0081] Figure 2A The right robot arm 103 ( Figure 1A ). The arm includes a plurality of robot joints. The first actuator 110 ( Figure 2A ) connected to the central body 101 ( Figure 1A ). In one embodiment, a first actuator is used to allow the surgeon to operate on either side of the device and straighten the device for insertion and removal from the abdominal cavity. Figure 2B The first actuators of the left and right arms are shown oriented so that the robotic arms are positioned to operate on one side of the device. Similarly, Figure 2C The left and right arms of the first actuator are shown, oriented so that the surgical arm is positioned to operate on the second side of the device. When operating on the second side of the device, the right arm becomes the left arm and the left arm becomes the right arm. This change is implemented in the software controlling the device.

[0082] In addition, the camera assembly 102 ( Figure 1A) can rotate more than 180 degrees. This range of motion allows the surgeon to place the device anywhere in the patient's abdominal cavity and view the abdomen from any direction. For example, to operate on a patient's gallbladder, the surgeon can place the device on the patient's left side and orient the arm and camera to face the patient's right side. Alternatively, for operating on the stomach, the surgeon can place the device on the patient's right side with the arm and camera orientated to face the patient's left side. This versatility allows one device to be used for many different surgeries.

[0083] Figure 2A Also shown are a second actuator 111 connected to the first actuator 110 and a third actuator 112 connected to the second actuator. The second actuator provides rotational actuation about an axis along the center of the arm. The third actuator provides articulation actuation using rotation about an axis perpendicular to the axis of the second actuator. Together, the second and third actuators provide a robotic shoulder 105 ( Figure 1A ). These degrees of freedom of motion mimic human shoulder abduction adduction and human arm flexion / extension. In alternative embodiments, other actuator types may allow shoulder degrees of freedom including ball-and-socket joint actuators.

[0084] Figure 2A Additionally shown are a fourth actuator 113 and a fifth actuator 114. The fourth actuator connects the third and fifth actuators and provides rotational actuation about an axis along the center of the arm. The rotational actuation of the fourth actuator mimics the motion of a human arm rotating outward / inward. The fifth actuator is connected to the fourth actuator and forms the robot elbow 106 ( Figure 1A ). The fifth actuator provides articulation actuation using rotation about an axis perpendicular to the axis of the fourth actuator. The articulation actuator of the fifth actuator mimics the flexion / extension movement of a human elbow.

[0085] Figure 2A Additionally shown are the sixth actuator 115 and the seventh actuator 116. The sixth actuator connects the fifth actuator and the seventh actuator and provides a rotational actuation around an axis along the center of the arm. The rotational actuation of the sixth actuator imitates the up / down movement of a human palm. The seventh actuator is connected to the sixth actuator and forms a robotic wrist. The seventh actuator provides an articulated actuation using a rotation around an axis perpendicular to the axis of the sixth actuator. The articulated actuator of the seventh actuator imitates the movement of wrist extension and flexion. The surgical end effector 109 is connected to the seventh actuator. In one embodiment, the surgical end effector provides the surgeon with a robotic gripper with a movement similar to the squeezing of the thumb and index finger (first and second fingers).

[0086] According to the combination of actuators as described above, the robotic arm has degrees of freedom that mimic the human arm. Therefore, the arm is able to reproduce human arm movements almost accurately. The specific design of the rotary actuator and elbow actuator as described below enables this multi-degree-of-freedom robotic arm to mimic human movements and fit through a standard 12mm trocar.

[0087] In one embodiment, the rotary actuator does not provide unlimited continuous rotation. Therefore, the arm cannot perfectly imitate all the movements of the human shoulder without limit. When repeated many times, some movements will cause the second actuator to reach its hard limit. In order to overcome this limitation, in one embodiment, the computer control algorithm limits the movement of the shoulder joint, thereby not requiring continuous rotation. The software prevents the surgeon from moving the robot elbow through a virtual plane. As long as the virtual plane is set for each robot arm so that the axis of each second actuator coincides with the plane of its corresponding arm, continuous rotation is never required. The plane can be oriented differently according to the needs of each operation. For example, when operating completely below the device, the plane for the two arms can be parallel to the ground so that the robot elbow never passes above the height of the robot shoulder. Alternatively, for external surgery in front of the device, the plane can be set perpendicular to the ground.

[0088] In one embodiment, the robotic device does not include a degree of freedom to simulate radial / ulnar flexion. Although the human arm has this degree of freedom, our experiments have found that this degree of freedom is not critical for the operation of the device. However, alternative embodiments of the device provide this degree of freedom.

[0089] Insertion and removal of the device

[0090] Figure 3A One embodiment of a dual-arm robotic surgical device configured for insertion into and removal from the abdominal cavity of a patient is shown. For insertion, all articulated joints are positioned in a straight line, such as Figure 3A As shown. In one embodiment, the hinge joints are straightened by removing the force on the cables, thereby allowing all hinge joints to become relaxed. The relaxed joints are manually straightened as needed. In an alternative embodiment, the actuators are driven to a straight position. In some embodiments, the actuators can continue to be powered, still controlled using a damping algorithm that simulates a free motion actuator through damping. In another embodiment, the joints are actuated to a non-linear orientation for passing through a curved trocar as described below.

[0091] In some embodiments, the robotic surgical device can be inserted into the abdomen through a trocar 117. In one embodiment, the trocar is designed with a cross-sectional profile similar to the cross-sectional profile of the device during insertion. During insertion, the device passes through the trocar with minimal clearance to allow for the smallest possible incision in the patient's abdominal wall. In an alternative embodiment, the device can be inserted through a standard commercial trocar.

[0092] Figure 3B The trocar is shown with the device already inserted. Figure 3C and Figure 3D The trocar and the inner cannula 118 are shown. Once inserted, the conduit 100 ( Figure 3B ) occupies the portion 119 of the non-circular trocar 117 ( Figure 3D ). In one embodiment, the inner sleeve 118 of the trocar is configured to hold the tubing in place, thereby retaining a 12 mm circular opening 120 in the trocar. In some embodiments, the inner sleeve forms an air seal with respect to the trocar. In other embodiments, the inner sleeve contains a rubber check valve to help maintain an air seal for surgical infusion. In an alternative embodiment, the trocar is flexible and / or curved. This flexibility will allow curved devices or tubing to be passed through the trocar.

[0093] In some embodiments, the inner sleeve contains a tube that connects a gas port on the outside of the patient and the inside of the patient's abdomen. The tube and gas port allow for insufflation into the abdominal cavity. Alternatively, the trocar 117 may contain such a tube and gas port to allow for insufflation. In some embodiments, the trocar contains a check valve to maintain insufflation pressure prior to insertion of the robotic device. In other embodiments, a removable plug blocks the opening of the trocar for insufflation with the robotic device removed.

[0094] Figure 4 An embodiment of a device positioned in an insertion body 121 is shown. The insertion body allows the device to be easily moved into the abdomen of the patient. The device can be transported in the insertion body to prevent the need for the surgeon to manually straighten the hinge joint of the device. The surgeon only slides the distal end of the insertion body 122 through the trocar. The distal end of the insertion body is formed to be soft and rounded to avoid hurting tissue during insertion. The surgeon slides the insertion body into the patient's abdominal cavity until he or she encounters resistance, indicating contact with the abdominal wall or organ. Upon contact, the surgeon retracts the insertion body while advancing the device. When the device is out of the protection of the insertion body, the loose hinge joint bends to allow the device to curl in the abdomen. In some embodiments, the insertion body distal end 122 includes a sensor to detect contact with and / or proximity to the abdominal wall using any standard sensing device (capacitance, resistance, conductivity, pressure, etc.). In other embodiments, the entire insertion body is flexible and / or curved.

[0095] The insertion body can also assist in the removal of the device. An assistant can place the insertion body through a trocar. The surgeon can move the arm closest to the insertion body into the insertion body so that the insertion body slides over the entire device when removing the insertion body from the abdomen.

[0096] Figure 5A Shown is an additional trocar inner sleeve 123. This sleeve is extremely thin and is inserted into the trocar once the trocar is positioned across the abdominal wall. A camera 124 is attached to the end of the trocar inner sleeve so that it does not block the opening of the trocar. Figure 5B The additional inner sleeve of the trocar is shown as it is inserted through the trocar. Figure 5C An additional trocar inner sleeve is shown in place and fully inserted into the trocar. Once inserted, the camera provides visualization of the abdominal cavity and assists in the safe insertion and removal of the device into and from the abdominal cavity. The camera may also include a light source and other sensors to assist and acquire data during surgery. The camera may additionally include multiple cameras that provide multiple views within the abdomen.

[0097] Cameras and visualization systems

[0098] Fig. 6A A camera and visualization system configured for use within the abdominal cavity of a patient is shown. Figure 6B The camera and visualization system are shown configured for insertion through a trocar. The camera system is positioned in a trocar-like manner by actuating the hinge joint 126 and the ball and socket joint 127. Fig. 6A The use position shown is the same as Figure 6B Move between the insertion positions shown. Figure 7C The hinge joint 126 is best viewed in the image. The hinge joint and ball joint can be actuated using any standard actuation means including cables, motors, magnets, electromagnets, etc. In one embodiment, the camera can be actuated to Fig. 6A The spring-actuated hinge joint is shown in the use position. When the device is retracted through the trocar, the direction of the force applied by the end of the trocar forces the spring-actuated hinge joint to move to Figure 6B In the insertion position shown, the position is to allow it to fit within a trocar or insertion tube.

[0099] Fig. 6AThe camera system shown comprises a first camera 128 and a second camera 129 arranged in the camera body 125, near the camera body 125 or arranged on the top of the camera body 125. The camera body pivots with two degrees of freedom by the actuation of a ball joint 127. The motion of these two degrees of freedom forms the panning system and the tilting system of the camera. The panning system adjusts the visual field of the camera along the panning axis, and the tilting system adjusts the visual field of the camera along the tilting axis simultaneously. In an alternative embodiment, two hinge joints or a swivel joint and a hinge joint are utilized to replace the ball joint. In yet another embodiment, the swivel joint rotates the camera body around the vertical axis, and tilting motion is provided by digitally adjusting the camera visual field simultaneously. Position sensors accurately measure the position of each joint that moves the camera body. Position sensors can comprise any one of any other standard devices of a Hall sensor, an optical encoder, a resistance position sensor or a measurement position.

[0100] In another embodiment, the camera moves in the camera body so that one or both of the pan adjustment and the tilt adjustment are provided by the movement of the camera in the camera body. This adjustment can be used in conjunction with the camera body movement, or alternatively the camera body movement. The camera can move together or move separately. The camera can be moved by motor actuation, cable actuation or any other standard actuation device. Alternatively, the camera can rotate freely with two degrees of freedom and move under the guidance of the magnetic field generated by the electromagnetic coil surrounding the camera.

[0101] In certain embodiments, panning motion and tilting motion are provided by digital panning and tilting adjustment. Digital adjustment is provided by trimming the digital camera image. The trimming image is automatically adjusted so that when panning motion or tilting motion are needed, the partial changes of the image displayed to the user are produced thus as an illusion of camera movement. In another embodiment, a combination of digital machinery and mechanical adjustment is adopted so that digital panning and tilting adjustment are realized as small and fast adjustment, while mechanical panning and tilting adjustment allow larger panning and tilting motion.

[0102] In another embodiment, the camera assembly is inserted into the abdomen as a separate unit from the rest of the device. The separate camera assembly can be removably coupled to the device inside the abdominal cavity, or it can be used as a separate unit.

[0103] In some embodiments, the camera has a wide angle lens that allows wide visualization of the operating area. In other embodiments, the camera has an aspherical lens that allows a wide vertical field of view with a narrow horizontal field of view. Distortion is removed using digital adjustments. This wide vertical field of view allows tilt motion to be provided using only digital technology. In yet another embodiment, the camera body 125 includes multiple camera devices that further increase the field of view. The camera fields of view are digitally interlaced to form a large image with a panoramic field of view. Standard digital techniques known in the art are used to interlace the images. In another embodiment, the camera body additionally includes other sensors or any other sensor devices that sense any one of pressure, capacitance, temperature, infrared, ultraviolet.

[0104] In one embodiment, the camera body 125 also includes a row of LEDs positioned between the first camera 128 and the second camera 129. These LEDs are used to illuminate the operating area. These LEDs are powered via wires fed to the outside of the body. Heat is lost from the LEDs within the camera body. In some embodiments, a small amount of sterile saline or other biocompatible fluid can flow through the camera body to cool the camera body. Other embodiments also include a temperature sensor to ensure that the camera body remains within a safe temperature range. In another embodiment, the LEDs are set in other bodies of the device that provide light for different angles and a larger heat sink body.

[0105] It is believed that the abdomen can also be illuminated via an optical fiber or another light source. The optical fiber can be supplied to the body using an actuating cable or through another incision. In one embodiment, the optical fiber is screwed into the abdomen through, for example, a 21-caliber vascular catheter. The fiber can be matched to the device inside the abdomen, or can be used to provide illumination without matching the device. This lighting system will be used to provide enhanced light with reduced heat, but at the expense of increased complexity of the entire system.

[0106] The camera body is inserted so that its field of view is perpendicular to the direction of insertion through the trocar. This allows the camera to be placed on or in the camera body 125 ( Fig. 6A ), such that the intermediate camera distance 130 exceeds the size of the incision through which the device is inserted. Based on the increased intermediate camera distance, the camera system has improved ability to observe the amount of parallax and allow the user to perceive depth. The intermediate camera distance is selected to maintain a natural and human-like system such that

[0107]

[0108] Human-device interaction

[0109] The natural human machine interface (HMI) is designed to optimally utilize humanoid robotic devices and natural visualization systems. One embodiment allows the surgeon to control the device using the motion of his own arm. The surgeon wears a series of elastic bands; each band fastens the sensor to the surgeon's arm. In one embodiment, the sensor is an MPU-6050 sensor. The MPU-6050 includes a MEMS gyroscope, an accelerometer, and a digital motion processor to calculate the orientation of the sensor.

[0110] Reference Fig.15 -A, in one embodiment, the surgeon wears eight elastic bands 226 and 227. These bands secure eight MPU-6050 sensors to the surgeon's arms, such as Fig.15 As shown. One band is set on each of the left and right index fingers 224, the back of the hand 223, the dorsal side of the distal forearm 222 and the dorsal side of the distal upper arm 221. The housing containing each upper arm sensor also contains a microcontroller, a battery and a Bluetooth module. Data from the remote sensors are collected along the wire 225 using the I2C protocol and transmitted to the central computer via Bluetooth.

[0111] Based on data from eight MPU-6050 sensors, the central computer is able to calculate the position and orientation of each part of the surgeon's arm. Future solutions include tracking the surgeon's torso or any other body part. In addition, alternative embodiments include adding a MEMS magnetometer with each accelerometer, gyroscope, and motion processor unit. A MEMS chip such as the MPU-9250 enables all of the above components to be located in a single package. Since the magnetic head allows the sensor offset around the vertical axis to be reduced, the addition of a magnetometer is conventional practice in the art. Alternative embodiments also include being set in surgical materials such as gloves, surgical brushes, or surgical gowns. These sensors can be reusable or disposable.

[0112] Yet another embodiment includes adding sensors that track the position of the surgeon's arms and body. Such sensors are similar to those on an Xbox. The sensors allow tracking the absolute position of the surgeon's arms and tracking the arm positions relative to each other. In some embodiments, these additional sensors are worn on the surgeon's body. In other embodiments, the sensors are positioned at fixed locations in the room.

[0113] Based on the ability to track the surgeon's arm, a control loop within the central computer drives the servo motors that control the device's humanoid robotic arms. Fig.16The arm is controlled to follow the movements of the surgeon's arm on a scaled-down basis. The robotic elbow follows the position and orientation of the human elbow. The robotic wrist follows the position and orientation of the human wrist. The surgical end effector follows the movements of the surgeon's index finger as the surgeon presses his index finger and thumb together.

[0114] As the arms of the device follow the movements of the surgeon's arms, the shoulders of the device are fixed in place. In one embodiment, the position and orientation of the surgeon's torso is subtracted from the position and orientation of the surgeon's arms. This subtraction allows the surgeon to move his torso without moving his arms. Alternative embodiments include a chair with pads to encourage the surgeon to keep his shoulders fixed in place. By preventing the surgeon from moving his shoulders, the surgeon avoids making movements that the device cannot reproduce, thereby increasing the natural feel of the device.

[0115] The surgeon wears a virtual reality head mounted display 220 ( Fig.15 ) in order to optimally view the device. A suitable head mounted display, such as the Oculus Rift, provides the user with a head mounted display, a lens that allows the field of view of the display to be focused, and a sensor system that provides position and orientation tracking of the display. The position and orientation sensor system may include accelerometers, gyroscopes, magnetometers, motion processors, infrared tracking, computer vision, any other method of tracking at least one of position and orientation, or any combination thereof. With many displays appearing on the market, it is important to select the best display for our system. Display features that generally result in improved device functionality for our system include increased field of view, reduced latency, reduced persistence, increased resolution, reduced weight, increased comfort, and improved display position and orientation tracking.

[0116] Through head mounted display, the computer Fig.16 In one embodiment, the video from the first camera 128 and the second camera 129 ( Fig. 6A ) are collected and processed as described later in this section. The processed video from one camera 128 is displayed to the surgeon's right eye. Similarly, the processed video from one camera 129 is displayed to the surgeon's left eye. The combination of the left and right eye views from separate cameras spaced apart in the abdominal cavity provides a stereoscopic image to the surgeon.

[0117] To maintain a full virtual reality experience, a sensor system tracks the position and orientation of the surgeon's head-mounted display. The sensor system relays data to a central computer in real time. The central computer adjusts the pan and tilt of the device's camera system as quickly as possible to follow the movement of the user's head. Because it is difficult to adjust the pan and tilt of the camera quickly enough so that the surgeon cannot perceive the delay, the software slightly adjusts the camera field of view to compensate for any difference between the camera position and the surgeon's head position.

[0118] While some embodiments provide only visual feedback to the surgeon, alternative embodiments provide many additional feedback systems. In one embodiment, the surgeon holds a device to provide tactile feedback. Such a device can be as simple as a small servo motor connected to two members. As the surgeon squeezes between the members, the servo motor resists motion. With the servo motor providing tactile feedback and the position and force sensed in the robotic gripper, standard force control algorithms can be used to enable the surgeon to "feel" the force applied by the gripper.

[0119] In an alternative embodiment, the surgeon has an exoskeleton-like device to wear on each of his arms. Such a device would contain the servos for each actuator of the robotic arms and would allow the surgeon to experience tactile feedback for each robotic actuator. In yet another embodiment, the surgeon interacts with the device using known standard tactile interaction devices commercially available today.

[0120] In one embodiment, the movement of the surgeon's arm is converted into movement of the device's arm using only a direct ratio. However, other embodiments may include an adjustable ratio of the movement. In one embodiment, the movement is also scaled down so that a 10 degree movement of the surgeon's elbow causes a similar 5 degree movement of the device's elbow. This ratio allows for increased agility in exchange for a reduced natural feel of the device. Another embodiment includes an adjustable ratio where the scaling factor is related to the speed of the movement. For example, if the surgeon's elbow moves 10 degrees at 10 degrees per second, the device's elbow moves 3 degrees. If the surgeon's elbow moves 10 degrees at 50 degrees per second, the device's elbow moves 15 degrees.

[0121] Fig.16The block diagram of provides an overall view showing how the device as a whole collects and utilizes information. Sensors 229 track the movements of the surgeon's body and communicate that information to a central computer. The central computer contains a control loop 230 and video execution and augmentation software and hardware 231. Information about the surgeon's arm and body position is used to calculate predetermined robotic actuator positions. The control loop calculates the power output to the servo motor 232 using the desired robotic actuator position in combination with processed values ​​from the servo motor, encoder, servo motor torque, device encoder, and any other related systems. These control loops can use standard tuned proportional integral derivative "PID" control to determine the power output to the servo motor. Alternatively, a custom control loop can be used. The servo motor is connected to the device 233 inside the patient as described later in this application.

[0122] The device inside the patient collects the video signals from the camera system and transmits these signals to the video execution and augmentation system 231 of the central computer. This system combines information about the position and orientation of the camera, the video signal, and the position and orientation of the surgeon's head. Based on this information, the video execution and augmentation system generates a video signal and transmits the signal to the surgeon's virtual reality display 228. It should be noted that this block diagram generally describes the device, and alternative embodiments have additional sensors and elements and additional connections between the block diagram parts, allowing more complex processing and use of data within the system.

[0123] Augmented Reality

[0124] In order to further improve the surgeon's ability to operate, reality can be increased to provide enhanced information to the surgeon. This enhancement of reality is used to further improve the surgeon's ability to operate by adding to the virtual reality experience. For example, in one embodiment, the camera of the device has an image magnification function. Since the surgeon's own eyes cannot zoom according to instructions, it would be unnatural for the surgeon to use this image magnification function during operation. However, using animation, the surgeon can select a magnifying glass or a magnifying glass for the eye contact, and have the glass be located in front of his actual eye during use. This enhanced reality allows the surgeon to feel as if it increases the focal length, thereby maintaining a natural virtual reality connection between the surgeon and the device.

[0125] In another embodiment, the surgeon can place an augmented reality element within the patient's abdomen. For example, to view a radiographic scan of the patient, the surgeon may choose to place a virtual computer monitor within the patient's abdominal cavity (possibly in an area outside of the surgical field). The virtual reality monitor allows the surgeon to view the image without leaving the operating virtual reality.

[0126] In another embodiment, the computer tracks the position of the robotic arm within the surgeon's field of view. If the surgeon applies too much force, the computer increases the color of the robotic arm to appear red within the surgeon's field of view. Similarly, the robotic arm or surgical end effector can be set to change color when the surgeon is able to cauterize on a cautery instrument.

[0127] Center connection system

[0128] Fig. 7A and Figure 7B A central connection system is shown. The central connection system serves multiple purposes, including support of the arm, support of the camera assembly 102, and routing of cables and power systems. The central connection system includes a main central body 131 that is connected to the pipe 100. The connection is achieved via any standard attachment method known to those skilled in the art, such as splines, press fits, glues, welding, or any other existing attachment means. Figure 7B A cable track 137 is shown for connecting wires in an embodiment having four arms as described later. A front cover 133 holds the cables within the track.

[0129] The cable enters the patient's body through a tube such as Figure 7C The pulley system 132 shown in the exploded view of FIG. 1 is routed to the appropriate actuator. These pulleys capture the cable in a v-shaped groove 135 so that the cable remains in place even if there is slack. The shaft 136 for the pulleys is machined in place or fits into a pre-drilled hole. The rear cover 134 is screwed in place by bolts that screw into the tapered holes 138.

[0130] While a central connection system as used for one embodiment allows the entire system to be inserted through one trocar, alternative embodiments allow multiple single-arm units to be inserted through separate trocars. For example, three trocars can be used to introduce two humanoid robotic manipulators and a virtual reality camera. This configuration would accomplish the same virtual reality surgery without the need for a central connection system. However, this would increase the number of incisions required for the patient and reduce the maneuverability of the device once inserted.

[0131] Hinge Actuator Design

[0132] The first actuator 110 ( Figure 2A ), the third actuator 112, the fifth actuator 114 and the seventh actuator 116 are hinge actuators. In one embodiment, the hinge actuators are cable driven to provide appropriate torque and speed in the smallest possible space. The small actuator design allows insertion through a very small incision. Fig. 8A and Figure 8BA perspective view of a hinge actuator is shown. The hinge actuator includes two concave parts 139 with multiple bolt holes 140, which are used to attach the concave parts to another actuator or other device elements. The attachment forms a proximal connection part. Alternative designs include attachments to other known devices. These methods may include the manufacture of the concave hinge parts and the proximal attachment parts as a single body. The concave parts are spaced apart, leaving an area for the convex parts of the actuator. Each concave part includes multiple string guide holes 142 for allowing strings, cable wires and other systems to pass through the actuator from the proximal connection system to the distal actuator. In an alternative design, the string guide holes are replaced by grooves or single holes to allow the system to pass similarly. In addition, each concave hinge part includes a concave bearing surface 141. In one embodiment, the concave bearing surface includes a smooth machined surface. However, alternative designs include ball bearings, needle bearings, hydraulic bearings or any other bearing type.

[0133] Fig. 8A and Figure 8B The main hinge body 143 is further shown. The main hinge body includes a distal connection component, which includes a plurality of bosses 144 at its distal end for connecting to another actuator or system. These bosses include a plurality of threaded holes 145 to allow the distal system to be attached to the actuator bosses. Although one embodiment includes attachment using bolts, any attachment means may be appropriate, including making the distal attachment component and the main hinge body as a single body. The main hinge body includes a plurality of string guide holes 153 for allowing strings, cable wires and other systems to pass through the actuator from the proximal connection system to the distal actuator. These string guide holes function in the same manner as the string guide holes 142 of the concave component. The main hinge body also includes a pulley or capstan 146. The capstan is a device by which the hinge joint is actuated. Two actuation cables extend from a conduit 149 in the proximal attachment body along a path 147 to the pulley. The cables continue to wrap around the pulley and terminate at a cable terminal position 148.

[0134] In one embodiment, the cable is terminated by clamping the cable between a rigid surface and a set screw disposed in the threaded hole 150. Alternatively, the cable can be terminated in any suitable manner known in the art. For example, a polymer optical cable can be terminated by a node bundled in the cable, or a metal optical cable can be terminated by a crimp connection. In one embodiment, each of the two actuation directions is provided with an independent cable spirally wound around the pulley in opposite directions. An alternative design includes a single cable that enters via the first actuation cable conduit hole 149, wraps around the pulley once or multiple times, and is led out via the second cable conduit hole. Although this design eliminates the need for a cable terminal in the actuator, it allows the cable to slide relative to the pulley. Any embodiment may be appropriate depending on the specific application. In another embodiment, a single-core cable actuates the hinge actuator in one direction, while a spring or other energy storage device provides actuation in the opposite direction.

[0135] Fig. 8A and Figure 8B Also shown are contoured surfaces 152 made into both sides of the main hinge body 143. This contouring allows actuator cables, wires and any other systems to pass through the hinge joint from the proximal actuator and device to the distal actuator and device. Fig.9A and Fig. 9B A cross-sectional view showing the functionality of the profile path is shown. The path is shaped so that it passes through the cable and system 158 ( Fig.9A and Fig. 9B ) is always maintained at approximately the same length. The above length is relatively unchanged while the main hinge body 143 rotates relative to the concave hinge component. The molded shape is manufactured using CAD analysis of the cable passing through the actuator. In one embodiment, the outer shape is adjusted until the cable has approximately 1% of its starting length remaining at all actuation positions. In the current embodiment, the molded surface is an aluminum style. The polymer optical cable passes along the surface. Alternative designs include any of a surface in the form of a low-friction material (or coated with a low-friction material), a cable coated with a low-friction material, or a surface in the form of a wear-resistant material. In addition, the surface may further include a plurality of capture rollers, balls, or pulleys to further reduce the friction between the surface and the cable.

[0136] With the addition of the contoured surface 152, the hinge actuator can be actuated through a wide range of angles without consequential actuation of the distal system. The contoured surface allows many cables to pass through the hinge actuator without coupling motion. This allows multiple cable driven actuators to be distally attached to the hinge actuator without any significant coupling motion between the actuators. Most alternative actuator designs used in surgery today allow several cables to pass through with mild coupling motion. The contoured surface allows for almost complete decoupling of motion from multiple cables, thereby allowing a seven degree of freedom robotic arm to fit through a 12 mm trocar.

[0137] In some embodiments, the length of the actuator cable within the actuator changes by less than about 10% as the actuator moves through a range of motion of 110 degrees. In other embodiments, the length changes by less than about 9%. In further embodiments, the length changes by less than about 8%. In further embodiments, the length changes by less than about 7%. In further embodiments, the length changes by less than about 6%. In further embodiments, the length changes by less than about 5%. In further embodiments, the length changes by less than about 4%. In further embodiments, the length changes by less than about 3%. In further embodiments, the length changes by less than about 2%. In further embodiments, the length changes by less than about 1%.

[0138] Each hinge actuator also includes two convex hinge bodies 154, such as Figure 8B As shown. Each male hinge body has a curved profile surface 152, which has the same profile as the curved profile surface of the main hinge body 143. The profile surface provides the same function as the profile surface of the main hinge body. In addition, each male hinge body includes a plurality of bolt holes 157 for attaching the male hinge body to the main hinge body. The bolts are screwed into the threaded holes 151 on the main hinge body to semi-permanently fix the male hinge body to the main hinge body. The male hinge body also includes a convex boss 155 that acts as a hinge pin for the hinge joint. The bearing surface 156 along the convex boss works in conjunction with the bearing surface 141 on the female hinge component 139. These surfaces provide bearings for the hinge joint. The bearings work together to form a bearing system. Like the concave bearing surface, the convex bearing surface can be a cylindrical machined surface, or alternatively can include a race and any type of bearing such as a ball, needle, fluid, etc.

[0139] Although one embodiment utilizes a plurality of bolts to fasten the male hinge body 154 to the main hinge body 143, any alternative fastening method may be acceptable. An alternative design fastens the body by adhesive. In another design, the body is manufactured as a single body, thereby eliminating the need for fastening means. Manufacturing as one part will increase the complexity of the manufacturing operation while reducing the complexity of the assembly.

[0140] In one embodiment, all hinge actuators are designed to be identical, however alternative embodiments include different hinge actuators to meet the specific needs of the first, third, fifth, and seventh actuators. For example, the first actuator may require a range of motion from negative 50 degrees to positive 50 degrees. Thus, an actuator as currently designed will work ideally. However, the fifth actuator (representing an elbow joint) may require a range of motion from approximately 0 degrees to 160 degrees. Thus, an improved actuator with an appropriate range of motion would be ideal for this joint.

[0141] Additionally, alternative embodiments may include hinge-type actuators that differ significantly from the actuators of one embodiment. The actuators may include motors, gear boxes, pistons, or any other means of joint actuation.

[0142] In another embodiment, the actuator also includes an encoder for measuring the position at the joint. The measurement is obtained using a Hall effect encoder similar to the AS5055 of amsAG. This encoder is easily assembled in the hinge joint and provides real-time measurement values ​​with 12-bit resolution of the position. Alternatively, any device for measuring the position can be used, including optical and resistive encoders. The data is transmitted to the system outside the abdominal cavity via a wire. In some embodiments, the data is transmitted wirelessly. In other embodiments, data is transmitted using a conductive actuator cable as a wire. The actuator cable or its individual twisted wires are coated with an electrical insulator, allowing the cable to transmit at least one of electrical power and data in addition to transmitting mechanical power.

[0143] An alternative embodiment further comprises at least one strain gauge fixed on a member subjected to strain during loading of the hinge joint.Such a strain gauge allows the force subjected to by the actuator to be measured.

[0144] Rotary Actuator Design

[0145] The second actuator 111, the fourth actuator 113 and the sixth actuator 115 ( Figure 2A ) is a rotary actuator. Fig. 10A , Fig. 10B and Fig. 10C A view showing a rotary actuator of one embodiment. As with the hinge actuator used in one embodiment, the rotary actuator is cable driven. The actuator is designed to provide maximum torque and speed with minimum size.

[0146] The rotary actuator includes a rotary concave body 160. The rotary concave body has a proximal connection component, which includes a plurality of proximal connection bosses 179 each having a plurality of bolt holes 172. Bolts fix the proximal connection bosses to the proximal actuator or structure. Although one embodiment utilizes bolts as fastening means, various alternative fastening means are acceptable. In an alternative design, the rotary concave body and the proximal structure fastened thereto are manufactured as a single body.

[0147] The rotating convex body 159 is inserted into the rotating concave body 160. The rotating convex body has a distal connection protrusion 180 ( Fig. 10A and Fig. 10B ) of the distal connection part, the distal connection protrusion 180 each has a plurality of branch holes 168 ( Fig. 10C ). Bolts screwed into the tapping holes are used to connect the distal system to the rotating male body. Again, any alternative attachment means may be appropriate.

[0148] By including two ball bearings, a small ball bearing 161 and a large ball bearing 162 ( Fig. 10A and Fig. 10B ) bearing system constrains the rotating convex body relative to the rotating concave body. These ball bearings act by Fig. 10D The V-shaped bearing races 173 and 174, best seen in the cross-sectional view of FIG. 1 , support both axial and radial loads. With the V-shaped races, the bearing is able to substantially accommodate axial and radial loads, but the bearing is no longer a perfect rolling element bearing. In the bearing ball 181 ( Fig. 10C ) experiences a small amount of sliding as it rolls along the bearing race, but this sliding is minimized by the very small balls. In one embodiment, the diameter of the bearing balls is approximately 1 mm. The associated wear is small due to the low RPM of the device.

[0149] In order to minimize the size of the device, ball bearings are built into the device. The large ball bearing 162 has two bearing races 173. One race is formed into the distal surface of the rotating concave body, while the other race is formed into the large bearing race ring 163. Although one embodiment includes a bearing race formed into a separate ring to reduce manufacturing costs, an alternative embodiment includes a bearing race formed directly into the rotating convex body. This alternative design increases the complexity of manufacturing while reducing the size of the device.

[0150] A small ball bearing 161 is disposed at the proximal end of the actuator. The small bearing is used to limit the rotating convex body 159 on the proximal side of the rotary actuator in the rotating concave body 160. The small ball bearing is smaller than the large ball bearing 162 so as to be assembled between the proximal connection bosses 179. The small ball bearing includes a V-groove race formed in the small ball bearing race ring 164 and the spline ball bearing race ring 165. The spline ball bearing race has a spline on its inner surface, which matches the spline formed on the surface of the proximal end of the rotating convex body. Finally, a nut 166 is screwed on the proximal surface of the rotating convex body. The nut squeezes both the small ball bearing and the large ball bearing, and is used to clamp the rotating concave body between the two bearings using a preload applied by rotating the nut. The preload limits the rotating convex body to the rotating concave body in all degrees of freedom except for the predetermined actuation axis.

[0151] In addition, the spline surface between the rotating convex body and the splined ball bearing race ring 165 prevents the splined ball bearing race ring from rotating relative to the rotating convex body during use. This eliminates any torque on the nut that may loosen the nut over time. In an alternative embodiment, the nut is held in place by a thread locker, adhesive, cotter pin, or any other device. In another alternative embodiment, the nut is replaced by an E-shaped clip or other attachment device, and the rotating convex body can be tightened during attachment to provide preload for the bearing. In another embodiment, one or both of the small bearing race ring and the splined bearing race ring are removed, and the bearing race is formed by one or more of the nut and the rotating concave body. This change reduces the overall length of the device and the number of device components.

[0152] It should be noted that although both the small ball bearing and the large ball bearing in the rotary cable actuator are V-shaped race ball bearings, any type of bearing capable of supporting the appropriate load may be used. For example, an alternative embodiment utilizes four ball bearings, two each for thrust and axial loads. Another alternative embodiment utilizes a plain bearing for axial loads and a rolling element bearing for thrust loads. Yet another embodiment utilizes only plain bearings for axial and thrust loads. Although plain bearings have increased friction, their reduced size and complexity are advantageous in some applications. Another embodiment includes a tapered needle roller bearing, which is theoretically ideal for the type of loads experienced in a rotary cable actuator, but is more difficult and expensive to manufacture.

[0153] Since the rotating male body 159 is fixed in the rotating female body 160, the actuating cables extend along the path from the actuator attached at the proximal end. These cables are fed into the space between the rotating male body and the rotating female body. Fig. 10DA cross-sectional view of a rotating cable actuator is shown. In this cross-sectional view, it is easy to see the cable separation ridge 175 formed in the rotating female body 160. The separation ridge is used to form an almost completely closed recess 176 for each actuator cable wrapped around the rotating male body 159.

[0154] Each of the actuation cables extends along the shaped paths 169 and 177 to the actuator ( Fig. 10A and Fig. 10B These paths allow the actuator cable to enter the enclosed recess 176 ( Fig. 10D ). A path is formed in the rotating concave body 160 and is shaped so that the cable extends along a smooth path into the enclosed recess and surrounds the rotating convex body. It is important to note that each of the two actuator cables is wound around the rotating convex body in opposite directions. Therefore, the rotating convex body forms two pulleys, one for actuation in each rotational direction. Each actuation cable is wound around the rotating convex body multiple times, allowing multiple actuation rotations. After winding around the rotating convex body, the actuation cable enters the rotating convex body via the cable terminal hole 171 and is rigidly attached to the rotating convex body by a set screw set in the threaded hole. The hole 170 in the rotating concave body allows a wrench to access the set screw.

[0155] Alternative embodiments of the rotary actuator include attachment of a termination actuating cable to any suitable alternative means. For example, the cable may be terminated using a knot bundled in the cable. Other alternative terminal means include adhesives, crimp connections, clamps, capstans, welding, and any other suitable means. Alternatively, the actuating cable may not be terminated at all. A single actuating cable may be inserted from one side of the actuator, wrapped multiple times around the rotating convex body, and led out from the other side of the actuator. Such a rotary actuator will allow continuous rotation, yet will also allow the cable to slide along the rotating convex body.

[0156] The center of the rotating male body is hollow. This forms a hole 178 through which the actuator passes. The hole is Fig. 10D is best seen in the cross-sectional view, while Fig. 10C178. The holes are critical to the function of the device because they allow cables and other systems to pass through the rotary actuator from the proximal actuator and systems to the distal actuator and systems. Chamfer 167 reduces wear on the cables by removing sharp edges as the cables pass through the holes. By providing a straight path through the center of the rotary actuator, the cables pass through with little change in length when the rotary actuator changes position. This is very important because the distal actuator is almost completely decoupled from the motion of the rotary actuator. It should be noted that a certain amount of coupled motion will occur due to the torsion of the cable passing through the hole 178, but the torsion provides a negligible effect on actuation within one complete rotation (360 degrees) and a negligible effect on actuation within two complete rotations. Of course, due to more than one cable passing through the hole, the rotary actuator cannot perform infinite rotation due to the torsion of the cable bundle.

[0157] The inclusion of hole 178 in the cable driven rotary actuator enables the multi-DOF device to be assembled through a 12 mm trocar. By providing a small cable driven actuator with a means to allow the cable to pass through the actuator with a near-discrete motion, multiple rotary actuators can be used in series in a robotic arm. The cables from the distally attached rotary actuator and any distally attached system can simply pass through hole 178, thereby allowing daisy chaining of actuators with sufficient decoupling motion.

[0158] In some embodiments, the length of the actuator cable within the actuator changes by less than about 20% as the actuator moves through a 360-degree range of motion. In other embodiments, the length changes by less than about 19%. In further embodiments, the length changes by less than about 18%. In further embodiments, the length changes by less than about 17%. In further embodiments, the length changes by less than about 16%. In further embodiments, the length changes by less than about 15%. In further embodiments, the length changes by less than about 14%. In further embodiments, the length changes by less than about 13%. In further embodiments, the length changes by less than about 12%. In further embodiments, the length changes by less than about 11%. In further embodiments, the length changes by less than about 10%. In further embodiments, the length changes by less than about 9%. In further embodiments, the length changes by less than about 8%. In further embodiments, the length changes by less than about 7%. In further embodiments, the length changes by less than about 6%. In further embodiments, the length changes by less than about 5%. In further embodiments, the change in length is less than about 4%. In further embodiments, the change in length is less than about 3%. In further embodiments, the change in length is less than about 2%. In further embodiments, the change in length is less than about 1%.

[0159] Grip design

[0160] To allow for maximum utility of the device, the end effector of one embodiment is designed to accomplish multiple tasks. During non-robotic minimally invasive surgery, the surgeon can simply remove an instrument and insert another instrument. However, with our surgical device as described in one embodiment, it may be impractical to dismantle the device to change the end effector. For this reason, a universal end effector is incorporated in one embodiment.

[0161] During a typical surgical procedure, a surgeon must grasp both soft tissue and hard instruments. For example, a surgeon may want to move two soft parts of a bowel together and then suture them with a small hard needle. Grasping the bowel requires a gripper capable of precision manipulation, while grasping the needle requires greater force. In order to maximize the capabilities of a single end effector, the gripper of one embodiment is designed to grasp with variable force.

[0162] Fig.11A A perspective view of a gripper is shown in one embodiment. Fig.11A Also shown is the attached distal hinge joint, seventh actuator 116 ( Figure 2A ) has a concave hinge component 139 that is identical to the concave hinge components of the other hinge joints in the device. The grasper includes two jaws 187 and 188 designed to provide sufficient length for general tissue manipulation. In one embodiment, the grasper jaws have ridged teeth 189 to allow for additional grasping adhesion. Alternatively, the grasper can be designed with any suitable surface known in the art.

[0163] The jaws are held in place between the male gripper body 185 and the female gripper body 186 and the curved clamping bodies 182 and 183. The male gripper body, the female gripper body and the curved clamping body together form the main gripper body. Alternative embodiments include a main gripper body that includes a single body, several bodies or more bodies. These bodies support the components in the gripper and are used to form the distal portion of the hinge joint including the seventh actuator 116. In order to provide the minimum possible distance between the gripper jaws and the seventh actuator, the distal end of the hinge joint is formed as a male gripper body 185 and a female gripper body 186. In an alternative embodiment, a separate hinge joint is adopted, and the gripper body alternatively includes an attachment site to fasten the proximal end of the gripper body to the distal end of the hinge joint.

[0164] In one embodiment, the male gripper body 185 is fastened to the female gripper body 186 via mating features 198 and a plurality of bolts threaded into threaded holes. In alternative embodiments, the bodies may be fastened together via means such as adhesives, or the parts may be manufactured as one piece. In addition to housing the gripper, the male and female gripper bodies are assembled to form the distal end of the hinge with the male hinge body 184. This hinge joint functions in the same manner as the other hinge joints described in the system.

[0165] Fig. 11B An exploded view of the gripper is shown. At the center of the gripper are the gripper jaws 187 and 188. These jaws are coupled to the male gripper body 185 and the female gripper body 186 via pins 199, 200, 201 and 202. These pins slide fit into holes 197 in the gripper jaws. In one embodiment, the pins do not extend to the center of the gripper jaws, thereby maintaining an open cutout slot 207 in the center of the gripper jaws.

[0166] Pin 203 is formed in the end of each gripper jaw 187 and 188 ( Fig. 11B ). These pins slide into holes 204 in link members 190 and 191 to form a hinge connection. Each link member connects the gripper jaw pin 203 to one of the pulley pins 205 formed in the body of each pulley 192 and 193. In addition, the pulley pin 210 moves in a groove formed in the male gripper body 185 and the female gripper body 186. Fig.11A The groove 206 formed in the male gripper body is clearly seen in FIG. An identical groove (not shown) is formed in the female gripper body.

[0167] The linkage mechanism formed by the above-mentioned gripper components forms a linkage so that each pulley 192 and 193 is restricted to linear motion in the distal ←→ proximal direction. The actuation cable extends from the main gripper body around each pulley to a terminal position located proximally in the main gripper body. When each actuator cable is tensioned, it pulls its corresponding pulley in the proximal direction. The pulleys for each linkage provide a 2-1 mechanical advantage via a pulley block mechanism. Alternative embodiments include alternative pulley arrangements or no pulleys at all for a 1-1 mechanical advantage. In one embodiment, the cable is clamped in a threaded hole by a set screw. In other embodiments, the cable can be terminated by any means known in the art, such as a knot, a bent clamp, or an adhesive.

[0168] In one embodiment, there are two pulleys, a distal pulley 192 and a proximal pulley 193. When the gripper jaws move, the pulleys move in opposite directions. Fig. 11C and Fig.11D As can be seen in the cross section of Fig. 12A and Fig. 12B As shown in the linkage diagram in FIG. 21, when the gripper tips separate the pulleys separate. When the gripper tips move together the pulleys move together. When the first cable is tensioned as shown by arrow 211, it pulls the proximal pulley in the proximal direction. The force is transmitted through the proximal link 214, which in turn causes the gripper jaw tip 208 to move as shown in FIG. Fig. 12A 209 as depicted in the figure. When the second cable is tensioned as shown by arrow 212, it pulls the distal pulley in the proximal direction. The force is transmitted through the distal link 213, which in turn causes the gripper jaw tip 208 to separate as shown in the figure. Fig. 12B 210. The gripper mechanism in its current embodiment has a non-linear relationship between the motion of the actuator cable and the motion of the gripper jaw tip. This non-linear motion provides the highest mechanical advantage when the jaws are closed. This has the beneficial effect of enabling the gripper to handle the high loads it encounters during operation of small tools such as needles, while also providing less force when grasping larger targets such as tissue.

[0169] In an alternative embodiment, a single link and pulley is used to move the grasper tip in two directions. One cable pulls from the proximal direction and one cable pulls from the distal direction. While the dual links of one embodiment allow both cables to pull from the proximal direction thereby simplifying cable routing, it has increased complexity and more parts. In another embodiment, a spring or other energy storage device provides actuation in one direction while a single actuation cable provides actuation in the opposite direction.

[0170] In one embodiment, jaws 187 and 188 ( Fig. 11B ) and all connected links and pulleys are inserted into the assembled male and female gripper bodies. As previously described, these parts are held in place by pins formed in the curved clamping bodies 182 and 183. The pins 199 and 200 that hold one gripper jaw 187 are rigidly attached to the curved clamping bodies. However, the pins 201 and 202 that hold the second gripper 188 are not rigidly fixed to the curved clamping bodies. The curved portions 194 and 195 couple the pins 201 and 202 to their respective curved clamping bodies.

[0171] When a force is applied to the gripper tip, it provides a force that acts to move the position of pins 199, 200, 201 and 202. While pins 199 and 200 remain rigidly fixed, pins 201 and 202 move slightly under the load as the flexure deforms elastically. The exact displacement of the flexure can be measured by a strain gauge fixed to the flexure and positioned in the strain gauge pocket 196 ( Fig.11A), another similar recess is located on the other side of the device, not visible in the figure. Standard techniques can be used to acquire the information and calculate the bending strain and gripper force.

[0172] The dimensions of the flexure are selected so that the strain gauge can measure the force applied. If the gripper force exceeds the elastic deformation range of the flexure, the flexure contacts both sides of the strain gauge pocket and is stopped.

[0173] Using the size and mechanical properties of the curved portion, the force acting at the end of the grasper jaw is continuously calculated. The force can be fed back to the surgeon via various means such as tactile feedback or via visual indicators on the console. In addition, a force control algorithm can provide closed-loop control of the grasper force. Software limits can prevent damage to tissue by applying unacceptable forces. These limits can be changed manually or automatically during surgery.

[0174] Alternative embodiments include strain gauges placed elsewhere in the system, such as on a connecting link or the gripper itself. True load cells can be designed into elements of the gripper, thereby providing improved accuracy and repeatability of measurements.

[0175] Arm length adjustment

[0176] In one embodiment, the robot arms 103 and 104 ( Figure 1A ) is constructed with an extension section 108 to increase the arm length. This increased arm length allows for an increased reach within the abdominal cavity. The extension is used to allow the arm length to be adjusted without changing the actuator design. For example, an embodiment of the device for pediatric applications may require a smaller arm for less intra-abdominal sensitivity. Such a device can be achieved by removing the arm extension. Alternative embodiments with very significant length changes require improvements in camera position and shoulder width in order to maintain human-like proportions.

[0177] Fig.13 A view of an extension segment for use in one embodiment is shown. Such an extension segment is formed from a single piece of metal having a proximal end and a distal end to allow connection to an existing proximal actuator at a proximal connection location 215 and to a distal actuator at a distal connection location 216. Additionally, a central channel 217 formed in the device allows a cable to be passed from the proximal actuator through the extension to the distal actuator.

[0178] Complex surgeries often require more than one surgeon to perform the operation. Accordingly, we designed the device for a four-arm embodiment to be used by two surgeons. Fig.14AA four-arm embodiment of the device is shown. This device has four arms and two cameras, all inserted through a single trocar. The four-arm embodiment allows two surgeons to work together. In the four-arm embodiment, two devices similar to the device of one embodiment are inserted separately into the abdominal cavity, thereby allowing two surgeons to utilize a dual robotic device.

[0179] Fig.14A A four-arm embodiment configured for two surgeons working along one side of the device is shown. In this orientation, it is as if one surgeon is standing behind the other surgeon. Utilizing the ability of each two-arm device to operate on both sides of the device as described above, the surgeon can choose the area in which the four-arm embodiment is employed. The surgeon can orient the two two-arm systems to operate on both sides of the four-arm device (wherein the two two-arm devices face the same direction). Alternatively, the surgeon can orient the two two-arm devices to face each other to operate in the area between the two devices. The surgeons can even face away from each other to operate independently on both sides of the patient.

[0180] Depending on the two devices in the abdominal cavity, an optional connecting link 218 allows the two devices to be rigidly interconnected. The link mechanism and the two robotic devices are inserted into the abdomen in sequence. For example, the surgeon can insert one robotic device all through a trocar, then insert the connecting link, and then insert the second robotic device. The connecting link mechanism can also include an optional central support 219. This support is used to rigidly fix the connecting link and thus the robotic device to the external space through a second incision in the patient's abdominal wall. This second support in addition to the support of each conduit 100 allows the rigidity of the device to be increased. A very thin central support can be passed through the abdominal wall via a catheter with a needle similar to a vascular catheter. This small catheter almost completely eliminates the damage to the patient caused by the stroke of the central support of the abdominal wall.

[0181] In one design, additional cables are passed through each conduit 100 ( Figure 7B ) and into each main hub 131. These cables continue through path 137 and extend through at least one cable terminal in the center support to each member of the connecting link 218 ( Fig.14A ). The cables are given significant slack for insertion of the device and all parts into the abdominal cavity. Once all device parts are inserted into the abdominal cavity, the surgeon can pull on the ends of the cables, thereby pulling all components of the connecting rod together. The assembly process works in a similar manner to the assembly of a camping tent in which the tent pole segments are held attached by ropes. The constant mechanical force applied to these cables can be used to firmly hold the connecting rod together during use.

[0182] Alternatively, the surgeon may remove the connecting rod or use without providing the connecting rod. Two devices described in one embodiment may be used during a single procedure where a connecting rod is not required. Some embodiments of the four-arm system are designed for insertion through a single four-arm trocar having a plurality of Fig. 14B Space for two conduit members is shown. Other embodiments are provided by two standard double-arm trocars. Some embodiments allow for software controlled interconnection, so that the surgeon can use software instructions to switch which arm he controls, or even select either or both of the arms for control within the abdomen. When not under the surgeon's control, the arms can optionally be locked in place using software, allowing a single surgeon to alternate between control of multiple devices.

[0183] In addition to the following claimed aspects of the invention, the present invention also includes the following aspects and embodiments. The system of the present invention may include a system for coupling two dual-arm robotic devices together. The system of the present invention may also include a trocar for inserting and supporting the robotic device, a trocar sleeve for supporting the device conduit, and a trocar sleeve with a camera and lighting system. Sensor structures for capturing the movements of the surgeon's arms and hands are also considered part of the present invention. The present invention is also considered to include an apparatus for virtual reality robotic surgery, wherein the robotic actuator is not permanently coupled to the camera system and apparatus for virtual reality robotic surgery, wherein the robotic actuator is not coupled to the intra-abdominal camera system. The present invention also includes a virtual reality robotic apparatus for small parts assembly, bomb diffusion, maintenance in enclosed spaces, and any other use of virtual reality robots.

[0184] Other aspects of the invention include virtual reality cameras positioned relative to robot actuation similar to the positioning of a human eye relative to a human arm and virtual reality cameras with human-like proportions of the distance between cameras and the size of the robot arm. A further aspect of the invention is computer limitation of robot motion. Also inventive is the use of an insertion body to insert the device, the insertion body including sensors to detect contact or proximity with the patient's body. With respect to cameras for virtual reality surgical systems, inventive aspects include camera motion within the camera body, cameras with aspherical lenses combined with computer image adjustment and correction, cameras with wide angle lenses, cameras with wide angle lenses combined with computer image adjustment and correction, and cameras with digital zoom or mechanical zoom capabilities. Display magnifiers or eyepiece magnifiers to allow natural human-like zoom and augmented reality of augmented reality devices set within the abdomen, viewing a virtual computer monitor within the abdomen and feedback to the surgeon via augmented reality and displaying robotic arms or related devices are also aspects of the invention. Another inventive aspect of the system is interweaving images from multiple cameras to form a single image.

[0185] Cooling of the system using saline is also part of the invention. Another aspect of the invention is to provide light and optical fiber inserted through multiple small incisions. Still another part of the inventive system is to calibrate the motion between the surgeon's arm and the robotic device by a constant scale factor, to calibrate the motion between the surgeon's arm and the robotic device by a user configurable scale factor, and to calibrate the motion between the surgeon's arm and the robotic device by a scale factor that is adjusted based on the rate of change of the position of the surgeon's arm. Further inventive aspects of the system include using a single ball bearing to provide support for axial and radial loads, an actuator having a range of motion that exceeds the desired motion to allow the device to operate on a first side and a second side, and using a Hall effect encoder to obtain position data from inside the abdominal cavity. Inventive aspects regarding the grasper include the use of pulleys to provide a mechanical advantage within the grasper, a dual linkage mechanism within the grasper to allow two actuation cables to be pulled from the proximal side of the grasper with one cable opening the grasper and a second cable closing the grasper, utilization of a curved portion of a pivot pin attached to the grasper jaws to allow for precise measurement of grasper forces by utilizing strain gauges disposed along the curved portion, and utilization of a load cell designed into the linkage of the grasper.

[0186] The techniques and systems disclosed herein may have some control components implemented as a computer program product for a computer system or computerized electronic device. Such implementation may include a series of computer instructions or logic, which are fixed on a tangible medium such as a computer readable medium (e.g., a floppy disk, CD-ROM, ROM, flash memory or other memory or fixed disk), or transmitted to a computer system or device via a modem or other interface device such as a communication adapter connected to a network through the medium.

[0187] The medium may be a tangible medium (e.g., optical or analog communication lines) or a medium implemented using wireless technology (e.g., Wi-Fi, cellular, microwave, infrared, or other transmission technology). A series of computer instructions includes at least a portion of the functionality described herein with respect to the system. Those skilled in the art will appreciate that such computer instructions can be written in a variety of programming languages ​​for a variety of computer architectures or operating systems.

[0188] Furthermore, such instructions may be stored in any tangible storage device, such as semiconductor, magnetic, optical or other storage device, and may be transmitted using any communication technology, such as optical, infrared, microwave or other transmission technology.

[0189] It is contemplated that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., compressed packaged software), preloaded with a computer system (e.g., preloaded on a system ROM or fixed disk), or distributed over a network (e.g., the Internet or World Wide Web) from a server or electronic bulletin board. Of course, some embodiments of the present invention may be implemented as a combination of both software (e.g., a computer program product) and hardware. Further other embodiments of the present invention are implemented as either entirely hardware or entirely software (e.g., a computer program product).

Claims

1. A system for generating for a user a humanoid robotic surgical visualization of the interior of a patient's body, the system comprising: (a) A robotic arm assembly configured to be inserted into a human body, the robotic arm assembly comprising: a central body, a first robotic arm attached to the central body at a first robotic shoulder, and a second robotic arm attached to the central body at a second robotic shoulder; (b) a camera assembly configured to be inserted into the human body, the camera assembly comprising: a horizontal distance between a point at which the first robot shoulder is attached to the central body and a point at which the second robot shoulder is attached to the central body; and wherein the camera assembly is attached to the central body such that a ratio of a horizontal distance value to a vertical distance value between the robot shoulder and the camera assembly is between 1 and 4, and wherein the system is disposed within the human body, wherein, when the system is disposed within the human body, the camera assembly is located above the first robot shoulder and the second robot shoulder and centered between the first robot shoulder and the second robot shoulder, the ratio being equivalent to the ratio of the horizontal distance of the user's shoulder to the vertical distance between the user's shoulder and the user's eyes, and wherein the camera assembly provides a human perspective to a user within the human body; and (c) a processor configured to generate for the user a representation of at least a portion of the robotic arm assembly within the human body using data from the camera assembly.

2. The system of claim 1, wherein the user is a surgeon performing remote surgery on the patient.

3. The system of claim 1, wherein the first robotic arm and the second robotic arm each have the same range of motion as a human arm.

4. The system of claim 3, wherein the first robotic arm and the second robotic arm each include joints equivalent to a human shoulder, a human elbow, and a human wrist.

5. The system of claim 1, wherein the robotic arm assembly and the camera assembly are insertable into the human body through a surgical trocar.

6. The system of claim 1, wherein the robotic arm assembly and the camera assembly are insertable into the human body through a single incision.

7. The system of claim 1, wherein the camera assembly includes a pan or tilt system configured to pan or tilt the camera assembly.

8. The system of claim 7, wherein the camera assembly is panned, tilted, or both in response to a sensed position of the camera assembly relative to a reference point inside the human body.

9. The system of claim 1, wherein the robotic arm assembly is coupled to the camera assembly within the human body.

10. The system of claim 1, wherein the ratio of the horizontal distance value to the vertical distance value is equal to 2.

11. The system of claim 1 , wherein the robotic arm assembly comprises a cautery tool.

12. The system of claim 11, wherein the cautery tool is configured to cauterize tissue inside the human body of the patient.

13. The system of claim 1, wherein the representation comprises a virtual reality representation.

14. The system of claim 1, wherein the representation comprises an augmented reality representation.

15. The system of claim 1, wherein movement of the user's head is tracked and the position of the camera assembly is adjusted in response to the movement.

16. The system of claim 15, wherein the camera assembly includes a pan and tilt system, and wherein adjusting the position of the camera assembly in response to the movement includes panning or tilting the camera assembly.

17. The system of claim 16, wherein during panning or tilting of the camera assembly, the field of view of the camera assembly is adjusted to compensate for a delay between sensing the motion and panning or tilting the camera assembly.

18. The system of claim 1, wherein the camera assembly comprises a first camera assembly and a second camera assembly, and the data is used to generate a stereoscopic image.

19. The system of claim 1, wherein the portion of the human body of the patient is the abdomen of the patient.

20. The system of claim 1, wherein the robotic arm assembly moves in a plane parallel to a viewing plane of the camera assembly.

21. The system of claim 1, wherein a ratio of the horizontal distance value to the vertical distance value is 4.

Images