Robotic Anatomical Manipulation System and Method

Abstract

The present invention provides, in various embodiments, a system and method for robotic manipulation of a patient's anatomical tissue, such as the uterus, during surgery.

Description

Technology Field

[0001] Related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 829,311 filed on April 4, 2019, the entirety of which is incorporated herein by reference.

[0003] Computer Program List Appendix

[0004] A list of computer programs containing a single ASCII text file named "Kinematics Code.txt" has been submitted electronically, the entirety of which is incorporated herein by reference. Background Technology

[0005] In gynecological surgeries such as hysterectomy, the patient's uterus must be precisely positioned during surgery according to the specific requirements of the case. Improved uterine manipulation systems and methods are required in the industry. The problem to be solved

[0006] The present disclosure generally relates to robotic anatomical manipulation systems and methods, and more specifically, to robotic uterine manipulation systems and methods for use in minimally invasive hysterectomy. means of solving the problem

[0007] In various embodiments, the present invention provides a robotic anatomical manipulation system for use with an end effector configured to position and hold a patient's anatomical tissue during minimally invasive surgery, the system comprising: a robot comprising a cart and an arm, wherein the proximal end of the arm is connected to the cart; a console comprising a joystick and a user interface, wherein the joystick is configured to provide real-time motion input to the arm; a mechanical interface connected to the distal end of the arm and configured to hold and release a handle of the proximal end of the end effector; a sensor disposed between the distal end of the arm and the mechanical interface, configured to measure at least one of a force and a torque generated by the end effector; and a non-transient computer-readable medium storing computer-executable code for a control system that controls the arm in response to user input through the joystick and the user interface. The end effector has a substantially curved body, a distal end tip, and an adjustable fulcrum configured to be positioned at a distance to the tip unique to each patient in each surgery. The control system includes a kinematic unit configured to determine the motion of the arm required to generate user-specific motion of the tip based on the curvature of the end effector and the patient-specific fulcrum position entered into the user interface. The control system also includes a force-torque reading unit configured to monitor a sensor and, when a predetermined force threshold or torque threshold is reached in a given direction, to transmit a stop signal to the arm to prevent motion in that direction. In some embodiments, the anatomical tissue is the uterus, and the minimally invasive surgery is a robot-assisted or manual laparoscopic hysterectomy.

[0008] The cart may be mobile or stationary. The console communicates with the remote machine via a wired or wireless connection. In some embodiments, the console further includes a video feed configured to display a live signal from a camera used to monitor surgery.

[0009] In some embodiments, the arm comprises six joints: a base (A), a shoulder (B), an elbow (C), and wrists (D, E, F). In some embodiments, the elbow is positioned below the wrist at the baseline position. In some embodiments, the arm comprises at least one joint having a marking, the marking indicates a safe range for the baseline position.

[0010] In some embodiments, the mechanical interface includes a quick release mechanism configured to release the end effector within 5 seconds without moving the robot arm or cart.

[0011] In some embodiments, the sensor has six channels of force and torque: F x , F y , F z , T x , T y , and T z It is a 3-axis force and torque sensor configured to acquire.

[0012] In some embodiments, the force threshold is a warning force threshold and the torque threshold is a warning torque threshold. In some embodiments, the warning force threshold is 10 lbf and the warning torque threshold is 70 inch-lbs.

[0013] In some embodiments, a non-transient computer-readable medium further stores computer-executable code for an embedded safety system that calculates the force and torque generated by the end effector, and the embedded safety system is configured to generate a digital disable command to disable control of all arms until the system is manually re-enabled when a predetermined safety force threshold or safety torque threshold is exceeded.

[0014] In some embodiments, the embedded safety system includes an analog-to-digital converter (ADC) unit configured to acquire force and torque data from a sensor at least 100 times per second.

[0015] In some embodiments, the embedded safety system further includes a force unit configured to receive a raw voltage communicated to an ADC unit by a sensor and to convert the raw voltage into force and torque values.

[0016] In some embodiments, the safety force threshold is 12 lbf and the safety torque threshold is 90 inch-lbs.

[0017] In some embodiments, the console includes a mechanism for enabling / disabling control of the arm from the joystick and user interface, and the embedded safety system is configured to capture the enable / disable input and generate a command to the robot in response.

[0018] In some embodiments, the user interface includes a stop axis display configured to provide a visible indication of an axis where a predetermined safety force threshold or safety torque threshold is exceeded.

[0019] In some embodiments, the cart includes a setup positioning system on its upper surface, the proximal end of the arm is connected to the setup positioning system, and the setup positioning system is configured to provide forward and reverse translational motion in the X, Y, and Z directions.

[0020] In some embodiments, the console includes an alarm that provides an audible indication of at least one of the movement and speed of the distal end of the arm.

[0021] Additional features and advantages of the present invention are further described below. These summaries are intended merely to describe specific features of the present invention and are not intended to limit the scope of the present invention in any way. The fact that specific features or embodiments of the present invention are not described, or that one or more features are not included in these summaries, should not be construed as limiting the claimed present invention. Brief explanation of the drawing

[0022] The following detailed description of various embodiments of the present invention, in addition to the gist of the above, will be better understood with reference to the accompanying drawings. Specific embodiments are illustrated in the drawings for the purpose of explaining the system and method of the present invention. However, it should be understood that the present invention is not limited to the precision devices and mechanisms illustrated. FIG. 1a is a perspective view of a robotic uterus manipulation system according to various embodiments of the present invention. FIG. 1b is a side view of the uterine manipulation system of FIG. 1a at a baseline position according to various embodiments of the present invention. FIG. 2 is a side view of a robot arm with dimensions of mm. FIG. 3 is a perspective view of the arm of FIG. 2, showing six joints. Figure 4 is a side view of the uterine manipulation section. FIGS. 5a to 5h are, respectively, a front perspective view, a rear perspective view, a front view, a rear view, a left side view, a right side view, a top view, and a bottom view of a mechanical interface according to various embodiments of the present invention. Figure 6 is a perspective view of the sensor. Figure 7 is a side view of the sensor of Figure 6. FIG. 8 is a perspective view of the uterine manipulation system of FIG. 1a, in which the mechanical interface is in an open position and the uterine manipulator is detached from it. FIG. 9 is a side view of the uterine manipulator of FIG. 4, illustrating reference points (tip, fulcrum, handle) for kinematics. FIG. 10 is a side view of the uterine manipulator of FIG. 9 at the baseline position, showing the pitch. FIG. 11 is a top view of the uterine manipulator of FIG. 9 at a baseline position illustrating yaw. FIG. 12 is a side view of the uterine manipulator of FIG. 9, illustrating translational movement from a baseline position to a translated position. FIG. 13 is a drawing of a software system architecture according to various embodiments of the present invention. Figure 14 is a table of individual software units for each software item. Figure 15 is a drawing of a top-level item finite state machine. FIG. 16 is a drawing of a graphic user interface according to various embodiments of the present invention. FIGS. 17a to 17d are a schematic side view, front view, top view, and perspective view of a robotic uterine manipulation system according to various embodiments of the present invention, illustrating exemplary degrees of freedom for an arm. FIGS. 18a to 18c are front and side views of a robotic uterine manipulation system as illustrated in FIGS. 17a to 17d, illustrating an exemplary cart having a setup positioning system. FIGS. 19a and FIGS. 19b are perspective views illustrating exemplary visible indications of an acceptable range for a starting position according to various embodiments of the present invention. FIGS. 20a and FIGS. 20b are a perspective view and a top view, respectively, of an exemplary user control unit on a console of a robotic uterus manipulation system according to various embodiments of the present invention. FIGS. 21a to 21c are a perspective view, a top view, and a side view, respectively, of an exemplary user control unit on a console of a robotic uterus manipulation system according to various embodiments of the present invention. FIG. 22 is a top view illustrating a movable user control unit on a console of a robotic uterus manipulation system according to various embodiments of the present invention. FIG. 23 is a schematic perspective view of a console of a robotic uterus manipulation system according to various embodiments of the present invention. FIG. 24 is a schematic side view illustrating the transport of a robotic uterine manipulation system according to various embodiments of the present invention. FIG. 25 is a schematic side view of an exemplary wired connection between a console and a robot of a robotic uterine manipulation system according to various embodiments of the present invention. FIG. 26 is a schematic top view of a robotic uterine manipulation system according to various embodiments of the present invention illustrating symmetrical insertion into a patient. Specific details for implementing the invention

[0023] In minimally invasive (robot-assisted or manual laparoscopic) gynecological procedures such as hysterectomy, the patient's uterus needs to be positioned precisely during surgery according to the specific requirements of the case. Uterine manipulation is a critical component of a successful minimally invasive hysterectomy, and anatomical tissues are manipulated to tilt and / or displace the uterus for safe incision of, for example, the ureters, the suspensory ligaments of the uterus, and the uterine blood supply.

[0024] The current standard of practice is for a surgical resident or fellow to manually manipulate the uterus using one or more handheld tools, receiving positional instructions from the surgeon. This practice can present significant physical challenges because, during the operation, the resident or fellow must maneuver themselves around other equipment used in the surgery, such as the large footprint of a surgical robot. This practice can be a difficult challenge because the individual manipulating the uterus may be positioned at an uncomfortable angle or on the side of the perineum. It can also be physically demanding.

[0025] The present invention provides a robotic anatomical manipulation system and method for surgery that overcomes such problems and, in various embodiments, can provide more accurate and precise manipulation and positioning of target anatomical tissue compared to current practice standards. When using the robotic anatomical manipulation system and method according to an embodiment of the present invention, the surgeon can control the position of the anatomical tissue from the control console away from the operating table and away from any surgical robot equipment / separate surgical robot control console (if any). The surgeon can control the manipulation instrument and the patient's anatomical tissue more accurately and precisely to the desired position, and can achieve superior surgical results.

[0026] System hardware

[0027] Embodiments of the present invention provide a robotic anatomical manipulation system comprising a robot and a console. The robot may include, for example, a cart, an arm (e.g., having 6 degrees of freedom), and arm support hardware including a computer and a power supply. The cart is a movable or stationary unit on which the arm is positioned. The arm holds and places a separate end effector. A mechanical interface connects the end effector to the arm. The console may include, for example, a joystick (or other input device), a user interface, an enable / disable button, and a video feed, which can be integrated into a single unit used for interfacing with the arm. The joystick is used to provide real-time motion input to the arm. As used herein, the term “joystick” refers to a control column / stick pivoting on a base (optionally including one or more buttons, switches, triggers, scroll wheels, etc.) or any other input / control device currently known or to be developed, configured to provide substantially similar control functions. The user interface provides the user with feedback regarding the status of the arm and provides a means to input configuration parameters. The video feed displays live signals from the camera used to monitor the surgery.

[0028] The robot is configured to be positioned near the operating table (e.g., on the foot of a table for gynecological surgery). In some embodiments, the console may be positioned near the robot within the operating room and may be connected by a wired or wireless connection. In other embodiments, the console may be positioned away from the robot (e.g., in another room or at a different geographical location) and may be connected by a wireless connection.

[0029] FIG. 1a is a perspective view of a robotic uterine manipulation system (100) according to an exemplary embodiment of the present invention. A robotic arm (200), which may be covered wholly or partially by a drape (300), holds and positions a uterine manipulator (400), which is a patient interface end effector. A custom mechanical interface (500) connects the uterine manipulator (400) to the arm (200) and supports the kinematic and other specifications of the uterine manipulator (400) in use. A sensor (not shown; see FIG. 8) may be positioned between the mechanical interface (500) and the arm (200). FIG. 1b is a side view of the uterine manipulation system of FIG. 1a in a baseline arm position, and according to some embodiments, the uterine manipulator (400) is held in a vertical plane with the manipulator handle substantially parallel to the floor and the manipulator tip facing upward. In some embodiments (e.g., for gynecological procedures), the elbow of the arm (200) may be positioned below the wrist portion as shown in FIG. 1a (to utilize the space under the operating table). In some embodiments, visible markings (202) (e.g., range bar and alignment arrows; see FIG. 19a and FIG. 19b) may be provided on one or more joints of the arm (200) to indicate a safe range for the user’s own angle and / or starting position for surgery or range of surgical action. A safe range may be obtained, for example, when the arrows on each joint are within a band on the other side of the joint. Thus, the surgeon or the physician’s assistant may manually position the robotic arm in a safe / acceptable starting position by using the markings (202) as a visible guide (e.g., when a console is not nearby).

[0030] In some embodiments, the arm (200) comprises a cooperative industrial robot arm, such as Universal Robots' UR5. The UR5 has six swivel joints, each with a working range of ± 360°, supports a payload of 5 kg (11 lbs), and has a reach of 850 mm (33.5 inches). For example, refer to universal-robots.com / media / 1801303 / eng_199901_ur5_tech_spec_web_a4.pdf for technical specifications. A side view of the UR5 is shown in FIG. 2, with its dimensions indicated in mm (a drawing obtained from universal-robots.com / download / under Mechanical documentation, UR5 documentation, Robot working area, Robot dimensions area PDF). A drawing of the UR5 is shown in FIG. 3, illustrating the robot's joints: A (base), B (shoulder), C (elbow), and D, E, F (wrist parts (1, 2, 3)). For example, refer to the Universal Robots User Manual, UR5 / CB3, Euromap67, Version 3.5.3, Original instructions (en), US Version. Robot arms such as the UR5 are used in various industries for tasks such as assembly and packaging. However, prior to the present invention, they were not configured for the manipulation of human anatomical tissues in surgery.

[0031] In some embodiments, the uterine manipulator (400) is ConMed TMThe VCare® Vaginal-Cervical Ahluwalia's Retractor-Elevator from Corporation may be used, but other end effectors may also be used. The VCare® is a disposable, single-use device for the manipulation and elevation of the uterus and cervix in surgical and diagnostic procedures. The OD is 5 mm (0.2 inches) and the length is 42 cm (16.4 inches), including the ergonomic handle (406). A side view of the VCare® is shown in FIG. 4 (a drawing obtained from Instructions for Use for Product Catalog No. 60-6085-200A at www.conmed.com / en / customer-service / instructions-for-use / eifu-fmder). In the VCare® device (400), an insulated manipulator tube (407) is provided, which includes a visible gradient mark (not shown) to help obtain an appropriate insertion depth and has a 10-cc inflatable balloon (401) at the patient end. A syringe (not shown) is attached to an inflation valve (409) on a pilot balloon (408) to inflate the balloon (401) with air. The manipulator tube (407) has a natural curved shaft that corresponds to the patient's anatomical tissue (e.g., the angle of the sacral curve). Gradient marks provide guidance for comparison with a gradient uterine sound (and, as described more specifically below, also indicate patient-specific distance / fulcrum locations). The balloon (401) fixes the uterus and stabilizes the shaft within the uterine cavity. The cervical (distal) cup (402) is ordered in various sizes (e.g., S, M, L, XL) depending on the size of the patient's cervix. The cervical cup (402) provides guidance for the vaginal incision and includes a hole for suturing the cup in place for improved stability of the uterus.The vaginal (proximal) cup (403) can be advanced by means of a sliding locking assembly (404) having a handle screw (405) and can be locked in place at the posterior margin of the cervical cup (402), thereby securing the cervical cup (402) and maintaining the pneumoperitoneum.

[0032] In other embodiments, different robot / arms (200) and / or different uterine manipulators (400) (or other types of end effectors configured to manipulate different anatomical tissues), which are commercially available or custom-designed, may be used. Different end effectors (400) may be selected by the surgeon based on the needs of the patient and each surgery. The mechanical interface (500) may be configured accordingly.

[0033] In the aforementioned exemplary embodiment using a UR5 robot arm (200) and a VCare® end effector (400), the mechanical interface (500) may be configured, for example, as shown in FIGS. 5a through 5h illustrating a closed / locking position. FIG. 5a is a front perspective view of an exemplary mechanical interface (500) illustrating the front and left side. FIG. 5b is a rear perspective view of the mechanical interface of FIG. 5a illustrating the rear and right side. FIGS. 5c through 5h illustrate the front, rear, left side, right side, top, and bottom views of the mechanical interface of FIG. 5a, respectively. The top surface of the mechanical interface (500) includes an extension / connection portion (510) configured to be bolted to the robot arm (200) (or, as described below, a sensor (600) attached to the robot arm (200)). The lower portion of the mechanical interface (500) includes an opening (520), through which the handle (406) of the uterine manipulator (400) is inserted into the mechanical interface (500). The left and / or right side of the mechanical interface (500) may have a slot (530) (e.g., an opening or window) inside, so that the user can see how deeply the handle (406) is inserted. A quick release mechanism is preferably provided, thereby allowing the front portion (540) of the mechanical interface (500) to be quickly detached from the rest of the mechanical interface (500) by pushing against, for example, a protrusion (542) (for example, in the direction of the arrow indicated on the front portion (540)) and sliding the front portion (540) until a retaining tab (not shown) is aligned with a recess (544), so that the front portion (540) can be released and detached from the handle of the actuator.The rear surface of the mechanical interface (500) includes a latching mechanism (550) for holding the handle (406) of the uterine manipulator (400) in a releaseable position when the handle (406) is inserted into the mechanical interface (500) (see FIG. 1a and FIG. 1b). Pushing the latch (550) at position (552) presses the latch into a closed / locking position and pushes the screw (554) through an opening (556) within the rear surface of the mechanical interface (500), thereby allowing the screw (554) to be in contact / engaged with the manipulator handle and hold it in place. A stop (555) (see FIG. 8) made of rubber or other elastomer material is provided on the distal end of the screw (554) to help secure the handle (406). Pushing the latch (550) at position (558) can release / unlock the latch and allow the screw (554) to be retracted from the opening (556) (see FIG. 8 illustrating the open / unlock position).

[0034] In some embodiments, a force / torque (F / T) sensor (600), such as the Mini40 from ATI Industrial Automation, Inc., may be provided at the distal end of the arm (200). The Mini40 is an EDM wire cut from stainless steel with a high yield strength and has high strength. It is also a silicon strain gauge that provides a signal 75 times stronger than a conventional foil gauge and has a high signal-to-noise ratio. For example, see www.ati-ia.com / Products / ft / ft_models.aspx?id=Mini40. FIG. 6 is a perspective view of the Mini40-E transducer showing the tool side. FIG. 7 is a drawing showing a specific side view of the Mini40-E transducer with the mounting side shown on the left and the tool side shown on the right (drawing obtained from www.ati-ia.com / app_content / Documents / 9230-05-1314.auto.pdf). In Mini40, the mounting adapter and tool adapter may be made of aluminum or stainless steel, and the transducer may be hardened stainless steel. In other embodiments, different types of F / T sensors (600), or individual force and / or torque sensors may be used.

[0035] FIG. 8 is a perspective view of the uterine manipulation system of FIG. 1a with the mechanical interface (500) unlatched and the VCare® uterine manipulator (400) detached therefrom. As shown in FIG. 8, a sensor (600), such as Mini40, may be attached to the distal end of the UR5 arm (200), and in this case, the mechanical interface (500) may be bolted to the sensor (600) instead of being bolted directly to the arm (200). A rubber stop (555) may be attached to the end of a screw (554) on the mechanical interface latch (550). As shown in FIG. 8, a disposable sterile drape (300) (e.g., a clear plastic drape) may be provided to cover at least a portion of the arm (200), the sensor (600), and the mechanical interface (500) (to maintain a sterile field between the patient and the robot). The handle (406) of the uterine manipulator (400) can be inserted into an opening (520) on the mechanical interface (500), and the mechanical interface latch (550) can be closed (for example, so that a rubber stop (555) fits into a groove on the handle (406).

[0036] FIGS. 9 through 12 illustrate the movement of a uterine manipulator (400) according to some embodiments of the present invention. FIG. 9 is a side view of the uterine manipulator of FIG. 4 illustrating reference points (handle, fulcrum, tip) for kinematics, repeated in FIGS. 10 through 12. The distance from the fulcrum to the tip varies depending on the individual patient. This patient-specific distance is measured using numbered marks on the VCare®. A number corresponding to the locking position can be entered into the kinematic software via a control console located in the robot setup after insertion of the VCare® (see, for example, "fulcrum position (cm)" in FIG. 15). FIG. 10 is a side view of the uterine manipulator (400) in the baseline position illustrating the pitch. In the baseline position, the uterine manipulator is maintained in a vertical plane with the handle parallel to the floor and the tip facing upward. The baseline / start position (0, 0, 0) can be hardcoded as a point in 3-D space. FIG. 11 is a top view of a uterine manipulator (400) at a baseline position illustrating the urine. FIG. 12 is a side view of a uterine manipulator (400) illustrating a representative translational movement from the baseline position to a translated position. Joystick movement on the console determines the motion of the tip.

[0037] Arm motion

[0038] FIGS. 17a through 17d are schematic side, front, top, and perspective views of a robotic uterine manipulation system (100) according to various embodiments of the present invention, each comprising a movable cart (700), an arm (200), and a mechanical interface (500) (attached / not illustrated end effector (400)). As illustrated, the arm (200) is configured to provide + / - translational motion along the X, Y, and Z directions as well as clockwise and counterclockwise rotation.

[0039] In some embodiments, the arm can manually position the uterine manipulator with a starting pitch of +0 / -30 degrees from the baseline position. The arm can also be controlled by a programmable controller. In some embodiments, the arm can position the uterine manipulator tip with a pitch of at least +50 degrees and -40 degrees from the previously defined starting position. The uterine manipulator tip pitch is measured from the fulcrum to the tip. In some embodiments, the arm can position the uterine manipulator tip with a pitch of at least +45 degrees and -45 degrees from the previously defined starting position. In some embodiments, the arm can translate the uterine manipulator tip 8 cm forward from the previously defined starting position. The translation follows a line formed between the fulcrum and the tip. In some embodiments, the arm can position the uterine manipulator at the extremes of the pitch, the translation, and the translation simultaneously. In some embodiments, the arm can apply a force of 5 pounds in the direction of the aforementioned forward translation. In some embodiments, the uterine manipulator handle may move 1 cm or less when a force of 5 pounds is applied to the uterine manipulator tip in any primary direction. This describes arm motion and handle / receiving interface tilt.

[0040] cart

[0041] FIG. 18a is a front view of a robotic uterine manipulation system (100) as illustrated in FIG. 17a through 17d according to various embodiments of the present invention, and FIG. 18b and FIG. 18c are side views. As illustrated in FIG. 18a through 18c, in some embodiments, the cart (700) may include a setup positioning system (710) (e.g., including a substantially horizontal movable platform / base), and the arm (200) may be releaseably or permanently fixed to the setup positioning system. The positioning system (710) may be configured to provide + / - translation in the X, Y, and / or Z directions, for example. The setup positioning system (710) may be used for initial positioning prior to the precise placement of the arm (200) (and mechanical interface (500), and end effector (400) attached thereto) for anatomical manipulation.

[0042] In some embodiments, the cart can position the uterine manipulator in a baseline position within a height range of about 39 inches to about 48 inches (e.g., corresponding to the 5th percentile and 95th percentile elbow heights). When the brake is applied and a force of up to 22 pounds is applied to the upper edge of the cart, the cart can be held in a fixed position. When a force of 22 pounds is applied to the arm in the baseline arm position and the wheels are locked, all wheels of the cart can be held in full contact with the floor. While immobilization of 22 pounds of force was considered sufficient stability for safe use in uterine manipulation cases; the cart may be configured to withstand different force limits (greater or smaller) in other applications. The robot can position the uterine manipulator in a baseline position with a surgical table tilt of 0 to -30 degrees in a reverse Trendelenburg position. The robot may be powered by a general-purpose 100 to 240 VAC 50 to 60 Hz. The robot may monitor the force and torque of the arm for independent safety override functionality. Safety monitoring and intervention may be independent of the robot positioning system. The robot may include an emergency stop capability.

[0043] mechanical interface

[0044] FIG. 19a is a detailed view of an exemplary marking (202) located at the distal end of an arm (200) connected to a mechanical interface (500). The marking (202) can generally function to provide a visible indication of an acceptable setup range for a joint position. For example, a range indicator strip (204) may be provided on one side of a given joint. A central portion (205) of the range indicator strip (204) may indicate an acceptable setup range. An optional end portion (206) of the range indicator strip (204) may indicate an outer limit of the acceptable setup range. A range position arrow (207) may be provided on the other side of the joint to indicate whether the position is within the acceptable range. When the arrow (207) is within the central portion (205) of the range indicator strip (204), the user can identify an acceptable starting position. As schematically illustrated in FIG. 19b, in some embodiments, visible markings / markings (202) may be provided on all joints. In other embodiments, visible markings / markings (202) may be provided on only a subset of joints.

[0045] In some embodiments, the arm may maintain a connection with the uterine manipulator through a mechanical interface when, for example, about 5 pounds of force is applied to the tip of the uterine manipulator in any primary direction. In some embodiments, the mechanical interface may include a “quick-release” mechanism that allows the uterine manipulator to be released, for example, within 5 seconds without movement of the robot arm or cart.

[0046] console

[0047] FIGS. 20a and FIGS. 20b are a perspective view and a top view, respectively, of an exemplary user control unit on a console (800) which is a component of a robotic uterine manipulation system (100) according to various embodiments of the present invention. While a joystick (802) may provide translational motion in the X and Y directions and clockwise / counterclockwise rotation, a switch or roller (804) may provide translational motion in the Z direction. In other embodiments, other user control units may be provided to control the robotic degrees of freedom. For example, a joystick (806) may be provided that provides translational motion in the X and Y directions as shown in FIG. 21a, rotation in both directions as shown in FIG. 21b, and translational motion in the Z direction as shown in FIG. 21c. In some embodiments, the user control units may be movable, and accordingly, they may have a left or right orientation on the control panel. For example, as illustrated in FIG. 22, the user control unit as illustrated in FIG. 20b can be rotated from a left orientation to a right orientation and vice versa around a pivot point (808). FIG. 23 is a schematic diagram of a control console (800) according to some embodiments. Preferably, the console provides one or more control surfaces / mechanisms, video capabilities, and / or a user interface having a touch screen or other capabilities. The cart (700) and the console (800) are preferably configured to be easily transported (e.g., by one person to another area of ​​the hospital). FIG. 24 is a schematic side view of an exemplary system transport / entire movement solution for a robotic uterine manipulation system (100) according to various embodiments of the present invention.

[0048] In some embodiments, the robot may receive control input from a user via a wired remote console. The wired connection may be at least 15 feet long, for example (to provide sufficient space for a surgeon and / or surgical robot), but other lengths may also be used. FIG. 25 is a schematic side view of an exemplary cable connection / cable management solution for a robotic uterine manipulation system (100) according to various embodiments of the present invention. Preferably, the cable may be connected / disconnected (e.g. for transport / shipping, storage, service, etc.) and efficiently stored when not in use. The console may be a position control unit for the arm. The console may provide a user interface to the user to input configuration parameters (e.g., sounding depth, boundary, speed). The console may include a mechanism (e.g., a button) capable of disabling movement control of the arm from the joystick and user interface when operated. This is intended to ensure safety during critical surgical times, such as cauterization. This ensures that user and system errors cannot cause serious incorrect arm movements. The console may include a user interface capable of providing visible feedback (e.g., warning messages and / or lighting). The console may also include a user interface capable of providing audible feedback (e.g., beeps or other warning sounds). The audible feedback may indicate motion and may function as, for example, a movement alarm and / or speed alarm. The volume of the alarm may be programmed, and pitch / tone / pattern may provide an audible indication, for example, of both movement and speed. User control and user experience may be preferably customized. For example, force levels, the physical placement of the control unit, and / or alarm levels may be customized according to the preferences of the surgeon (or other user).In some embodiments, a back-off mode may be provided, thereby allowing the user to manually return the system to a safe move or lock position when a force or geometric limit is reached. In some embodiments, a stop axis indicator may be provided, thereby allowing the system to visually indicate the locked / maximum axis / control unit. Such visualization may include text and / or diagrams, preferably both.

[0049] Preferably, the robotic uterine manipulation system (100) according to an embodiment of the present invention is configured for symmetrical insertion and can provide the same range of motion in both directions (+ / -) along all axes. For example, as illustrated in FIG. 26, which illustrates a schematic robot (cart (700) and arm (200)) and a patient (900), in some embodiments, the distances (D) may be substantially the same within all ranges of motion.

[0050] Sterile

[0051] In some embodiments, the arm can satisfy the transmitted motion requirements while covered by a sterile drape. The drape can provide a sterile boundary between the arm and the uterine manipulator, thereby ensuring that the sterility of the uterine manipulator is maintained during docking. As used herein, docking refers to the act of mating the uterine manipulator into the mechanical interface of the robot arm. The console can also perform its intended function while covered by a sterile drape. The robot can be cleaned with a standard hospital cleaning agent. The console can be cleaned with a standard hospital cleaning agent.

[0052] Standard compliance

[0053] In some embodiments, the system may include Instructions for Use (IFU). All labeling complies with the requirements described in 21 CFR 801. System leveling complies with the U.S. Food and Drug Administration's Unique Device Identification requirements. The system preferably satisfies the requirements of IEC 60601-1:2005 (Medical Electrical Equipment - Part 1: General Requirements for Basic Safety and Essential Performance).

[0054] System software

[0055] Embodiments of the present invention also provide software for the operation of a robotic anatomical manipulation system. The software may include, for example, conventional robot arm software (e.g., UR5 Software), conventional robot arm software and custom system software (e.g., UR5 Software + Lab VIEW System), and embedded software (e.g., Embedded Safety System), as further described below.

[0056] Placement and Motion

[0057] The software can place the arm in Freedrive Mode, in which the user can manually move each joint of the arm to a desired position. The software can place the arm in Manipulation Mode, in which the user can control the placement of the arm via a joystick. In Freedrive Mode (UR5 Software), the software can maintain the new position of the arm whenever the user manually moves the arm. During the transition from Freedrive Mode to Manipulation Mode, the software can maintain the arm in its manually set position. In Manipulation Mode (UR5 Software + Lab VIEW System), the robot arm does not respond to external forces that do not exceed the maximum load of the robot arm (i.e., the robot arm can maintain its position unless commanded by user input via the joystick or GUI). When the user removes input (via the joystick or GUI), the software can stop the arm motion and maintain the arm's position. The software can record the position of the arm, and while in operation mode, the user can have the arm automatically return to that position with a small input, such as a single input. One such position is the Home Position. The Home Position refers to the initial position of the arm after free-movement to engage with the uterine manipulator (this position will generally have a yaw of 0). When a force and / or torque reading in a specific direction of movement during motion in operation mode exceeds a predetermined Lab VIEW force or torque threshold, the software can prevent continued motion of the arm along that direction.In some embodiments, for use in uterine manipulation, it was determined that a preferred Lab VIEW force threshold was 10 lbf (pound-force) or about 10 lbf and a preferred Lab VIEW torque threshold was 70 inch-lbs (inch-pound-force) or about 70 inch-lbs. In other embodiments, different Lab VIEW torque and / or force thresholds may be set. When an upward pitch input is specified by the user, the software moves the arm to perform an upward pitch. When a downward pitch input is specified by the user, the software moves the arm to perform a downward pitch. When a left yaw input is specified by the user, the software moves the arm to perform a left yaw. When a right yaw input is specified by the user, the software moves the arm to perform a right yaw. When a forward translation input is specified by the user, the software moves the arm to perform a forward translation. When backward translation input is specified by the user, the software moves the arm to perform backward translation.

[0058] safety

[0059] Embedded software (Embedded Safety System) is: F x , F y , F z , T x , T y , and T zSix corresponding force and torque channels can be acquired. The embedded software can acquire force and torque values ​​at least 100 times per second. The embedded software controller can monitor the force and generate a digital signal to disable all or a subset of arm motion when the force reading satisfies or exceeds a predetermined Embedded System Force Threshold. The embedded software controller can also monitor the torque and generate a digital signal to disable all or a subset of arm motion when the torque reading satisfies or exceeds a predetermined Embedded System Torque Threshold. Torque calculations assume that the force is applied at the furthest tip of the uterine manipulator. In some embodiments, for use in uterine manipulation, it was determined that a preferred embedded system force threshold was 12 lbf or about 12 lbf and a preferred embedded system torque threshold was 90 inch-lbs or about 90 inch-lbs. In other embodiments, different embedded system force and / or torque thresholds may be set. The embedded software controller enables the user to disable motion control by activating the 'Safeguard Stop' function of the robot arm. Disabling motion control by the embedded software controller can prevent / cancel the arm's free drive capability and may be controlled, for example, via an enable / disable button on a console cart.

[0060] User interface

[0061] The software (Lab VIEW System) can generate a graphical user interface (GUI) to provide touchscreen control and display feedback (e.g., as illustrated in FIG. 15). The GUI can provide the user with the ability to toggle between free drive mode and operation mode. The GUI can provide feedback indicating the status of the robot (e.g., operation mode or free drive mode). The GUI can provide the user with feedback indicating whether the starting position (home position) is valid or invalid. A valid home position means that the arm can achieve a full desired range of motion from that position. The GUI can display measured real-time force and / or torque values ​​to the user and, accordingly, notify the operator when the force or torque threshold is approached. This can be displayed on the user interface, for example, as a resulting force value and / or a resulting torque value. The GUI can provide visual feedback when the Lab VIEW force or torque threshold is approached or reached. The GUI can provide audible feedback when the Lab VIEW force or torque threshold is approached or reached. The GUI can allow the user to set the fulcrum position before entering the operation mode.

[0062] Software System Architecture

[0063] outline

[0064] An embodiment of the present invention provides the following software components: a UR5 robot and control system; a Lab VIEW control system; and an embedded safety system.

[0065] The UR5 robot and control system is a ready-made closed-loop robot comprising a motorized arm, a single-board computer control system, a user interface touch panel, and a supervisory safety processor. The UR5 robot and control system interfaces with external applications through TCP / IP interfaces and digital I / O lines. Software running on the UR5 single-board computer, safety processor, and touch panel is considered a "SOUP" unit (software of unknown origin) that interfaces with the rest of the software architecture through TCP / IP and digital I / O interfaces.

[0066] The Lab VIEW control system is a custom Lab VIEW development environment that runs on a Windows PC. In several embodiments, the Lab VIEW application provides one or more of the following functions: the ability to capture operator motion control from a joystick / mouse; the ability to provide a touchscreen user interface for system control; the ability to enable and disable free-drive mode for the user; the ability to calculate kinematics for robot motion and send commands to the robot; the ability to monitor force and / or torque feedback from an end effector; and the ability to send a stop signal to the robot when a force or torque threshold is reached or exceeded.

[0067] The embedded safety system (embedded system) is custom firmware running on the Atmel® XMEGA® C3 platform and includes a 16-bit mega microcontroller (e.g., ATxmega384C3) to receive force and torque feedback from the end effector and to generate a digital disable command for the robot when the force or torque applied to the patient reaches or exceeds a safety threshold. The embedded system also captures enable / disable input from the user and generates commands for the robot. The embedded system utilizes a main loop unit designed to provide round-robin and scheduling among software units. The ADC unit controls the analog-to-digital conversion hardware interface unit. The force calculation unit converts all voltages acquired by the ADC unit into force and torque values. The robot relay control unit generates a disable signal for the robot when the force or torque applied to the patient reaches or exceeds a predetermined threshold value.

[0068] In some embodiments, the safety features of the software system architecture include one or both of the following.

[0069] Force Threshold Stop: Both the Lab VIEW control system and the embedded safety system can acquire force and torque signals from the F / T sensor and limit UR5 motion when a set threshold is reached or exceeded. The Lab VIEW control system can compare the force to a conservative "warning" threshold, and if it is reached or exceeded, the system adjusts its processing to stop motion control along the force-increasing direction(s) but allows motion control along the force-decreasing direction(s), thereby allowing the physician to disconnect the system and reduce the force. The embedded safety system can compare the force to a safety threshold greater than the "warning" threshold, and if such a value is reached or exceeded, an emergency signal to the UR5 safety controller unit can be activated, which can disable all control of the UR5 until the system is manually re-enabled by the physician.

[0070] Safety Enable / Disable: To protect against unintended movement at critical points (e.g., during cutting), the embedded safety system may include a push button that toggles between enable and disable modes, which may be indicated by an enable / disable LED. At an appropriate time, the physician may disable the motion capabilities of the uterine manipulation system by toggling the push button to disable mode. In this mode, an output signal is generated by the Robot Relay Control, which triggers a relay within the UR5 robot's safety controller, causing motion or free-drive commands from Lab VIEW to be ignored.

[0071] Software System Items and Units

[0072] FIG. 13 illustrates a software system architecture according to an exemplary embodiment of the present invention. The primary custom software components are an “Embedded / Safety System” that monitors force and / or torque and disables the robot when a threshold value is exceeded, and a “Lab VIEW System” that provides real-time control of the UR5 robot arm. Each of the top-level software items is broken down into individual software units as illustrated in FIG. 14. Example 1 further describes the Embedded / Safety System, and Example 3 provides additional details regarding each of its supporting units. Example 2 further describes the Lab VIEW System, and Example 4 provides additional details regarding each of its supporting units. Refer to FIG. 13 and FIG. 14:

[0073] In the "Lab VIEW system" ("Lab VIEW control unit"), the "Control" unit provides central coordination of the Lab VIEW unit; this unit receives user input from "Kinematics," "Force / Torque Readings," and the user interface via the display and generates commands for the robot. The "Kinematics" unit converts joystick input into desired uterine manipulation movements. The "Force / Torque Readings" unit monitors the F / T sensor and limits the movement of the uterine manipulator based on the F / T sensor values. The "TCP / IP Reading" unit reads the UR5 status using the UR5 real-time communication interface. The "TCP / IP Write" unit writes commands for the UR5 using the URScript programming language. "Lab VIEW" is a real-time processing platform for acquisition, computation, and control. "Joystick Firmware" is firmware within the joystick that converts position into electrical signals. "Display Firmware" ("Touchscreen Firmware") is firmware within the touchscreen that provides control and touch processing. "NI module firmware" is the firmware within the NI digital output module.

[0074] In the "embedded / safety system," the "Main Loop" is an infinite loop that provides time-based functions and coordination for other units. The "GPIO (General Purpose Input / Output) unit acquires and conditioneds digital inputs from an enable / disable switch. The "ADC (Analog-to-Digital Converter) unit acquires voltage readings from each of the six force and torque channels generated by the F / T sensor. The "Force Calculation" unit converts the measured voltage from the F / T sensor into force and torque measurements. The "Robot Relay Control" unit provides control of the robot power relay; the digital I / O output can enable / disable the robot by connecting / disconnecting power from the system.

[0075] In the "UR5 software system," the "main controller" provides coordinated motion control of the UR5 robot arm. The "safety controller" is supervisory firmware for the motion control output of the UR5 main controller; this firmware receives enable / disable inputs and emergency stop, and controls the power output to the robot motor controller.

[0076] In some embodiments, “SOUP” software units that may be used in an exemplary software system architecture are as follows. “Lab VIEW” may include, for example, National Instruments Lab VIEW 2016 vl6.0. “Joystick Firmware” may include, for example, APEM HF46S10U HID Game Controller 6.1.7601.1899. “Touchscreen Firmware” may include, for example, Gechic On-Lap 15031 358B10L5T0309. “NI Module Firmware” may include, for example, National Instruments NI-9485 MAX 17.0. UR5 “Main Controller” may include, for example, UR5 3.4.1.59. UR5 “Safety Controller” may include, for example, UR5 URSafetyA 504 and URSafetyB 256. In other embodiments, depending on the selected specific robot arm, joystick, display, and software development environment, different versions and / or different software units may be used.

[0077] Software calculation

[0078] Force / Torque: In some embodiments, when operation is performed, the system continuously monitors force and torque readings. Three components of force and torque (F x , F y , F z , T x , T y , T zThis is combined with one force magnitude and one torque magnitude. Limits for force and torque (e.g., the aforementioned Lab VIEW and embedded safety force and torque thresholds) are preset within the software (and preferably independent of each other). It should be noted that in certain embodiments, different force and / or torque thresholds may be set for different movements. When the force or torque reaches the preset limit, a "trip" switch is flipped within the controller's logic, and the operation method in which the limit was exceeded (in any direction or combination, such as upward pitch, forward translation, etc.) is saved. When the trip is enabled, the user may use the "Home" button on the user interface or use a joystick to move in the opposite direction to the movement that caused the trip. For example, if the force or torque limit was exceeded by forward translation, the only permitted movement may be home or reverse translation. Any other joystick input may be ignored by the controller, and the robot may not move. Trip is disabled only when the force and torque are reduced to a slight margin below a preset limit. For example, if a force limit of 7 lbf is exceeded and trip is enabled, trip may not be disabled until the force is reduced to less than 5 lbf (2-pound margin). This deadband (margin) prevents minor fluctuations in the measurement from affecting the state of the trip and requires the user to take a separate action to disable the trip. When trip is disabled, movement restrictions are released, and normal operation resumes, including the direction that caused the trip.

[0079] Pitch / Yaw: In some embodiments, pitch and yaw movements are calculated as follows. In each iteration of the kinematic module (preferably operating at 10 Hz), pitch and yaw inputs are read from the joystick (in this embodiment, if translational inputs are received, pitch and yaw inputs are ignored). Relatively small pitch and yaw are selected to calculate a new manipulator tip position. The use of small increments results in smooth robot arm motion (better approach to the curve). Subsequently, a new manipulator handle position and orientation (i.e., robot arm position and orientation) is calculated, which will place the tip in the new position and keep the manipulator aligned with the pivot point. By comparing the newly calculated robot arm position and orientation with the current position and orientation, a movement velocity vector is generated. The uterine manipulation system controller sends a command to the UR5 to move along a specific vector at a specific speed. This command is updated 10 times per second, resulting in a series of vectors that approximate the desired motion curve.

[0080] Translational Motion: In some embodiments, translational motion is calculated in a manner similar to pitch / yaw motion. In each iteration of the kinematic module, a trajectory is generated that passes through both the fulcrum and the tip of the uterine manipulator. This is the line along which translational motion will follow, as illustrated in FIG. 12. The direction of translational motion (e.g., forward / into the uterus or backward / outto the uterus) is read from the joystick. Subsequently, the new handle / robot arm position and orientation are calculated in two steps. First, it is assumed that the entire uterine manipulator is moved to place the manipulator tip in a new position (i.e., the orientation does not change). However, this causes the manipulator to be misaligned from the fulcrum. Second, it is assumed that the manipulator is rotated around the new tip position so that the manipulator is realigned with the fulcrum. Next, the robot arm position and orientation newly calculated from these two steps are compared with the current position and orientation to generate a velocity vector, which is then transmitted to the UR5 as a command and updated 10 times per second.

[0081] Examples

[0082] Example 1: Embedded / Safety System

[0083] In some embodiments, the embedded / safety system is configured as follows.

[0084] 1.1 - Hardware Platform

[0085] The controller board is an ATMEL XMEGA-C3 development board. The controller board utilizes an Atmel ATxmega384C3 8 / 16-bit microcontroller operating at 32 MHz via an internal oscillator. The internal timer uses 32 MHz as its fundamental frequency and divides it as needed. Firmware developed for this microcontroller performs all real-time control and monitoring functions required for this device. An operating system is not used with this microcontroller.

[0086] The microcontroller includes a watchdog timer configured to reset the processor when the software is interrupted or fails to satisfy its execution time.

[0087] An IEEE 1149.1 compliant JTAG interface can be used on the embedded board of a uterine manipulator device to program and debug the microprocessor. This JTAG interface allows developers to load application programs and bootloader programs of the microprocessor, and to set or clear Atmel AVR ATxmega384C3 bits for firmware protection, brownout detection, bootloader flash address range, JTAG enable, watchdog enable, etc. The JTAG interface is used only for development and is not accessible to end users.

[0088] 1.2 - Main Loop

[0089] The microprocessor firmware program is structured around a service loop. The service loop is driven by an internal 16-bit timer that interrupts the AVR processor 100 times per second. The purpose of the main loop is to initialize the unit and then provide accurate timing for the various tick functions of the complex unit.

[0090] The 100 Hz service loop resets the watchdog timer in each of its executions. During software initialization, the processor STATUS register is checked to determine if the reset was caused by the watchdog timer expiring. In such cases, the robot enters a safe state (opening the safety and emergency relays), further code execution is prevented, and the use of the robot is consequently stopped.

[0091] 1.3 - Finite State Machine

[0092] FIG. 14 illustrates states and transitions including an embedded safety system finite state machine (FSM).

[0093] When the power is turned on, the first state is the initialization state. The ADC software unit interfaces with an external analog-to-digital converter chip that has a specific timing sequence required for initialization. The entire FSM remains in the initialization state until the ADC unit initialization is successful. The enable / disable output for the UR5 robot is enabled, allowing the UR5 robot to be initialized and enabling the user to configure the robot.

[0094] The unit remains in a monitoring state for most of the device's operation. During this state, the lower-level unit monitors force and torque and allows the user to enable and disable the robot.

[0095] Errors at any given time will result in a transition to an error state. Conditions causing errors include unexpected states within the entire finite state machine, unexpected finite state machines within the ADC unit, and excessive force or torque values.

[0096] The FSM has no inherent timing considerations and simply monitors signals from other units (primarily buttons and various process data sources) and subsequently generates commands for the units accordingly. The commanded units then space out their activities in relation to time. To process and respond to new user input or detected conditions as quickly as possible, the FSM exits the service loop at every iteration of the infinite loop and executes thereafter.

[0097] 1.4 - Support Units

[0098] Refer to Example 3 for a detailed description of each support unit that manages the subsystem and is coordinated by the main loop unit.

[0099] Example 2: Lab VIEW System

[0100] In some embodiments, the Lab VIEW system is configured as follows.

[0101] 2.1 - Hardware Platform

[0102] Lab VIEW hardware includes the Lab VIEW computer and the Lab VIEW hardware controller.

[0103] The Lab VIEW computer is a Windows 7 computer (Dell Optiplex 3050) that runs a custom Lab VIEW VI (virtual mechanism) on top of the Lab VIEW 2016 runtime engine. The Lab VIEW computer provides an interface with a touch screen panel, a USB interface for communicating with a Lab VIEW hardware controller, and an HDMI output for an Ethernet port to communicate with the UR5 robot.

[0104] The Lab VIEW hardware controller (National Instruments cDAQ-9174) provides an interface between the Lab VIEW computer and hardware modules (NI-9205 and NI-9485). The NI-9205 hardware module is an analog voltage input module used to digitize force and torque signals from the ATI-IA F / T sensor on the end effector. Similarly, the NI-9485 hardware module provides a solid-state relay that controls the free drive and enable / disable inputs of the UR5 robot.

[0105] 2.2 - Architecture

[0106] A custom LabVIEW VI contains multiple units running in parallel loops, utilizing data and state communicated between the loops. The top-level units operating in parallel are listed in the hierarchy described below. The Control and Force-Torque Reading units are loops containing subsequent units. TCP Creation, Kinematics, and TCP Reading are sub-VIs within the higher-level Control unit.

[0107] ● Control

[0108] o TCP creation

[0109] ■ kinematics

[0110] o TCP read

[0111] ● Force-Torque Reading

[0112] 2.3 - Control

[0113] The control unit is primarily responsible for controlling the UR5 robot. The control unit acquires user input through the GUI (e.g., as illustrated in FIG. 15). The control unit also receives user position input via the joystick. The input is processed in a loop running for 10 milliseconds. The fulcrum position (e.g., cm) is entered based on the patient-specific distance measured by the uterine manipulator. This patient-specific distance will change the geometry and handle motion.

[0114] Input is used to influence the state of the UR5 robot. User position input is provided to the kinematic unit. A corresponding change to the UR5 robot is commanded through the TCP authoring unit. The TCP authoring unit generates a command for the UR5 to change the state or position. The control unit reads the state of the UR5 robot through the TCP reading unit, which analyzes feedback from the robot.

[0115] The control unit also generates a log file containing the position and status of the UR5 robot.

[0116] Lab VIEW VI has five states that control its behavior: Init, Freedrive, Write Data / Manipulation, Not Normal, and Close.

[0117] Init: This is the initial state of the Lab VIEW software. A log file is created, and variables are commanded in that initial state. The state transitions to the free-run state or the Write Data / Manipulation state.

[0118] Freedrive: This state is entered after initialization and when the user commands the drive mode to become free drive mode. Only on the first call does this state initialize the joystick firmware. The UR5 robot is commanded to free drive mode. This state is maintained until the user changes the operation mode in the UI, closes the UI, or feedback from the UR5 robot indicates an abnormal state.

[0119] Write Data / Manipulation: This state is input when the user sets the drive mode to enter operation mode. The joystick state is read, transmitted via the kinematic unit, and finally used by TCP writing to send commands to the UR5 robot. The TCP reading unit is also executed to monitor the robot's state.

[0120] Not Normal: This state is entered when UR5 robot feedback, as analyzed by the TCP read unit, returns to an unexpected state. In response, the robot is commanded to return to a safe state. This state is also logged.

[0121] Close: This state is executed when the user closes the application. All resources, including log files, are properly released.

[0122] 2.4 - Force Torque Reading

[0123] The force-torque reading unit configures the NI-9205 to acquire six channels of analog voltage at 1,000 samples per second. These analog channels are wired to the outputs of the ATI-IA Mini40 3-axis F / T sensor. The acquired voltages are converted to lbf and lbf-inch for the force and torque channels, respectively.

[0124] 2.5 - Support Unit

[0125] Refer to Example 4 for a detailed description of each of the support units.

[0126] Example 3: Support unit for embedded / safety systems

[0127] In some embodiments, the remaining units of the embedded / safety system are as shown in Table 1.

[0128] [Table 1]

[0129]

[0130] 3.1 - ADC Unit

[0131] outline

[0132] The ADC unit communicates with the analog device AD7616 analog-to-digital converter chip via the Serial Peripheral Interface (SPI). The ADC chip acquires data from the ATI-IA 3-axis F / T sensor at a rate of 100 samples / second.

[0133] Common interface

[0134] Table 2 describes an ADC common method including any object or data item transmitted from a unit to be used by another software unit.

[0135] [Table 2]

[0136]

[0137] Normal operation

[0138] The ADC unit has an internal FSM to properly initialize the AD7616 IC. The sequence of operation transitions through the states is listed below:

[0139] STATE WAIT FOR RESET: The reset pin on the AD7616 is activated upon entry into this state. 20 milliseconds may elapse prior to further initialization using the IC. When 20 ms expire, the unit transitions to STATE SET REGISTERS.

[0140] STATE SET REGISTERS: During the first execution of this state, the unit sends an SPI command to set the configuration registers of the ADC. During the second execution, the unit sets the termination of the sequence registers.

[0141] STATE INITIAL PSEUDO ACQ: The first conversion of the ADC is expected to be meaningless. In this state of the FSM, the first conversion is acquired and the data is discarded.

[0142] STATE ACQUISITION: Voltages are acquired for 6 voltage channels.

[0143] STATE ADC FAULT: An unexpected input from the ADC leads to a transition to this state.

[0144] 3.2 - Strength Unit

[0145] outline

[0146] The power unit communicates with the ADC unit and receives the raw voltage from the ATI-IA F / T sensor. Based on manufacturer-provided calibration information, the unit converts the raw voltage into power and torque values.

[0147] Common interface

[0148] Table 3 explains the power unit common method.

[0149] [Table 3]

[0150]

[0151] Normal operation

[0152] The power unit acquires the average of 10 data points for the following channels (sample rate: 100 Hz; average window: 10 ms):

[0153] ● Force of the X-axis

[0154] ● Force on the Y-axis

[0155] ● Z-axis force

[0156] ● X-axis torque

[0157] ● Y-axis torque

[0158] ● Z-axis torque

[0159] The raw voltage is converted into force and torque using a matrix of values ​​provided by the manufacturer specified in the data section below.

[0160] The force unit also monitors each force and torque channel (voltage) to determine whether they are below the set force or torque limits for a specific anatomical operation.

[0161] data

[0162] The ATI-IA Mini40 3-axis F / T sensor has 6 voltage readings (F x , F y , F z , T x , T y , T z A correction table is provided to correct the ) to force and torque values ​​in lbf and lbf-inch, respectively. The conversion table (correction table) is provided in Table 4 below. This data is extracted from ATI certification number FT21031-20171219.

[0163] [Table 4]

[0164]

[0165] The correction transformation is the voltage vector ([F x , F y , F z , Tx , T y , T z It is performed by matrix multiplication between ]) and the correction matrix listed in Table 4.

[0166] 3.3 - Relay Unit

[0167] survey

[0168] The relay unit removes the relay used to control the UR5 Robot Safeguard and Emergency Break inputs.

[0169] Using robot safety inputs, control whether motion is enabled or disabled during normal operation based on operator input. A call to the "enable" function (for enable_safeguard_relay()) causes the corresponding IO to logic level high, which enables motion within the UR5 robot. Alternatively, a call to the "disable" function (for disable_safeguard_relay()) causes the corresponding IO to logic level low, which stops motion within the UR5 robot. A call to the "toggle" function (for toggle_safeguard_relay()) changes the current state from disabled to enabled or vice versa.

[0170] Use the robot emergency stop input to immediately stop all robot motion. The "enable" function call (for enable_emergency_relay()) causes the corresponding IO to logic level high, which enables motion within the UR5 robot. Alternatively, the "disable" function call (for disable_emergency_relay()) causes the corresponding IO to logic level low, which immediately disables all motion within the UR5 robot.

[0171] Common interface

[0172] Table 5 explains the relay unit common method.

[0173] [Table 5]

[0174]

[0175] Normal operation

[0176] Upon initialization, the unit enables both the safety relay and the emergency relay.

[0177] After that, the unit enables and disables the output according to the common method listed above.

[0178] data

[0179] Table 6 describes the relay unit data.

[0180] [Table 6]

[0181]

[0182] 3.4 - GPIO / Button Unit

[0183] survey

[0184] The GPIO unit removes the user push button ("safety button"). The unit uses 50 Hz ticks to sample the button to provide de-bouncing. The unit provides a high-level query function to indicate whether the push button has been pressed since the last check.

[0185] Common interface

[0186] Table 7 describes the GPIO / button unit common method.

[0187] [Table 7]

[0188]

[0189] Normal operation

[0190] The unit monitors safety push button input for user presses.

[0191] data

[0192] Table 8 describes the GPIO / button unit data.

[0193] [Table 8]

[0194]

[0195] Example 4: Support Unit for Lab VIEW System

[0196] These embodiments describe the following support units for the Lab VIEW system "Control" unit / subsystem: kinematic unit, TCP read, and TCP write.

[0197] 4.1 - Kinematic Unit

[0198] survey

[0199] The kinematic unit receives and calculates the inputs listed below and commands the corresponding robot position:

[0200] ● Joystick Input

[0201] ● Current robot location

[0202] ● Kinematic initialization position: Robot setup or home position and orientation. When the operator switches from free drive to operation mode, the robot's setup / home position is recorded.

[0203] logic

[0204] Kinematic code according to exemplary embodiments is provided in the attachment to the list of computer programs, the entirety of which is incorporated herein by reference. The kinematic code of the embodiments of the invention is specific to the geometry of a specific end effector held / manipulated by a robot and to the anatomical tissue of an individual patient (e.g., the patient-specific distance from the tip to the fulcrum of the uterine manipulator as described above). Various existing tools for use with surgical robots are linear and, accordingly, have relatively intuitive kinematics. In contrast, tools for uterine manipulation, such as VCare®, are curved and have a patient-specific adjustable fulcrum; the kinematic code of the embodiments of the invention includes logic for controlling the end point of such tools and for providing smooth movement for anatomical manipulation during surgery. In the present embodiment, there is a hardcoded offset of about a few inches within the logic describing the thickness of the F / T sensor and the depth of the rear portion of the mechanical interface (i.e., the distance between the end of the manipulator handle and the distal end of the robot arm). In logic, this additional distance is added to the length of the manipulator tool, having the effect of making the tool longer, and accordingly, such distance is taken into account in the calculation of the movement vector.

[0205] Parameters within the exemplary kinematic code include the following: "L", the distance (cm) from the robotic tool to the VCare® tip along the VCare® handle axis; "tipY" / Tip Elevation (cm), the distance from the robotic tool (distal end) perpendicular to the VCare® handle axis to the VCare® tip; "ful_pos_pat" / Fulcrum Position (cm), the patient-specific distance measured from a gradient mark on the VCare® indicating the position of the cervix through an execution (which can be re-initialized); "theta max" / Max Optimization Speed ​​(degrees / sec), the maximum speed limiting how quickly the correction can be performed; and "dist lookup" / Distance Lookup Table (cm), a reference table used to correlate the VCare® coordinates from the tip to the patient-specific distance from the fulcrum. To perform pitch / yaw motion, additional parameters include the following: "dt"; "pitch_max" / Max Pitch Speed ​​(degrees / sec); and "yaw_max" / Max Yaw Speed ​​(degrees / sec). To perform distance movement, additional parameters include: "dD" (cm), an incremental distance used to plot a trajectory for distance movement; "minSpeed_dist" / Min Distance Speed ​​(cm / s), a distance speed corresponding to the minimum speed toggle; and "maxSpeed_dist" / Max Distance Speed ​​(cm / s), a distance speed corresponding to the maximum speed toggle. Distance (translational) movement is performed by selecting a new point for the VCare® tip along a straight line from the pivot point to the tip.As noted above, to calculate the new translational position of VCare®, the code first "translates" VCare® so that the tip is in the new position, and then "rotates" VCare® around the new tip position so that the new fulcrum position on VCare® returns to the fixed cervical canal. Using the final position and orientation of the handle based on the calculation, a robot command is provided, and accordingly, both translational and rotational motions occur simultaneously.

[0206] In some embodiments, the kinematic code includes four modules as described below: KINEMATICS INITIALIZE, PRE KINEMATICS, KINEMATICS - PITCH / YAW, and KINEMATICS - DISTANCE.

[0207] KINEMATICS INITIALIZE: This code runs only once each time the operation mode is entered. It sets parameters based on the robot's initial orientation used by the kinematics module in each iteration.

[0208] PRE KINEMATICS: This code operates in each iteration (10 times per second) during operation mode. It reads the robot's current orientation and calculates geometric parameters used by the kinematics module.

[0209] KINEMATICS - PITCH / YAW: This code operates in each iteration (10 times per second) while in operation mode when a pitch / yaw command is received. It reads the current joystick command (filtered by force / torque inputs and other controls), the current robot state output from Pre-Kinematics, and the initialized parameters from Kinematics Initialize, and calculates the appropriate speed command to transmit to the robot. It also performs some simple closed-loop control designed to prevent the VCare® from being pulled too far toward the patient at the insertion point (cervix). Finally, the code performs an "optimization" step designed to prevent the VCare® handle from moving too far laterally from its initial position. If the VCare® handle moves too far laterally, the robot may reach the edge of that range.

[0210] KINEMATICS - DISTANCE: This code operates in each iteration (10 times per second) while in operation mode when a distance (translational) command is received. It reads the current joystick command (filtered by force / torque input and other controls), the current robot state output from Pre Kinematics, and the initialized parameters from Kinematics Initialize, and calculates an appropriate speed command to transmit to the robot.

[0211] 4.2 - TCP Read Unit

[0212] The TCP reading unit analyzes feedback received from the UR5 robot.

[0213] 4.3 - TCP Authoring Unit

[0214] The TCP authoring unit converts UR5 robot commands into the syntax expected by the UR5 robot controller.

[0215] Although the basic novel features of the present invention as applied to preferred and exemplary embodiments have been illustrated and described, it will be understood by those skilled in the art that omissions, substitutions, and modifications to the forms and details of the disclosed invention may be made without departing from the spirit of the invention. Furthermore, as is very clear, many modifications and changes may be readily made by those skilled in the art. For example, various features and structures of the different embodiments described herein may be combined or interchanged. Accordingly, it is not desirable to limit the present invention to the exact configurations and operations illustrated and described, and accordingly, all appropriately modified equivalents may be included within the scope of the invention as claimed. Therefore, it will be limited only to what is indicated by the claims appended herein.

[0216] Claim scope

[0217]

[0218]

[0219]

[0220]

[0221]

[0222]

[0223]

[0224]

[0225]

[0226]

[0227]

[0228]

[0229]

[0230]

[0231]

[0232]

[0233]

[0234]

[0235]

[0236]

Claims

Claim 1 A robotic anatomical manipulation system for use with an end effector configured to position and maintain a patient's anatomical tissue during minimally invasive surgery, wherein the robotic anatomical manipulation system comprises: a robot comprising a cart and an arm, wherein the proximal end of the arm is connected to the cart and the arm comprises a plurality of joints that enable a multi-range of motion; a console comprising a joystick and a user interface, wherein the joystick is configured to provide real-time motion input to the arm; a mechanical interface connected to the distal end of the arm, wherein the mechanical interface is configured to releaseably support a proximal handle of the end effector outside the patient's anatomical tissue, wherein the end effector comprises a slender body that is inserted into a part of the patient's anatomical tissue and conforms to a part of the patient's anatomical tissue, a distal end tip, and an adjustable fulcrum positioned at a distance from the distal end tip unique to a part of the patient's anatomical tissue; and a sensor positioned between the distal end of the arm and the mechanical interface, wherein the mechanical interface is end A sensor configured to transmit at least one of force and torque from an effector to a sensor, wherein the sensor is configured to measure at least one of force and torque along each of three axes; and a control system communicating with the arm, console and sensor, wherein the control system comprises a computer processor configured to execute a program stored on a non-transient computer-readable medium that controls the arm in response to user input through the joystick and the user interface;The control system comprises a kinematic unit configured to receive input from a joystick and a sensor and generate control commands to control the motion of the arm in order to generate a user-specific motion of the tip based on the current position of the arm, the shape of the end effector, and the patient-specific fulcrum position of the end effector, wherein the control commands include one or a combination of translational motion, translational motion speed, rotation, rotational speed, pitch, yaw, enable, and disable; A robotic anatomical manipulation system comprising an embedded safety system including a force-torque reading unit configured to continuously monitor at least one of the force and torque detected by the sensor, calculate the sum of at least one of the force and torque, and provide a signal including the calculated sum to a user interface, wherein the embedded safety system further comprises an embedded safety system configured to compare the calculated sum with one or more predetermined threshold values ​​and, if one or more of the threshold values ​​are exceeded, generate a disable signal that disables arm motion within the whole or a subset thereof of a multi-range of motions. Claim 2 A robotic anatomical manipulation system according to claim 1, wherein the anatomical tissue is a uterus and the end effector is a uterine manipulator. Claim 3 delete Claim 4 delete Claim 5 A robotic anatomical manipulation system according to claim 1, wherein the console further comprises a video feed configured to display a live signal from a camera used to monitor the surgery. Claim 6 A robotic anatomical manipulation system according to claim 1, wherein the arm comprises six joints: a base (A), a shoulder (B), an elbow (C), and a wrist portion (D, E, F). Claim 7 In claim 6, the above-mentioned elbow is positioned below the above-mentioned wrist portion at a baseline position, in a robotic anatomical manipulation system. Claim 8 A robotic anatomical manipulation system according to claim 1, wherein the arm comprises at least one joint having a marking, and the marking indicates a safe range for a baseline position. Claim 9 A robotic anatomical manipulation system according to claim 1, wherein the mechanical interface comprises a rapid release mechanism configured to release the end effector without movement of the robot arm or cart. Claim 10 In claim 1, the sensor has six channels of force and torque: F x , F y , F z , T x , T y , and T z A robotic anatomical manipulation system designed to acquire Claim 11 A robotic anatomical manipulation system according to claim 1, wherein one or more of the threshold values ​​include one or more of a warning force threshold, a safety force threshold, a warning torque threshold, and a safety torque threshold. Claim 12 delete Claim 13 delete Claim 14 A robotic anatomical manipulation system according to claim 1, wherein the embedded safety system comprises an analog-to-digital converter (ADC) unit configured to acquire force and torque data from the sensor at least 100 times per second. Claim 15 In claim 14, the embedded safety system further comprises a force unit configured to receive a raw voltage communicated to the ADC unit by the sensor and convert the raw voltage into force and torque values. Claim 16 delete Claim 17 A robot anatomical manipulation system according to claim 1, wherein the console includes a mechanism for enabling / disabling control of the arm from the joystick and the user interface, and the embedded safety system responds to generate a command for the robot. Claim 18 A robotic anatomical manipulation system according to claim 1, wherein the user interface comprises a stop axis display unit configured to provide a visible display indicating an axis along an axis where one or more predetermined threshold values ​​are exceeded. Claim 19 A robotic anatomical manipulation system according to claim 1, wherein the cart further comprises a setup positioning system on its upper surface, the proximal end of the arm is connected to the setup positioning system, and the setup positioning system is configured to provide forward and reverse translational motion in the X, Y, and Z directions. Claim 20 A robotic anatomical manipulation system according to claim 1, wherein the control system is configured to generate an alarm that provides an audible indication of at least one of the movement and speed of the distal end of the arm.

Images