Systems and methods for generating and evaluating medical procedures

Abstract

The system may include a processor and a memory having computer-readable instructions stored thereon. The computer-readable instructions, when executed by the processor, may cause the system to generate a treatment plan for performing a procedure with a robot-assisted manipulator. The treatment plan may be based on a first plurality of treatment inputs. The system may also generate performance indicators from the performance of the procedure, evaluate the performed procedure based on the performance indicators to generate treatment evaluation information, and store the treatment evaluation information. The system may also generate a second treatment plan based on the stored treatment evaluation information and a second plurality of treatment inputs.

Description

Technical Field

[0001] Cross-Referenced Applications This application claims the benefit of U.S. Provisional Application No. 63 / 120,191, filed on December 1, 2020, which is hereby incorporated by reference in its entirety.

[0002] This application is hereby incorporated by reference in its entirety to U.S. Provisional Application No. 63 / 120,175, filed on December 1, 2020, entitled "SYSTEMS AND METHODS FOR GENERATING VIRTUAL REALITY GUIDANCE", and U.S. Provisional Application No. 63 / 120,140, filed on December 1, 2020, entitled "SYSTEMS AND METHODS FOR PLANNING A MEDICAL ENVIRONMENT".

[0003] The present disclosure is directed to systems and methods for robotic-assisted medical procedures, and more particularly, to the development of medical environment planning based on the operating modes of robotic-assisted medical systems.

Background Art

[0004] Planning tools for performing medical procedures with a teleoperated robot or robotic-assisted system are often generic and not adaptable to specific surgeons, patients, or other parameters. Further, the planning tools are static and may not respond to information that could improve patient outcomes. There is a need for systems and methods that assist medical personnel by providing treatment planning tools that adapt to various parameters and evaluate procedures performed to identify areas for improving efficiency and patient outcomes.

Summary of the Invention

[0005] Embodiments of the present invention are best summarized by the claims that follow.

[0006] In accordance with some embodiments, the system may include a processor and memory storing computer-readable instructions. When executed by the processor, the computer-readable instructions may cause the system to generate an action plan for performing an action with a robot-assisted manipulator. The action plan may be based on a first set of action inputs. The system may also generate performance metrics from the implementations, evaluate the actions performed based on the performance metrics to generate action evaluation information, and store the action evaluation information. The system may also generate a second action plan based on the stored action evaluation information and a second set of action inputs.

[0007] In some embodiments, the system may include a processor and memory storing computer-readable instructions. When executed by the processor, the computer-readable instructions may cause the system to receive the treatment type of a planned procedure for a robot-assisted manipulator, a set of patient information for a patient undergoing the planned procedure, and generate an analysis of previous treatment data based on the received treatment type and set of patient information. The system may also generate a set of setup instructions for the planned procedure based on the analysis.

[0008] It should be understood that both the general description above and the detailed description below are illustrative and descriptive in nature and are intended to provide an understanding of the disclosure without limiting its scope. In this regard, additional aspects, features, and advantages of the disclosure will be apparent to those skilled in the art from the detailed description below. [Brief explanation of the drawing]

[0009] [Figure 1] This flowchart shows a method for generating and evaluating treatment plans according to several embodiments. [Figure 2A] This shows the user interface for treatment planning. [Figure 2B] This shows the user interface for treatment planning. [Figure 2C] This shows the user interface for treatment planning. [Figure 2D] This shows the user interface for treatment planning. [Figure 2E] This shows the user interface for treatment planning. [Figure 2F] This shows the user interface for treatment planning. [Figure 2G] This shows the user interface for treatment planning. [Figure 2H] This shows the user interface for treatment planning. [Figure 2I] This shows the user interface for treatment planning. [Figure 3] This is a flowchart showing how to generate setup instructions for a medical procedure. [Figure 4] This is a schematic diagram of a robot-assisted medical system according to several embodiments.

[0010] The embodiments of this disclosure and their advantages will be best understood by referring to the detailed description below. It should be noted that similar reference numerals are used to identify similar elements shown in one or more figures, and that those shown therein are for illustrative purposes only and not for limiting purposes the embodiments of this disclosure. [Modes for carrying out the invention]

[0011] Procedure planning tools can support the efficient, safe, and effective use of robot-assisted systems in healthcare settings. Adaptive procedure plans can respond to specific inputs for particular healthcare procedures and incorporate improvements and efficiencies identified in previous procedures. As described below, evaluations and analyses performed in previous procedures can be used to generate procedure plans, including setup instructions for robot-assisted systems in healthcare settings.

[0012] Figure 1 is a flowchart of method 100 for generating and evaluating action plans according to several embodiments. The method described herein is shown as a set of actions or processes and will be described with further reference to additional figures. Not all processes shown are performed in all embodiments of the method. Furthermore, one or more processes not explicitly shown may be included before, after, between, or as part of the processes shown. In some embodiments, when one or more processes are performed at least partially by one or more processors (e.g., processors of a control system), they may be implemented in the form of executable code stored in a non-temporary, tangible, machine-readable medium that can cause one or more processors to execute one or more processes. In one or more embodiments, the processes may be performed by a control system.

[0013] In process 102, a treatment plan may be generated for use in a robot-assisted medical system in a medical environment. The treatment plan is developed based on various inputs 110, including, for example, treatment type 112, surgeon information 114, facility information 116, staff information 118, patient information 120, expert guidance 122, and previous treatment information 124. In some embodiments, the inputs are received by a user interface device, such as a display on an operator interface system, a fixed or movable auxiliary display in the medical environment, or a display on a mobile device such as a telephone, tablet, camera, laptop, or other portable device, and the treatment plan may be displayed on that user interface device. In some embodiments, the user interface device may measure, scan, image, or otherwise record spatial information about the medical environment from inside or near the medical environment. Thus, participants in the treatment, which may include, for example, surgeons, clinical staff, and / or instructors or supervisors, can review the treatment plan and adjust it before implementation.

[0014] Figure 2A shows a mobile user interface device display 200. The display 200 may include a user interface 204 for receiving a procedure type input 112 indicating the type of procedure to be performed by the robot-assisted medical system. The procedure type may be provided for selection from a menu including menu options 206 and 208. Any number of menu options representing any number of procedure types may be displayed for selection. In alternative embodiments, the procedure type input 112 may be indicated by other means, such as selection from a drop-down menu, index search, or other known selection techniques. In various embodiments, possible procedures to be selected include general surgical procedures, including ventral and inguinal hernia repair and bariatric surgery. In various embodiments, possible procedures to be selected include colorectal surgical procedures, including colectomy and rectal resection. In various embodiments, possible procedures to be selected include gynecological surgical procedures, including hysterectomy and myomectomy. In various embodiments, possible procedures to be selected include urological surgical procedures, including prostate, bladder, and kidney cancer surgery. In various embodiments, possible procedures to be selected include thoracic surgical procedures, including lobectomy and mediastinal mass surgery. In various embodiments, possible procedures to be selected include cardiac surgical procedures, including mitral valve repair. In various embodiments, possible procedures to be selected include head and neck surgical procedures, including pharyngeal cancer surgery. In various embodiments, procedures may include diagnostic or investigative procedures, including biopsy.

[0015] FIG. 2B shows a mobile user interface device display 200 including a user interface 210 for indicating an input to a selected procedure. The interface 210 may include, for example, a button 212 for receiving an input of surgeon information 114, a button 214 for receiving facility information 116, a button 216 for receiving an input of staff information 118, a button 218 for receiving an input of patient information 120, and a button 220 for receiving an input of display of requested guidance 122. The user interface 210 may include other input mechanisms for inputting an input to a selected procedure. The surgeon information 114 may include, for example, a database of ID information, physical characteristic information (e.g., height, dominant hand), training information, qualification information, preference information, history with staff, and / or previous procedure information of the surgeon.

[0016] The facility information 116 may include, for example, geographical location information of the medical facility where the procedure is performed, room information, utilities information, available equipment information, or other information regarding the facility where the selected medical procedure may be performed. In some embodiments, selecting the facility input button 214 may prompt the user to record or capture spatial information. A measurement device for measuring the dimensions of a room may be, for example, a rangefinder, a lidar system, a camera, or a single-purpose device or other measurement tool that may be incorporated into a mobile user interface device such as a phone, a tablet, a laptop. In some embodiments, a camera may capture facility information regarding a room, including, for example, the location of equipment, the location of electrical outlets, the location of furniture, and / or the location of doors.

[0017] The staff information 118 may include, for example, a database of the number of staff, ID information, physical characteristic information (e.g., height, dominant hand), training information, qualification information, preference information, history with the surgeon, and / or previous procedure information of the staff.

[0018] Patient information 120 may include, for example, ID information, gender information, medical history, physical characteristic information (e.g., height, weight), pre-operative medical images (e.g., CT, MRI, X-ray images), information regarding disease progression, previous medical treatment information, and / or monitoring information (e.g., blood pressure, blood oxygen concentration, pulse). The patient information may be provided, for example, by input from a surgeon, clinical staff, or a database in which the patient information is stored. In some embodiments, the patient information may include the expected or planned patient position information, including the position and orientation of a part of the patient's anatomical structure, including the head, torso, and limbs on the operating table. A headrest, pad, support, and / or other positioning fixture on the operating table may be used to determine the expected or planned patient position. In some embodiments, the patient information may include the actually sensed patient position information, including the position and orientation of a part of the patient's anatomical structure, including the head, torso, and limbs on the operating table. A camera, pressure sensor, force sensor, or other sensing system inside or around the operating table may be used to determine the position of the patient during the procedure.

[0019] Guidance 122 may include expert recommendations or prior expert actions that are taken when performing the selected treatment type. The guidance may include a set-up configuration preferred by the expert, selection of equipment, patient position and orientation, sequence of the process, or other suggestions or best practices for performing the selected treatment. The guidance may also or alternatively include a templated treatment that is a general plan that can be customized based on other inputs 110. The guidance may also or alternatively include a preferred plan (or steps of the plan components) previously performed by the surgeon, a plan (or steps of the plan components) identified as preferred by the surgeon, or a plan (or steps of the plan components) identified as not preferred by the surgeon. Guidance 122 may be, for example, stored in computer memory for later access in a subsequent procedure or may be live guidance information provided by an expert in the same location or an expert in a remote location.

[0020] In process 102, inputs 112-124 may be used with reference to or in combination with previous treatment information 124 to generate a treatment plan. Previous treatment information 124 may include, for example, information about previous treatments of the same type performed by the same or different surgeons, at the same or different facilities, with the same or different staff, or on the same or different patients. Previous treatment information 124 may include, for example, best practices or practices to avoid based on efficiency, effectiveness, patient prognosis, or preferences recorded by the surgeons and staff. Previous treatment information 124 may be generated based on an evaluation of previously performed treatments, as shown below.

[0021] The generated treatment plan may include a treatment outline. Figure 2C shows a mobile user interface device display 200 that includes a user interface 230 for providing an outline of the selected treatment. The treatment outline may include a treatment description 232, a catalog 234 of instruments and supplies used during the treatment, a set of treatment instructions 236, and a trigger 238 for initiating the selected treatment.

[0022] After the selected procedure is initiated, multiple modules of the selected procedure may be presented. Figure 2D shows a mobile user interface device display 200 including a user interface 240 with a selection bar 242 and a selection indicator 244. The selection bar 242 contains reference numbers (0-6) of multiple modules corresponding to modules or subunits of the selected procedure. The selection indicator 244 is movable relative to the selection bar 242 to indicate the selection of a module. For example, module 0 may correspond to the anatomy module. Module 1 may correspond to the initial exposure and setup module. Module 3 may correspond to the vascular control module. The remaining modules may correspond to subunits of the selected procedure. A procedure may include any number of modules or subunits. The same type of procedure may even have different numbers of models based on the customization provided by input 110.

[0023] After a module is selected, a virtual image of the medical environment may be displayed. Figure 2E shows a mobile user interface device display 200 including the user interface 250 of module 0, which displays an image of the patient's anatomy 252 with a proposed surgical port arrangement 253. In this image, the patient's anatomy 252 may be located on an operating table 254. In some examples, other equipment such as a robot-assisted manipulator, surgeon's console, anesthesia cart, or other components may be included in the image. The image of the patient's anatomy 252 and table 254 can be displayed as a three-dimensional image by selecting the 3D image option 256. In the 3D image, the patient's anatomy 252 and table 254 can be moved with 3 degrees of freedom rotation. The image of the patient's anatomy 252 and table 254 can be displayed as a two-dimensional image by selecting the 2D image option 258, as shown in Figure 2F. The image of the patient's anatomy 252 and table 254 can be displayed as an augmented reality image by selecting the AR image option 260. In augmented reality imagery, a camera may capture still or moving images of the medical environment, and images of the patient's anatomical structure 252 and the platform 254 may be superimposed or otherwise combined with images of the medical environment to show how the patient's anatomical structure and the platform may be positioned in the actual medical environment space. The patient in the actual medical environment space may be aligned and scaled to the image of the patient's anatomical structure 252 so that the position of the port placement 253 is precisely positioned in the medical environment space. As shown in Figure 2G, a menu 262 may be presented by selecting an option 263 that allows the user to switch the image of the equipment on and off using a toggle switch 264 and the image of the person using a toggle switch 266.

[0024] As shown in Figure 2H, the user interface 250 of module 0 may further illustrate organ information 268 describing organs of anatomical structures that may be involved in the planned procedure, vascular information 270 describing blood vessels that may be involved in the planned procedure, and arterial information 272 describing arteries that may be involved in the planned procedure.

[0025] Figure 2I shows a mobile user interface device display 200 including a user interface 280 of module 1 that provides instructions for setting up a robot-assisted manipulator 282 to perform a planned procedure on a patient's anatomical structure 252. Setup module 1 may include 2D or 3D images of the recommended orientation of the patient's anatomical structure 252 and the base 254, the recommended port placement 253, the recommended placement of the manipulator 282 relative to the patient's anatomical structure 252 and the base 254, docking instructions for docking the manipulator 282 to the patient, recommended and optional instruments, recommended configurations of other furniture or components in the medical environment, the recommended placement of the manipulator setup joint 284, and / or other orientations and placements of objects in the medical environment space. Setup module 1 may also include a detailed explanation of the setup procedure steps 286 and a menu for selecting a demonstration video 288 of the procedure or steps of the procedure. Each of modules 0 through 6 may include 2D images, 3D images, sub-process instructions, explanatory videos, kinematic information of the robot-assisted manipulator, descriptions of the affected anatomical structures, descriptions of the instruments used, descriptions of the equipment used, and other text, graphics, videos, or interactive communication tools that may help in performing the steps of the module.

[0026] Referring again to Figure 100, in process 104, performance indicators may be generated during and / or after the procedure. Since the generated surgical plan is performed using a robot-assisted manipulator, deviations from the generated plan may occur due to expected and unexpected circumstances, such as conditions encountered in the patient's anatomical structure, the manipulator's performance, the staff's reactions to the conditions encountered with the patient or manipulator, and / or the surgeon's reactions to the conditions encountered with the patient or manipulator. During the procedure, the performance indicators may include kinematic information generated for the robot-assisted manipulator assembly and / or attached instruments, and may include structural information such as the dimensions of the components of the manipulator assembly and / or medical instrument, joint placement, component position information, component orientation information, and / or port placement. The kinematic information may also include dynamic kinematic information such as the range of motion, velocity or acceleration information, and / or resistance force of the joints in the remotely operated assembly. Structural or dynamic kinematic constraint information can be generated, for example, by sensors within a remotely operated assembly that measure the configuration of a manipulator's arm, the configuration of a medical device, the configuration of a joint, the displacement of a component, the velocity of a component, and / or the acceleration of a component. Sensors may include position sensors such as electromagnetic (EM) sensors, shape sensors such as fiber optic sensors, and / or actuator position sensors such as resolvers, encoders, and potentiometers.

[0027] Performance metrics may also include the total time taken for the procedure, the time taken for individual subunits or multiple subunits of the procedure, and the time taken to complete activities such as tool changes or maintenance activities. In some examples, performance quality metrics may include the accuracy of human responses when following instructions. For example, if the first manipulator arm is indicated to change a tool, but the staff member changes the tool with the second manipulator arm instead, the performance metric may indicate a failure to follow instructions. Performance metrics may be binary metrics, such as compliance / non-compliance, or continuous metrics. Continuous metrics may include, for example, the total distance traveled by staff, the amount of errors or unplanned incidents during the procedure, the amount of equipment used, the amount of equipment damaged, the amount of tools replaced, or severity metrics indicating the type of human intervention required during the procedure performed.

[0028] Performance metrics may also include post-procedure measures, which include indicators of the quality of patient prognosis, such as blood loss during and / or after the procedure, patient recovery time, patient readmission to a medical facility for related complications, and time to patient discharge, and post-procedure infection. Post-procedure measures may also include measures of damage or wear to the manipulator assembly.

[0029] During the execution of a procedure, performance indicators and / or real-time status information may be communicated, for example, to surgeons, clinical staff, instructors or supervisors, or the facility's logistics organization. Indicators and / or status information may be presented in any sensory format, including displays such as operator interface displays, auxiliary fixed or movable display components, or mobile displays. Real-time status information may include a list of procedure steps (e.g., a checklist) indicating which steps have been performed and which have not yet been performed. Real-time status information may be useful for mentor interventions during staff changes or procedures in progress. Other presentation formats may include auditory information, such as announcements of elapsed time or feedback on compliance with the planned procedure.

[0030] In process 106, the performed procedure may be evaluated based on performance metrics. Performance metrics may be compared to criteria developed by specialist surgeons and staff, benchmarks developed by analyzing multiple previous procedures, and / or criteria provided by the planned procedure. For example, kinematic information from the performed procedure may be compared to kinematic information recommended by the procedure plan, and the kinematic information of the performed procedure may be evaluated as to whether and to what extent it matched the planned kinematic information. In some examples, the evaluation may generate a score. Evaluation of performance metrics, including elapsed time, may indicate where delays or errors in the procedure occurred (e.g., in which subunits). In some embodiments, evaluating the performed procedure may include comparing the performance metrics with benchmark metrics and identifying suboptimal outcomes such as delays or errors. In some embodiments, evaluating the performed procedure may include comparing the performance metrics with benchmark metrics and identifying favorable model outcomes compared to the benchmark metrics. In some embodiments, the evaluation of the procedure may be based on actions or performance observed by the surgeon, staff, camera, or other sensors in the environment. In some embodiments, performance metrics may be objective (e.g., measurement data) or at least partially subjective (e.g., human observation based on training or experience).

[0031] Optionally, in some embodiments, the results of the evaluation may be used during the ongoing execution of the procedure to adjust, modify, update, or otherwise recalculate subsequent steps of the treatment plan, as indicated by a feedback loop between process 106 and process 102. Thus, in-procedure evaluation can enable the treatment plan to respond dynamically and continuously to observations, sensor data, and other inputs regarding the patient and environment during the procedure. For example, during the patient port establishment process, the actual placement of the ports may differ from the port placement recommended in the treatment plan. These modifications may be determined based on, for example, the patient's anatomical structure, surgical experience, staff experience, or other considerations, whether or not they were included in the original treatment plan determination. These modified port locations may be used to modify subsequent steps of the treatment plan as the procedure is being performed.

[0032] As shown in Figure 1, the evaluation results in process 106 may be passed to the previous treatment information input 124 to improve the quality of the input in the generation of the subsequent treatment plan. Both suboptimal and model results from the evaluation of the performed treatment can provide useful information for improving the subsequent treatment. For example, a second treatment plan and a second set of treatment inputs based on the stored treatment evaluation information may be generated in the execution after process 102. In an optional process 108, the evaluation results may be provided to the surgeon, staff, or facility as performance feedback for training and professional development. In some embodiments, the performance feedback may be provided during the execution of the treatment.

[0033] Method 100 in Figure 1 shows a process that may be used to generate any of the various procedure plans that may be used for a complete medical procedure such as a colectomy, or a process that may be used for a part of a complete procedure that may be performed, for example, by different teams of staff or surgeons. Figure 3 is a flowchart of Method 300 for generating a part of a procedure, i.e., the robot-assisted manipulator, peripheral medical components, and patient setup at the start of a medical procedure. In process 302, the procedure type to be performed in the robot-assisted medical system may be entered in the user interface (e.g., user interface 204). As previously stated, the procedure type may be provided via selection from a menu or other selection techniques. The procedure type may include a variety of surgeries such as general surgery, colorectal surgery, gynecological surgery, urological surgery, thoracic surgery, cardiac surgery, head and neck surgery, or it may include procedures for a variety of diagnoses or investigations. In process 304, patient information (e.g., patient information 120) may be received. In an optional process 306, staff information (e.g., staff information 118) may be received, and in an optional process 308, surgeon information (e.g., surgeon information 114) of the surgeon operating the robot-assisted manipulator may be received.

[0034] In process 310, previous procedure information may be analyzed or accessed depending on the procedure type and information about the patient, staff, and / or surgeon. For example, previous procedure information for the same type of procedure with surgeons and staff of similar experience levels and patients with similarly located target tissues (e.g., tumors) may generate recommendations for the optimal setup of a robot-assisted medical system to achieve the most effective, most efficient, or safest medical procedure. Over time, analysis, including machine learning based on previous procedure information, may generate customized recommendations for optimal procedure setup based on a specific set of inputs. For example, the analysis may include identifying previous procedures of a model from previous procedure information based on common inputs such as surgeon, patient characteristics, and staff training levels. In some examples, the analysis may include combining information from multiple previous procedures stored as previous procedure information. In some examples, the analysis may include analysis of performance metrics, including kinematic scores based on kinematic information from a robot-assisted manipulator generated during the previous procedure. In some examples, the analysis may include analysis of the elapsed time of a previous procedure or a subunit of a previous procedure. In some cases, the analysis may include an analysis of quality indicators, such as the quality of staff intervention in previous treatments or the quality of patient outcomes from previous treatments.

[0035] In process 312, setup instructions may be generated based on an analysis of previous procedure information. These setup instructions may be provided, for example, on a display device 200, and may be used to train or instruct medical staff on the optimal configuration of the robot-assisted system, peripheral components, and patient in the medical environment. As described in method 100 above, the execution of the setup plan may be evaluated against the generated plan to provide feedback to staff on why errors occurred during execution, why there were delays, or why the patient's prognosis was poor. This feedback may provide individualized training tailored to specific surgeons, procedure types, and patient types.

[0036] Optionally, setup instructions may be displayed on a display device (e.g., display device 200). The displayed setup instructions may include an augmented reality image containing a live image of the medical environment combined with at least one virtual image of a component (e.g., manipulator, instrument, peripheral component). In some examples, the displayed setup instructions may include a video demonstration from a previously recorded procedure.

[0037] Any of the treatment plans, including the aforementioned setup plan, can be executed in a medical environment using a robot-assisted medical system. Figure 4 shows a robot-assisted medical system 402, which may include components such as a robot-assisted manipulator assembly 404, an operator interface system 406, and a control system 408, in a medical environment frame of reference (X M , Y M , Z M An example of a medical environment 400 having ) is shown. In one or more embodiments, system 402 may be a robot-assisted medical system under the remote control of a surgeon. In an alternative embodiment, medical system 402 may be under the partial control of a computer programmed to perform medical procedures or sub-procedures. In yet another alternative embodiment, medical system 402 may be a fully automated medical system under the complete control of a computer programmed to perform medical procedures or sub-procedures in medical system 402. An example of medical system 402 that may be used to implement the systems and technologies described herein is the da Vinci® Surgical System manufactured by Intuitive Surgical Operations, Inc. of Sunnyvale, California. The medical environment 400 may be an operating room, a surgical suite, a medical treatment room, or any other environment in which medical procedures or medical training are performed.

[0038] The control system 408 may include a processing unit comprising at least one memory 410 and at least one processor 412 for communication, control, and data transfer between components in the medical environment. The control system 408 may employ a wide variety of centralized or distributed data processing architectures. Similarly, programmed instructions may be implemented as several separate programs or subroutines, or they may be integrated into several other embodiments of the system described herein, including a remote control system. In one embodiment, the control system 408 may support any of a variety of wired communication protocols or wireless communication protocols such as Bluetooth®, IrDA, HomeRF, IEEE 802.11, DECT, and wireless telemetry. In some embodiments, the control system 408 may be in a different environment, partially or completely separated from the manipulator assembly 404 and operator interface system 406, including different areas of a common surgical environment, different rooms, or different buildings.

[0039] The manipulator assembly 404 is sometimes referred to as the patient-side cart. One or more medical instruments 414 (also called tools) may be operably coupled to the manipulator assembly 404. The medical instruments 414 may include end effectors having a single working member, such as a scalpel, blunt blade, needle, imaging sensor, optical fiber, or electrode. Other end effectors may include multiple working members, examples of which include forceps, grippers, scissors, clip applicators, staplers, and bipolar electrosurgical instruments. The number of medical instruments 414 used at one time generally depends, among other factors, on the medical procedure and the space constraints in the operating room. The medical instruments 414 may also include imaging devices. The imaging devices may include endoscopic imaging systems using optical imaging techniques, or other types of imaging systems using other techniques (e.g., ultrasound, fluoroscopy, etc.). The manipulator assembly 404 may include a kinematic structure of one or more links coupled by one or more non-servo-controlled joints, and a servo-controlled robotic manipulator. In various implementations, non-servo-controlled joints can be manually positioned or locked to allow or restrict relative movement between links physically coupled to the non-servo-controlled joint. The manipulator assembly 404 may include multiple motors that drive inputs to the medical instrument 414. These motors may move in response to commands from the control system 408. The motors may include a drive system that, when coupled to the medical instrument 414, can advance the medical instrument into a naturally or surgically created anatomical opening in the patient. Other motor drive systems may move the distal end of the medical instrument with multiple degrees of freedom, which may include three linear motions (e.g., linear motion along the X, Y, and Z Cartesian axes) and three rotational motions (e.g., rotation around the X, Y, and Z Cartesian axes). Furthermore, motors can be used to actuate articulable end effectors of instruments for grasping tissue in the jaw, such as biopsy devices.Kinematic information relating to the manipulator assembly 404 and / or device 414 may include structural information such as the dimensions of the components of the manipulator assembly and / or medical device, the arrangement of joints, the positional information of components, the orientation information of components, and / or the arrangement of ports. The kinematic information may also include dynamic kinematic information such as the range of motion, velocity or acceleration information of joints in the remotely operated assembly, and / or resistance force. Structural or dynamic kinematic constraint information may be generated by sensors in the remotely operated assembly that measure, for example, the configuration of the manipulator arm, the configuration of the medical device, the configuration of joints, the displacement of components, the velocity of components, and / or the acceleration of components. Sensors may include position sensors such as electromagnetic (EM) sensors, shape sensors such as optical fiber sensors, and / or actuator position sensors such as resolvers, encoders, and potentiometers.

[0040] The operator interface system 406 allows an operator, such as a surgeon or other type of clinician, to view an image of the treatment site or an image representing the treatment site and to control the operation of the medical instrument 414. In some embodiments, the operator interface system 406 may be located in the same room as the patient during the surgical procedure. However, in other embodiments, the operator interface system 406 may be located in a separate room or even in a separate building from the patient. The operator interface system 406 may generally include one or more control devices for controlling the medical instrument 414. The control device(s) may include one or more of any number of different input devices, such as a handgrip, joystick, trackball, data glove, trigger gun, foot pedal, hand-operated controller, voice recognition device, touchscreen, body movement or presence sensor. In some embodiments, to provide the operator with telepresence, the control device(s) may have the same degrees of freedom as the medical tool of a robotic assembly; that is, the operator is given the perception that the control device(s) are integrated with the tool, so that the operator can have the feeling of directly controlling the tool as if it were present at the treatment site. In other embodiments, the control device(s) may have more or fewer degrees of freedom than the associated medical tool and may still provide telepresence to the operator. In some embodiments, the control device(s) may be a manual input device that moves with six degrees of freedom, which may also include an operable handle for operating the medical tool (e.g., for closing a gripping jaw end effector, applying potential to an electrode, capturing an image, administering drug therapy, etc.). While the operator views the treatment site through a display on the operator interface system 406, the manipulator assembly 404 may support and operate the medical instrument 414. Images of the treatment site may be obtained by an imaging instrument such as a monoscopic or stereoscopic endoscope that can be operated by the manipulator assembly 404.

[0041] Optionally, another component that may be placed in the medical environment 400 is a display system 416 that can be communicatively coupled to a control system 408. The display system 416 may, for example, display images, commands, and data for performing robot-assisted procedures. The information presented on the display system 416 may include endoscopic images from within the patient's anatomical structure, guidance information, patient information, and procedure plan information. In some embodiments, the display system may be supported by an electronic cart that enables the movable positioning of the display system. In some embodiments, the display system may be a display device 200. Multiple display systems may be present in the medical environment 400.

[0042] The input source 418 may be communicatively coupled to the control system 408 or stored in memory 410. The input source 418 may store one or more of the inputs 110 and / or may be a user interface for receiving one or more of the inputs 110. In some embodiments, the input source 418 may be a database stored outside the medical environment 400 and accessed by the control system 408. In some embodiments, the display system 416 and the input source 418 may be a common device.

[0043] Other components in the medical environment 400, which may or may not be communicatively coupled to the control system 408, may include a patient table 420 and auxiliary components 422 such as instrument tables, instrument washbasins, anesthesia carts, supply carts, cabinets, and seats. Other components in the medical environment 400, which may or may not be communicatively coupled to the control system 408, may include utility ports 424 such as electrical, water, and pressurized air outlets.

[0044] Persons in or able to enter the medical environment 400 may include a patient 426 who may be positioned on the patient table 420, a surgeon 428 who may have access to the operator interface system 406, and staff members 430, which may include, for example, surgical staff or maintenance staff.

[0045] Elements described in detail with reference to a particular embodiment, implementation, or application may, whenever feasible, be included in other embodiments, implementations, or applications that are not specifically illustrated or described. For example, if an element is described in detail with reference to one embodiment but not with reference to a second embodiment, that element may be included in the second embodiment and described in the claims. Accordingly, to avoid unnecessary repetition in the following description, one or more elements illustrated and described in relation to one embodiment, implementation, or application may be incorporated into other embodiments, implementations, or aspects, unless otherwise specifically described, provided that one or more elements do not render the embodiment or implementation non-functional or that two or more elements provide conflicting functions.

[0046] Changes and further modifications to the apparatus, systems, fixtures, and methods described herein, as well as further applications of the principles of this disclosure, are given due consideration to those who are skilled in the art to whom the disclosure is relevant. In particular, features, components, and / or steps described in relation to one embodiment are given due consideration to be able to be combined with features, components, and / or steps described in relation to other embodiments of this disclosure. Furthermore, the dimensions provided herein are for specific examples and are given due consideration to be able to utilize different sizes, dimensions, and / or proportions to implement the concepts of this disclosure. To avoid unnecessary repetition of descriptions, one or more components or operations described according to one exemplary embodiment may be used or omitted from other exemplary embodiments as appropriate. For brevity, numerous repetitions of these combinations will not be described individually.

[0047] Various systems and parts of systems are described in terms of their state in three-dimensional space. Herein, the term "position" refers to the location of an object or part of an object in three-dimensional space (e.g., three translational degrees of freedom along the Cartesian X, Y, Z coordinates). Herein, the term "orientation" refers to the rotational orientation of an object or part of an object (e.g., three rotational degrees of freedom (e.g., roll, pitch, yaw)). Herein, the term "attitude" refers to the position of an object or part of an object in at least one translational degree of freedom and the orientation of that object or part of an object in at least one rotational degree of freedom (up to six total degrees of freedom).

[0048] Some of the examples described herein refer to surgical procedures or instruments, or medical procedures and medical instruments, but the disclosed technologies optionally apply to non-medical procedures and non-medical instruments. For example, the instruments, systems, and methods described herein may be used for industrial applications, general robotic applications, and non-medical purposes including sensing or manipulating non-tissue workpieces. Other application examples include cosmetic enhancement, imaging of human or animal anatomical structures, data acquisition from human or animal anatomical structures, and training of medical or non-medical personnel. Other application examples include use for procedures on tissues extracted from human or animal anatomical structures (without returning them to human or animal anatomical structures), and performing procedures on human or animal carcasses. Furthermore, these technologies may also be used for surgical and non-surgical therapeutic or diagnostic procedures.

[0049] A computer is a machine that, in accordance with programmed instructions, performs mathematical or logical functions on input information and produces processed output information. A computer includes logical units that perform mathematical or logical functions, and memory that stores programmed instructions, input information, and output information. The term "computer" is similar to similar terms such as "processor," "controller," or "control system."

[0050] While certain exemplary embodiments of the present invention are described and illustrated in the accompanying drawings, it should be understood that such embodiments merely illustrate and do not limit the broader invention, and that embodiments of the present invention are not limited to the specific structures and arrangements illustrated and described, as various other modifications may be conceived by those skilled in the art.

Claims

1. It is a system: Processor; and A memory containing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the system: A robot-assisted manipulator generates a procedure plan for performing a procedure, and the procedure plan is based on a first set of procedure inputs; A performance index is generated from the execution of the aforementioned procedure, the performance index being a score based on kinematic information from the robot-assisted manipulator during the performed procedure; In order to generate treatment evaluation information, the performed treatment is evaluated based on the performance indicators; The aforementioned treatment evaluation information is stored; Based on the stored treatment evaluation information and a second set of treatment inputs, a second treatment plan is generated. system.

2. It is a system: Processor; and A memory containing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the system: A robot-assisted manipulator generates a procedure plan for performing a procedure, and the procedure plan is based on a first set of procedure inputs; A performance indicator is generated from the execution of the aforementioned procedure, the performance indicator being the amount of staff intervention during the performed procedure; In order to generate treatment evaluation information, the performed treatment is evaluated based on the performance indicators; The aforementioned treatment evaluation information is stored; Based on the stored treatment evaluation information and a second set of treatment inputs, a second treatment plan is generated. system.

3. The first plurality of treatment inputs include the treatment type of the treatment, The system according to claim 1 or 2.

4. The first set of procedure inputs includes surgeon information, The system according to claim 1 or 2.

5. The first set of multiple treatment inputs includes previous treatment information, The system according to claim 1 or 2.

6. The environment of the robot-assisted manipulator further includes displaying the treatment plan on a display device. The system according to claim 1 or 2.

7. The displayed procedure plan includes an augmented reality image which includes a live image of the environment of the robot-assisted manipulator and a virtual image of at least one component within the environment. The system according to claim 6.

8. The displayed treatment plan includes a video demonstration of at least a portion of a previously recorded treatment. The system according to claim 6.

9. The aforementioned procedure includes a set of setup instructions. The system according to claim 1 or 2.

10. The performance indicator is the elapsed time of at least one component process of the executed procedure. The system according to claim 1 or 2.

11. The aforementioned performance indicator is a prognostic quality indicator. The system according to claim 1 or 2.

12. Evaluating the measures taken includes comparing the performance indicators with benchmark indicators and identifying optimal results. The system according to claim 1 or 2.

13. Further comprising evaluating the actions taken and then generating a revised action plan, The system according to claim 1 or 2.

14. The second action plan includes a set of setup instructions for the second action, The system according to claim 1 or 2.

Images