SYSTEM AND METHOD FOR GENERATING CUSTOMIZED MEDICAL SIMULATIONS - Patent application

Abstract

The medical system may include a display system, an operator input device, and a control system in communication with the display system and the operator input device. The control system may include a processor and a memory, the memory including machine-readable instructions that, when executed by the processor, cause the control system to access a user's experience factors and to reference a set of parameterized prior actions. The instructions may also cause the control system to identify a parameterized prior action associated with the experience factors from the set of parameterized prior actions, and to generate a simulation exercise including a plurality of parameters from the parameterized prior action. A model input to the operator input device associated with the plurality of parameters may be determined.

Description

[Technical field]

[0001] [Cross-reference application] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 320,553, entitled "Systems and Methods for Generating Customized Medical Simulations," filed March 16, 2022, which is incorporated by reference in its entirety.

[0002] [Technical field] The present disclosure is directed to systems and methods for generating customized medical simulations. [Background technology]

[0003] Minimally invasive medical techniques are intended to reduce the amount of tissue damaged during diagnostic or surgical procedures, thereby reducing patient recovery time, discomfort, and adverse side effects. Such minimally invasive techniques may be performed through one or more surgical incisions or through natural openings in the patient's anatomy. Through these incisions or natural openings, clinicians may insert minimally invasive medical instruments and perform medical procedures by manual or robotically assisted instrument actuation. Customized medical simulations may be developed to improve medical procedures, train clinicians, and / or evaluate the effectiveness of medical procedures. Summary of the Invention

[0004] Examples of the invention are summarized by the claims that follow this specification. Consistent with some examples, a medical system may include a display system, an operator input device, and a control system in communication with the display system and the operator input device. The control system may include a processor and a memory, and the memory includes machine-readable instructions that, when executed by the processor, cause the control system to access a user's experience factor and to reference a set of parameterized previous actions. The instructions may also cause the control system to identify a parameterized previous action associated with the experience factor from the set of parameterized previous actions, and generate a simulation exercise including a plurality of parameters from the parameterized previous action. A model input to the operator input device associated with the plurality of parameters may be determined.

[0005] Consistent with certain examples, a method for generating a customized medical simulation exercise may include accessing experience factors of a user, referencing a set of parameterized previous actions, and identifying a parameterized previous action from the set of parameterized previous actions associated with the experience factors. The method may also include generating a simulation exercise including a plurality of parameters from the parameterized previous actions, and determining model inputs to an operator input device associated with the plurality of parameters.

[0006] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory in nature and are intended to provide an understanding of the present disclosure without limiting the scope of the disclosure. In that regard, additional aspects, features, and advantages of the present disclosure will become apparent to those skilled in the art from the following detailed description. [Brief description of the drawings]

[0007] [Figure 1]1 is a method for customizing a simulation exercise according to some embodiments. [Diagram 2] 1 illustrates a user profile, according to some embodiments. [Diagram 3] FIG. 1 is a schematic diagram of a parameterized medical procedure record, according to some embodiments. [Figure 4] 1 is a flow chart illustrating a measurement and recording system for parameterizing a medical procedure. [Figure 5A] 1 is a method for generating a customized simulation exercise, according to some embodiments. [Figure 5B] 1 is a method for generating a customized simulation exercise, according to some embodiments. [Figure 5C] 1 is a method for generating a customized simulation exercise, according to some embodiments. [Figure 6] FIG. 1 is a schematic diagram of a robotic-assisted medical system, according to some embodiments. [Figure 7] FIG. 1 is a perspective view of a manipulator assembly according to some embodiments. [Figure 8] FIG. 1 illustrates a perspective view of an operator input system according to some embodiments.

[0008] The embodiments of the present disclosure and their advantages are best understood by referring to the following detailed description: It should be understood that for purposes of illustrating, but not limiting, embodiments of the present disclosure, like reference numerals are used to identify like elements shown in one or more of the figures. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The medical skill development system may identify a user's skill development needs and generate a customized simulation syllabus including one or more simulation exercises built from previous procedure data. The simulation exercises may enhance the user's skill capabilities. A system and method for generating customized medical procedure simulations is provided. A clinician profile and parameterized previous procedures may be used to create simulation exercises that address the clinician's development needs.

[0010] FIG. 1 illustrates a method 100 for generating a simulation exercise, according to some embodiments. The method 100 is illustrated as a set of operations or processes. The processes illustrated in FIG. 1 may be performed in an order different from that illustrated in FIG. 1, and one or more of the illustrated processes may not be performed in some embodiments of the method 100. Additionally, one or more processes not explicitly illustrated in FIG. 1 may be included before, after, between, or as part of the illustrated processes. In some embodiments, one or more of the processes of the method 100 may be implemented, at least in part, in the form of executable code stored on a non-transitory, tangible, machine-readable medium that, when executed by one or more processors (e.g., a processor of a control system), may cause the one or more processors to perform one or more of the processes. In process 102, user information may be received. In process 104, a catalog, database, or set including one or more parameterized previous actions may be referenced. In process 106, a simulation exercise may be determined based on the user information and the referenced parameterized previous actions.

[0011] More specifically, in process 102, user information may be received from a user profile. User experience factors may be determined from the user profile and may include skill development subjects for the user to improve clinical skills or may include the type of future procedure scheduled for the user so that the user can prepare for future experiences. For example, the user information may be received at a processor (e.g., processor 620 of medical system 610). In some examples, medical procedure records may be received from a memory device (e.g., memory 624 of medical system 610). FIG. 2 shows an example of a user profile 150 including previous procedure information 152 and future or anticipated procedure information 154. The user may be, for example, a surgeon, a medical equipment operator, a surgical assistant, or other clinician or operator involved in a medical procedure. In some examples, the user may be a synthetic user with artificial intelligence that may be trained using the methods and systems described herein. The previous procedure information 152 may include, for example, information about a patient involved in a previous procedure performed by the user. The patient information or characteristics may include data from the patient's electronic medical record, gender, height, weight, body mass index, medical imaging data (e.g., CT images, ultrasound images), medical measurement data (e.g., blood pressure, EKG results), and / or other information about patients involved in the user's previous procedures. The previous procedure information 152 may additionally or alternatively include, for example, team characteristics for previous medical procedures performed by the user. The team characteristics may include, for example, the roles of the team members involved in the previous procedures, the experience level of each team member involved in the previous procedures, and each team member's previous experience in collaborating with the user. The previous procedure information 152 may additionally or alternatively include, for example, information about a surgical system or instrument used in a previous medical procedure performed by the user. The surgical system or instrument may include, for example, a robotic assisted medical system, a laparoscopic instrument, an open surgical instrument, or any combination thereof. The previous procedure information 152 may additionally or alternatively include, for example, procedure type information for previous medical procedures performed by the user.The types of procedures may include, for example, abdominal, cardiac, colorectal, gynecological, head / neck, pulmonary, thoracic, and / or urological procedures. The previous procedure information 152 may additionally or alternatively include, for example, segment information regarding previous medical procedures performed by the user. The segment information may include, for example, difficulty information regarding the previous procedures or segments of the previous procedures. The previous procedure information 152 may additionally or alternatively include, for example, a user skill profile including evaluation information from previous medical procedures performed by the user. The evaluation information may include a performance evaluation based on objective performance indicators. The evaluation information may include an evaluation of technical aspects of the previous procedures including information regarding errors, inefficiencies, suboptimal patient outcomes, or other positive or negative indicia of the user's technical proficiency. The evaluation information may additionally or alternatively include an evaluation of non-technical aspects of the previous procedures including information regarding communication effectiveness, leadership, stress, or other positive or negative indicia of the user's non-technical proficiency. The previous treatment information may be generated from treatments performed on the patient, for example, during training sessions using synthetic tissue structures or during previous computer-generated simulations.

[0012] Future procedure information 154 may include information regarding future procedures that are to be performed by the user, scheduled, planned, or otherwise known or expected. For each future procedure, various information may be received including, for example, patient information, team composition information, surgical system information regarding systems and devices to be used, procedure type, procedure segment information, and / or expected difficulty level.

[0013] Referring again to FIG. 1 , in process 104, a parameterized prior procedure catalog, database, or other information set may be referenced or queried. During a medical procedure, a sensor system, a measurement system, and / or a recording system may capture and store data records in a medical procedure record. In some examples, a medical procedure record may capture information about a medical procedure performed with a robotic-assisted medical system, a laparoscopic medical system, a manual medical instrument, or a combination of systems and devices. Each medical procedure record may capture information about a medical procedure performed on a patient by a single clinician or a team of medical professionals. Medical procedures may be parameterized as described in U.S. Provisional Patent Application No. 63 / 320,538, filed March 16, 2022, entitled “Systems and Methods for Parameterizing Medical Procedures,” which is incorporated herein by reference in its entirety. Parameters may be associated with characteristic actions of a medical procedure and may be based on data records generated during previous medical procedures. A catalog, database, or set of parameterized procedures may include parameterized procedures associated with a particular clinician, parameterized procedures from a guide, trainer, or other expert, parameterized procedures from other users (e.g., peers, trainees), and / or parameterized procedures from synthetic procedures performed on virtual or synthetic patients.

[0014] FIG. 3 is a schematic diagram of a parameterized medical procedure record 200 that may be stored, for example, in a set of parameterized previous procedures. The parameterized medical procedure record 200 may include procedure information 202 and data records A-G from a medical procedure associated with the procedure information 202. The procedure information 202 may include various input parameters for the procedure. For example, operator information 210 may include clinician identification information, training history, experience, preferences, and / or other information regarding one or more clinicians involved in the medical procedure. An operator may be, for example, a surgeon, a surgical resident, a specialist, or a teaching physician. The procedure information 202 may include patient characteristics 212, including the patient's age, sex, height, weight, medical history, and / or other information regarding the unique patient on whom the medical procedure was performed. The procedure information may include team characteristics 214 of team members and support staff involved in the medical procedure, including identification information, role in the procedure, training history, experience, preferences, or other information regarding the personnel performing the procedure. The procedure information 202 may also include system information 216 regarding one or more systems used to perform the procedure. The system may include sensors or recording systems that capture information such as system settings or sensed parameters during a procedure. The system may include a robotic assisted medical system or components thereof, a laparoscopic medical system, a manual medical instrument, and / or an assistive or peripheral system. The system information 216 may include a manufacturer, model, serial number, time in service, time since last maintenance, usage cycle count, maintenance history, calibration information, or other information regarding the system used to perform the medical procedure. The procedure information 202 may also include a procedure type 218 that includes information regarding the type of medical procedure. The medical procedure type may include abdominal, cardiac, colorectal, gynecological, neurological, head / neck, pulmonary, thoracic, urological, or other categories or subcategories of anatomical systems involved in the medical procedure. The procedure information 202 may also include segment information 220. The segment information 220 may include information regarding the subdivided parts of the medical procedure.For example, a procedure segment may include a sequence or group of actions associated with ablation, stapling, suturing, incision, tissue resection, anastomosis, camera control, instrument list control, setting change, or tool change performed one or more times during a medical procedure. In some examples, the procedure information may also include other information regarding the procedure, including, for example, the date the procedure was performed, the time and duration of the procedure, and / or the location and facility identification where the procedure was performed.

[0015] The medical procedure record 200 also includes data records A-G captured during the medical procedure. In some examples, as shown in FIG. 4, the medical procedure 300 may be documented using various measurement and recording systems 400 to generate data records 402 (e.g., data records A-G) associated with the procedure 300 (e.g., previous medical procedures). The measurement and recording system 400 may include, for example, one or more sensor systems 410 associated with a robotic-assisted medical system (e.g., medical system 610 of FIG. 6). In some examples, the sensor system 410 may include a position, orientation, motion, and / or displacement sensor system of a manipulator or an instrument coupled to a manipulator of a robotic-assisted medical system. In some examples, the sensor system 410 may include a force sensor system, a clock, a motor encoder, an energy usage sensor, a user eye-tracking sensor, or other sensor system that measures and / or records data regarding the manipulator or instrument. The measurement and recording system 400 may additionally or alternatively include one or more sensor systems 412 associated with monitored medical equipment used in the medical procedure 300. In some examples, the sensor system 412 may track position, orientation, movement, displacement, force, energy usage, duration of use, and / or other measurements associated with the medical device.

[0016] The measurement and recording system 400 may additionally or alternatively include one or more imaging systems 414 used during the medical procedure 300. In some examples, the imaging system 414 may be an in vivo imaging system, such as an endoscopic imaging system or an ultrasound imaging system used during the procedure 300. In some examples, the imaging system 414 may be an ex vivo imaging system for the patient's anatomy, such as a computed tomography (CT) imaging system, a magnetic resonance imaging (MRI) imaging system, or a functional near-infrared spectroscopy (fNIRS) imaging system used during the procedure 300. In some examples, the imaging system 414 may be an environmental imaging system, such as an optical imaging system that tracks the position and movement of manipulators, instruments, equipment, and / or personnel within the patient's environment during the procedure 300.

[0017] The measurement and recording system 400 may additionally or alternatively include one or more audio systems 416 used during the medical procedure 300. The audio systems may capture and record audio from personnel in the medical area of ​​the procedure 300, the operator performing the procedure 300, the patient, and / or equipment in the medical area of ​​the procedure 300. The measurement and recording system 400 may additionally or alternatively include one or more in-procedure patient monitoring systems 418 used during the medical procedure 300. The patient monitoring systems 418 may include, for example, respiratory, cardiac, blood pressure, anesthesia, insufflation, and / or patient / table orientation monitoring systems. The measurement and recording system 400 may additionally or alternatively include one or more patient outcome recording systems 420 that may be referenced after the procedure 300 is completed. The patient outcome recording systems 420 may record information regarding length of stay after the procedure, complications, positive outcomes, negative outcomes, mortality, or other post-procedure information regarding the patient. The measurement and recording system 400 may additionally or alternatively include one or more procedure skill recording systems 422 that capture and record objective performance indicators of the clinician performing the procedure 300 .

[0018] Data records 402 may include data generated by measurement and recording system 400. For example, data records 430 may record the position, orientation, movement, and / or displacement of an instrument (e.g., instrument 614) controlled by a robotically assisted manipulator or by manual manipulation. In some examples, data records 432 may record the position, orientation, movement, and / or displacement of a robotically assisted manipulator assembly (e.g., 612), including any arm of the manipulator, during procedure 300. In some examples, data records 434 may record the position, orientation, movement, and / or displacement of an imaging system, such as an endoscope or other in vivo or ex vivo imaging system, during procedure 300. In some examples, data records 436 may record the position, orientation, movement, and / or displacement of an operator input device (e.g., 636) during procedure 300. In some examples, data records 438 may record the position, orientation, movement, and / or displacement of an operator (e.g., surgeon S) directing control of an instrument during procedure 300. For example, data records 438 may record the operator's hand movements or track head disengagement from the operator console. In some examples, data records 440 may record the position, orientation, movement, and / or displacement of one or more members of the medical team involved in the procedure 300. In some examples, data records 442 may record aspects of the initial setup of the procedure 300, including the location and placement of the robot-assisted manipulator assembly, patient port placement, and location of peripheral equipment. In some examples, data records 444 may include records of the location, frequency, and amount of energy supplied to or delivered by an instrument (e.g., an ablation instrument) during the procedure 300. In some examples, data records 446 may include records of instrument changes during the procedure 300. In some examples, data records 448 may include time-based records capturing dwell times, idle times, and / or durations or speeds of actions during the procedure 300. In some examples, data records 450 may capture aspects of the workflow, including the amount and / or sequence of actions during the procedure 300.For example, data records 450 may include sequences of positions, orientations, movements, and / or displacements associated with discrete activities. In some examples, data records 452 may capture errors, difficulties, accidents, or other unplanned episodes during procedure 300, such as a collision of manipulator arms, a condition leading to a conversion during procedure from robotic-assisted surgery to open surgery, a condition leading to a conversion during procedure from robotic-assisted surgery to laparoscopic surgery, or a condition leading to a conversion during procedure from laparoscopic surgery to open surgery. In some examples, data records 454 may capture aspects of the anatomical environment, including the size of organs, incisions, and / or treatment delivery areas. Other aspects of the anatomical environment that may be recorded include pelvic width, distances between anatomical structures, and / or location of the vasculature. In some examples, data records 456 may include intervention outcomes, such as measurements of bleeding, smoke, tissue movement, and / or tissue color changes. In some examples, the data records 458 may include a catalog of key skills for performing the procedure 300, associated objective performance indicators for experienced clinicians performing the same type of procedure, and objective performance indicators of the clinician who performed the procedure 300.

[0019] Referring again to FIG. 3, in the parameterized medical procedure record 200, data records A-G (e.g., data records 430-456) are associated with actions 244, 246, 248 performed during the medical procedure. The actions are associated with parameters 250, 252, 254, 256, for example, through the parameterization method described in U.S. Provisional Patent Application No. 63 / 320,538. In this example, the actions and parameters are optionally further grouped into segments 240, 242. For example, a procedure segment may include a sequence or group of actions associated with ablation, stapling, suturing, incision, tissue resection, anastomosis, camera control, instrument list control, setting change, or tool change performed one or more times during the medical procedure. In this example, the procedure may include segment 240 and segment 242. Segment 240 may include actions 244 and 246. Action 244 may include two parameters 250 and parameter 252. Parameter 250 is or is determined from a single data record A. Parameter 252 is or is determined from three data records B-D. Action 246 may include a single parameter 254 that is or is determined from data record E. Segment 242 includes a single action 248, which includes a single parameter 256. Parameter 256 is or is determined from data record F and data record G.

[0020] As an example, procedure record 200 may include segment 240, which is a tissue resection segment, and segment 242, which is an ablation segment. Suturing segment 240 may include an action to cut tissue in action 244, and an action to move the cut tissue in action 246. Cutting action 244 may include parameters 250, which includes a data record A that includes identification information for the cutting instrument. Cutting action 244 may include parameters 252, which includes a data record B that includes a position and orientation of the end effector of the cutting instrument at the start of the cut, a data record C that includes a position and orientation of the end effector of the cutting instrument at the end of the cut, and a data record D that includes a duration from the start to the end of the cut. Tissue moving action 246 may include parameters 254, which includes a data record E that includes a distance the tissue is moved. Ablation segment 242 includes a single action 248 to ablate tissue, which includes parameters 256 associated with a data record F for power level and a data record G for duration.

[0021] Referring again to FIG. 1, in process 106, a simulation exercise may be determined based on a user profile and a parameterized previous treatment catalog, database, or set. FIG. 5A provides an example of a method 500 for customizing a simulation treatment. A customized simulation treatment may, for example, allow a user to experience difficult cases previously performed by experts. The experience may be customized based on the user's experience to include guidance or other assistance. The experience may also be customized, scaled, or otherwise adapted to the user's schedule of upcoming treatments. In some examples, the experience may be customized to include improvement exercises to correct performance or to include increasingly difficult exercises to extend the user's capabilities based on the user's previous performance evaluations. The processes shown in FIG. 5A may be performed in a different order than the order shown in FIG. 5A, and one or more of the processes shown may not be performed in some embodiments of method 500. Additionally, one or more processes not explicitly shown in FIG. 5A may be included before, after, between, or as part of the processes shown. In some embodiments, one or more of the processes of method 500 may be implemented, at least in part, in the form of executable code stored on a non-transitory, tangible, machine-readable medium that, when executed by one or more processors (e.g., processors of a control system), may cause the one or more processors to perform one or more of the processes. Process 106 of FIG. 1 may be manual (where a clinician or other human operator(s) determines the simulation exercises), fully automated (where a control system determines the simulation exercises), or semi-automated (where some parts of process 106 are performed by a human operator and other parts are performed by a control system).

[0022] In process 502, user experience factors may be accessed or otherwise determined. The user experience factors may include skill development subjects for the user to improve clinical skills or may include the type of future procedure scheduled for the user so that the user can prepare for future experiences. Additionally or alternatively, the experience factors may include factors associated with past or upcoming procedures, such as procedure workflow step rankings or anatomical parameters such as patient size or condition. The experience factors may be accessed or determined based on information in a user profile, information received from the user or other operators (e.g., trainers), or any other source of information regarding skill development needs.

[0023] In one example, the user experience factors include one or more subjects or areas for skill development, and the experience factors may be determined from the user profile and / or parameterized previous procedure information. For example, the previous procedure information 152 from the user profile 150 may be compared to the objective performance indicators 458 of the procedure from one or more previous procedures 300 to assess the user's skill level and identify areas of skill development that are needed. For example, the user skill metrics may be compared to objective performance indicators or performance benchmarks to identify areas of weakness or opportunities for strengthening skills. For example, a benchmark or indicator for tissue ablation skills may include a measurement of the amount of smoke generated, a measurement of the amount of tissue color change, a measurement of the time to complete the procedure, and / or a measurement of the number of clamping actions performed. Comparing the user's skill metrics for these benchmark skills may indicate that the user should complete simulation training to improve ablation skills. As another example, the user skill metrics may be compared to skills needed for upcoming procedure types based on the user's schedule of upcoming or expected procedures. Comparing the user's skill metrics for the skills required for upcoming procedure types may indicate that the user should complete simulation training to acquire or improve the skills required for those types of upcoming procedures. As another example, the user skill metrics may be compared to the difficulty of upcoming procedures based on the user's schedule of the upcoming procedures. Comparing the user's skill metrics to the difficulty of upcoming procedures may indicate that the user should complete simulation training to acquire or improve the skills required for the upcoming difficulty level. As another example, the user patient history may be compared to the characteristics of patients scheduled for upcoming procedures.Comparing the patient history to the characteristics of patients scheduled for an upcoming procedure type may indicate that the user should complete simulation training to acquire or improve skills required to match the size, gender, medical condition type, comorbidities, or other characteristics of the scheduled patient. As another example, the user's team composition history may be compared to the team scheduled for the user's upcoming procedure. Comparing the user's team composition history to the team composition, skill level, and experience level for the upcoming procedure may indicate that the user should complete simulation training to acquire or improve skills required to work effectively with the planned team. As another example, the user's skill metrics may be evaluated to determine the next level of skill for incremental skill growth. As another example, the user may self-identify areas in need of growth or improvement based on the user's profile or based on parameterized previous procedures. As another example, a trainer, key operations leader, expert, or other instructor may identify areas for the user's growth or improvement based on the user's profile or based on parameterized previous procedures.

[0024] In process 504, one or more parameterized prior actions may be identified from the set of parameterized prior actions to address the identified skill development needs. For example, if the upcoming action involves a patient with an underlying condition that the user has not previously experienced, a parameterized prior action involving a real or simulated patient with the same underlying condition may be identified. As another example, if the upcoming action involves the use of a new robotic-assisted medical system that is different from the system the user previously used, a parameterized prior action using the new system may be identified. In some examples, the parameterized prior action may be an action of an expert, key operations leader, trainer, peer, or even the user that is associated with an efficient, minimally errored, or otherwise successful prior action. In some examples, the parameterized prior action may be a user or other's action that includes errors, suboptimal performance, inefficiencies, or other issues that may be investigated or retired to improve outcomes.

[0025] In optional process 506, aspects of the identified parameterized prior procedure may be modified. For example, segments, actions, or parameters from the identified prior procedure may be omitted or combined with segments, actions, or parameters from other identified prior procedures. In some examples, the identified parameterized prior procedure may be scaled based on the patient's dimensions, gender, age, or other conditions corresponding to the user's skill development needs. In some examples, the identified parameterized prior procedure may be modified to allow additional team members to participate in the simulation. In some examples, the identified parameterized prior procedure may be modified to include synthetic team members. In some examples, the port placement used in the parameterized prior robotic-assisted medical procedure may be modified. In some examples, the identified parameterized prior procedure may be modified to include a different system, device, or instrument.

[0026] In process 508, a simulation exercise may be generated. The simulation exercise may allow a user to virtually experience a parameterized previous procedure, including a simulated user interface that provides a visual, audio, and / or haptic simulation of the identified previous procedure. The simulation exercise may include some or all of the parameters from the parameterized previous procedure based on data recordings acquired during the previous procedure. The simulation exercise may simulate the robotic assisted medical system and instruments, laparoscopic instruments, and / or open surgical instruments used in the identified previous procedure. Image data, audio data, force transmission data, patient monitoring data, and / or patient outcome data recorded or collected from the identified previous procedure and parameterized may be included in the simulation exercise. In some examples, image data, audio data, force transmission data, patient monitoring data, and / or patient outcome data may be artificially generated and included in the simulation exercise to create a synthetic or hybrid synthetic recording environment and experience. The simulation exercise may be interactive and responsive to user input. In some examples, the simulation exercise may be presented to the user on a simulated user console that includes user interface components of an operator input system (e.g., operator input system 616) including a display system, an audio system, and user input control devices. In some examples, the simulation exercise may be presented to the user on an actual user console (e.g., operator input system 616) operating in a simulation mode. In some examples, the simulation exercise may be adapted for presentation to the user on a laptop, tablet, phone, or other user input device that may include a display, user input control devices, a control system, memory, and / or other components that support a visual, audio, and / or tactile user experience. In some examples, the simulation may be dynamically adapted based on the user's preferences.For example, after a simulation is initiated with a first instrument used by a first supervising clinician in a first previous procedure, the user may select to change the simulation to use a second instrument used by a second supervising clinician in a second previous procedure. In some examples, the simulation may include an inanimate anatomical model or a synthetic tissue model that is customized and constructed for the simulation. For example, the synthetic model may include a custom anatomical defect or customized instrument port placement for the defect.

[0027] In optional process 510, model operator inputs for performing the simulation exercise may be determined from the parameterized previous procedure. The model operator inputs may include hand, arm, body, eye, head, foot, or other movements or behaviors used to generate or otherwise associated with the data records on which the parameters of the simulation exercise are based. For example, parameters associated with the action of suturing tissue in a previous procedure were determined from data records or procedure information records associated with a sequence of steps in performing the act of suturing. Thus, model operator inputs associated with suturing based on parameters of the previous procedure may include selecting an instrument having the same identity used to grasp the suture filament, selecting the same manipulator arm used to control the instrument, applying the same or similar force to grasp the filament, rotating the wrist joint of the instrument the same or similar amount, completing the rotation in the same or similar duration, and releasing the filament at the same or similar position and orientation of the instrument. The model operator inputs may generate data records that are the same as or within a predetermined range of the corresponding data records for the previous procedure. In some examples, the user's inputs in the simulation may be evaluated or compared against the model operator inputs to provide guidance, error reports, success confirmation reports, or other indications that the user's inputs are the same as, similar to, or different from the model operator inputs.

[0028] In optional process 512, guidance for executing the model inputs may be generated. For example, the guidance may include simulated graphics for visually displaying to the user during the simulation exercise to explain or demonstrate the model inputs. The simulated graphics may include ghost tool illustrations. The guidance may also include haptic forces or audio guidance delivered to the operator input device. The guidance may include hand-over-hand demonstrations that allow the user to follow guided hand movements. The guidance may include pop-up graphics, textual information, and / or highlighted or emphasized graphics. The guidance may include graphical indicators, picture-in-picture displays, and / or synthesized or recorded videos of alternative techniques. The guidance may include the ability to rewind or fast-forward the guidance. The guidance may be responsive to measured or sensed user or team member inputs.

[0029] In some examples, the methods described herein may be used to generate customized camera control simulations. From parameterized previous procedures, a sequence of camera targets (e.g., the positions in the field of view where the camera focuses or the camera orientation) may be determined, and a simulation may be generated that prompts the user to take actions that follow the same sequence of camera movements. In some examples, the methods described herein may be used to generate customized suturing simulations. From parameterized previous procedures, a sequence of needle positions, orientations, and movements may be determined, and a simulation may be generated that guides the user to take actions that follow the same or similar sequence of positions, orientations, and movements. In some examples, the methods described herein may be used to generate customized incision simulations. From parameterized previous procedures, a sequence of instrument positions, orientations, and movements may be determined, and a simulation may be generated that guides the user to take actions that follow the same or similar sequence of positions, orientations, and movements. In some examples, the methods described herein may be used to generate customized energy delivery. From parameterized previous procedures, a location and amount of ablation energy may be determined, and a simulation may be generated that teaches the user to deliver energy with the same parameters. In some examples, the methods described herein may be used to perform a setup procedure including port placement for a robotic-assisted instrument. From a parameterized previous procedure, an optimized location for an initial manipulator assembly setup or arm placement and / or port placement may be determined, and a simulation may be generated to guide a user to select the same or similar manipulator assembly setup or port placement location.

[0030] In another example of process 106, the flowchart of FIG. 5B illustrates a method 520 for customizing a simulation exercise. In this example, the customized simulation exercise may be based on a current medical procedure (e.g., a diagnostic, therapeutic, or surgical procedure being performed in real time on a patient). In some examples, the simulation experience may be experienced simultaneously with the current medical procedure or after completion of the current medical procedure. The simulation experience may, for example, allow a clinician performing the current procedure to virtually re-experience or repeat a segment of the current procedure or may allow a surgical resident to virtually experience a segment simultaneously with the current procedure.

[0031] As a current medical procedure is being performed, a parameterized medical procedure record (e.g., record 200) may be generated in real time for the current procedure. In process 522, parameters may be received for the parameterized current procedure. For example, parameters from the medical record of the current procedure may include parameters associated with the procedure information (e.g., procedure information 202) or segments and actions of the procedure.

[0032] In process 524, an interval of the current treatment may be determined. The interval may be determined or identified in any of a variety of ways. For example, the interval may have a predetermined fixed duration relative to the duration trigger (e.g., 2 minutes of the current treatment before engaging a trigger switch on a user interface or control device). In another example, the current treatment interval may have a user-defined duration (e.g., 90 seconds before the trigger). In another example, the current treatment interval may be marked by a user engaging a trigger switch at the interval start time and the interval end time. In another example, the current treatment interval may be marked by rewinding a video recording of the current treatment from an interval end frame to an interval start frame. In another example, the interval may be determined by image analysis identifying a trigger action of a tool or tissue in the endoscopic image associated with an interval start or stop action. In another example, the interval may be determined by kinematic recognition of a trigger event, such as a movement or command to move a robotic arm or a tool attached to the robotic arm.

[0033] In process 526, a simulation exercise may be generated based on a plurality of parameters associated with the interval of the current procedure. The simulation exercise may be an augmented reality or virtual reality simulation that allows a user to experience the interval of the current procedure in a simulated environment. The simulation exercise may be provided to a trainee, for example, who has viewed the current procedure on a secondary operator's console and now wishes to perform the procedure segment he witnessed in a simulated environment. The simulation exercise may be performed by the trainee while the current procedure is continued by the original clinician, or may be performed at a later time (including a repeat) after the current procedure has ended. In some examples, the simulation exercise may be provided to the original clinician who performed the current procedure after the current procedure has ended, allowing the clinician to practice the identified portion of the procedure one or more times following the current procedure. As an example, if the current procedure includes an incision segment, an interval of the current procedure that includes the incision section may be identified. The generated simulation allows the trainee or original clinician to virtually experience the identified interval that includes the incision section under the same parameters as the original procedure. The virtual experience may include guidance (e.g., haptic guidance, visual guidance) that prompts the user to replicate the same hand movements and tool manipulations as in the original procedure. As another example, if the current procedure includes camera adjustments, intervals of the current procedure that include camera adjustments may be identified. The generated simulation allows the trainee or original clinician to virtually experience the identified intervals that include camera adjustments under the same parameters as the original procedure. The virtual experience may include guidance (e.g., haptic guidance, visual guidance) that prompts the user to replicate the same hand movements and camera manipulations as in the original procedure.

[0034] In another example of process 106, the flowchart of FIG. 5C illustrates a method 530 for customizing a simulation exercise. In this example, the customized simulation exercise may be based on a current medical procedure (e.g., a diagnostic, therapeutic, or surgical procedure being performed in real time on a patient), but may allow a user to preview anticipated or future segments of the current procedure. The simulation experience may be experienced simultaneously with the current medical procedure, such as during pauses in the current medical procedure. The simulation experience may, for example, allow a clinician performing the current procedure to virtually experience predictable segments of the current procedure as determined and modeled from previous procedures of the same type by the same or a different clinician.

[0035] As a current medical procedure is being performed, a parameterized medical procedure record (e.g., record 200) may be generated in real time for the current procedure. In process 532, parameters may be received for the parameterized current procedure. For example, parameters from the medical record of the current procedure may include parameters associated with the procedure information (e.g., procedure information 202) or segments and actions of the procedure.

[0036] One or more previous treatments may be identified that correspond to the multiple parameters from the parameterized current treatment in process 534. The identified previous treatments may have the same or similar parameters, including, for example, the same tools used, the same robotic manipulator setup configuration, the same instrument setup, and similar patient characteristics (e.g., BMI, stage of disease progression).

[0037] In process 536, a simulation exercise may be generated for an expected segment of the current procedure based on multiple parameters from the current procedure and one or more parameterized previous procedures. The simulation exercise may be an augmented reality or virtual reality simulation that allows the clinician performing the current procedure to experience the expected segment of the current procedure in a simulated environment. The simulation exercise may include guidance (e.g., haptic guidance, visual guidance) that prompts the user to practice or replicate the same hand movements and tool manipulations as a previous procedure having similar parameters. In some examples, the simulation exercise may allow the clinician to preview collision-prone segments of the instrument or arm and train with previous procedure parameters to learn to avoid collisions. In some examples, the simulation exercise may allow the clinician to practice camera movements in anticipation of upcoming treatment segments that will be performed, preferably under different camera angles. In some examples, the simulation exercise may allow the clinician to practice tool wrist movements in anticipation of upcoming segments of the current procedure. The simulation exercise may be performed during pauses in the current procedure using the same user-controlled device taken out of instrument following mode so that the control device movements do not actuate the surgical tool within the patient. The simulation exercise may allow a user to perform a segment simulation close in time to the subsequent performance of an actual treatment segment using the same tools, the same setup configuration, and the same patient characteristics.

[0038] The medical procedures and simulations described herein may be performed using a variety of manual or robotic-assisted techniques. Figures 6-8 together provide an overview of a robotic-assisted medical system 610 that may be used in a medical procedure or simulation, including, for example, a diagnostic, therapeutic, or surgical procedure. The medical system 610 is located within a medical environment 611. The medical environment 611 is shown in Figure 6 as a surgical room. In other embodiments, the medical environment 611 may be an emergency room, a medical training environment, a medical laboratory, or some other type of environment in which any number of medical procedures or medical training procedures may be performed. In still other embodiments, the medical environment 611 may include a surgical room and a control area located outside of the surgical room.

[0039] In one or more embodiments, the medical system 610 may be a teleoperated medical system under the remote control of a surgeon. In alternative embodiments, the medical system 610 may be under partial control of a computer programmed to perform a medical procedure or ancillary procedure. In yet other alternative embodiments, the medical system 610 may be a fully automated medical system under full control of a computer programmed to perform a medical procedure or ancillary procedure with the medical system 610. One example of a medical system 610 that may be used to implement the systems and techniques described in this disclosure is the da Vinci® Surgical System manufactured by Intuitive Surgical Operations, Inc. of Sunnyvale, California.

[0040] As shown in FIG. 6, the medical system 610 generally includes an assembly 612 that may be attached to or positioned near a surgical table O on which a patient P is positioned. The assembly 612 may be referred to as a patient side cart, a surgical cart, or a surgical robot. In one or more embodiments, the assembly 612 may be a remotely operated assembly. The remotely operated assembly may be referred to, for example, as a manipulation system and / or a remotely operated arm cart. A medical instrument system 614 and an endoscopic imaging system 615 are operably coupled to the assembly 612. An operator input system 616 allows a surgeon S or other type of clinician to view an image of or representative of a surgical site and control the operation of the medical instrument system 614 and / or the endoscopic imaging system 615.

[0041] The medical instrument system 614 may include one or more medical instruments. In embodiments in which the medical instrument system 614 includes multiple medical instruments, the multiple medical instruments may include multiple of the same medical instruments and / or multiple different medical instruments. Similarly, the endoscopic imaging system 615 may include one or more endoscopes. In the case of multiple endoscopes, the multiple endoscopes may include multiple of the same endoscopes and / or multiple different endoscopes.

[0042] The operator input system 616 may be located at a surgeon's control console, which may be located in the same room as the operating table O. In one or more embodiments, the operator input system 616 may be referred to as a user control system. In some embodiments, the surgeon S and the operator input system 616 may be located in a different room or in a different building entirely than the patient P. The operator input system 616 generally includes one or more control device(s), sometimes referred to as input control devices, for controlling the medical instrument system 614 or the imaging system 615. The control device(s) may include one or more of any number of various input devices, such as hand grips, joysticks, trackballs, data gloves, trigger guns, foot pedals, manual controllers, voice recognition devices, touch screens, body motion or presence sensors, and other types of input devices.

[0043] In some embodiments, the control device(s) are equipped with the same Cartesian degrees of freedom as the medical instrument(s) of the medical instrument system 614 to provide telepresence to the surgeon S, which is the perception that the control device(s) are one with the instruments, and the surgeon has a strong sense of directly controlling the instruments as if he were at the surgical site. In other embodiments, the control device(s) may have more or fewer degrees of freedom than the associated medical instruments and still provide telepresence to the surgeon S. In some embodiments, the control device(s) are manual input devices that move with six degrees of freedom, and may also include actuatable handles for actuating the instruments (e.g., for closing the grasping jaw end effectors, applying electrical potentials to electrodes, administering drug treatments, and actuating other types of instruments). Thus, the degrees of freedom and actuation capabilities of the control device(s) are mapped to the degrees of freedom and range of motion available to the medical instrument(s).

[0044] The assembly 612 supports and manipulates the medical instrument system 614 while the surgeon S views the surgical site through an operator input system 616. Images of the surgical site may be acquired by an endoscopic imaging system 615, which may be manipulated by the assembly 612. The assembly 612 may include multiple endoscopic imaging systems 615, as well as multiple medical instrument systems 614. The number of medical instrument systems 614 used at one time will generally depend on the diagnostic or surgical procedure to be performed and the space constraints within the operating room, among other factors. The assembly 612 may include one or more non-servo controlled links (e.g., one or more links that can be manually placed and locked in place, commonly referred to as a manipulator support structure) and the kinematic structure of the manipulator. When the manipulator takes the form of a teleoperated manipulator, the assembly 612 is a teleoperated assembly. The assembly 612 includes multiple motors that drive inputs on the medical instrument systems 614. In one embodiment, these motors move in response to commands from a control system (e.g., control system 620). The motor includes a drive system that, when coupled to the medical instrument system 614, can advance the medical instrument into a natural or surgically created anatomical opening. Other motorized drive systems can move the distal end of the medical instrument in multiple degrees of freedom, which can include three linear motions (e.g., linear motion along the X, Y, and Z Cartesian axes) and three rotational motions (e.g., rotation about the X, Y, and Z Cartesian axes). Additionally, the motor may be used to actuate an articulating end effector of the medical instrument for grasping tissue in the jaws, such as a biopsy device. The medical instruments of the medical instrument system 614 can include end effectors with a single working member, such as a scalpel, a blunt blade, an optical fiber, etc. Other end effectors can include, for example, forceps, graspers, scissors, or clip appliers.

[0045] The medical system 610 also includes a control system 620. The control system 620 includes at least one memory 624 and at least one processor 622 (which may be part of a processing unit) for effecting control between the medical instrument system 614, the operator input system 616, and other auxiliary systems 626, which may include, for example, an imaging system, an audio system, a fluid delivery system, a display system, a lighting system, a steering control system, an irrigation system, and / or an aspiration system. A clinician may navigate within the medical environment 611, for example, to access the assembly 612 during a set-up procedure or to view the display of the auxiliary system 626 from the patient's bedside. In some embodiments, the auxiliary system 626 may include a display screen separate from the operator input system 616. In some examples, the display screen may be a standalone screen that is movable within the medical environment 611. The display screen may be oriented such that the surgeon S and one or more clinicians or assistants can view the display screen simultaneously.

[0046] 6 as being external to assembly 612, control system 620 may in some embodiments be housed entirely within assembly 612. Control system 620 also includes programmed instructions (e.g., stored on a non-transitory computer-readable medium) for implementing some or all of the methods described in accordance with aspects disclosed herein. Although control system 620 is shown as a single block in the simplified schematic diagram of FIG. 6, control system 620 may include two or more data processing circuits, with a portion of the processing optionally performed on or adjacent to assembly 612 and another portion of the processing performed in operator input system 616, etc.

[0047] Any of a wide variety of centralized or distributed data processing architectures may be employed. Similarly, the programmed instructions may be implemented as a number of separate programs or subroutines, or may be integrated into a number of other aspects of the systems described herein, including the remote control system. In one embodiment, the control system 620 supports wireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and Wireless Telemetry.

[0048] The control system 620 communicates with a database 627 that may store one or more medical procedure records. The database 627 may be stored in memory 624 and may be dynamically updated. Additionally or alternatively, the database 627 may be stored on a device, such as a server or portable storage device, that is accessible by the control system 620 via an internal network (e.g., a secure network of a medical facility or remote operation system provider) or an external network (e.g., the Internet). The database 627 may be distributed across two or more locations. For example, the database 627 may exist on multiple devices, which may include devices of different entities and / or cloud servers. Additionally or alternatively, the database 627 may be stored on portable user-assigned devices, such as computers, mobile devices, smartphones, laptops, electronic badges, tablets, pagers, and other similar user devices.

[0049] In some embodiments, the control system 620 may include one or more servo controllers that receive force and / or torque feedback from the medical instrument system 614. In response to the feedback, the servo controller sends a signal to the operator input system 616. The servo controller(s) may also send a signal to instruct the assembly 612 to move the medical instrument system(s) 14 and / or the endoscopic imaging system 615 extending through an opening in the patient's body into an internal surgical site within the patient's body. Any suitable conventional or dedicated servo controller may be used. The servo controller may be separate from or integrated with the assembly 612. In some embodiments, the servo controller and assembly 612 are provided as part of a remotely operable arm cart that is positioned adjacent to the patient's body.

[0050] The control system 620 can be coupled with the endoscopic imaging system 615 and can include a processor for processing the captured images, for example, for later display to the surgeon on the surgeon's control console or on another suitable display located locally and / or remotely. For example, if a stereoscopic endoscope is used, the control system 620 can process the captured images to present an adjusted stereoscopic image of the surgical site to the surgeon as a field of view image. Such adjustments can include alignment between opposing images and can include adjusting the stereoscopic working distance of the stereoscopic endoscope.

[0051] In alternative embodiments, the medical system 610 may include two or more assemblies 612 and / or two or more operator input systems 616. The exact number of assemblies 612 will depend on the surgical procedure and space constraints within the operating room, among other factors. The operator input systems 616 may be co-located or may be located in separate locations. Multiple operator input systems 616 allow two or more operators to control one or more assemblies 612 in various combinations. The medical system 610 may also be used to train and rehearse medical procedures.

[0052] FIG. 7 is a perspective view of one embodiment of a manipulator assembly 612, which may be referred to as a manipulation system, patient side cart, surgical cart, teleoperated arm cart, or surgical robot. The assembly 612 shown provides for the manipulation of three surgical tools 630a, 630b, and 630c (e.g., medical instrument system 614), as well as an imaging device 628, such as a stereo endoscope (e.g., endoscopic imaging system 615), used to capture images of the site of the procedure. The imaging device 628 may send signals to the control system 620 via cable 656. Manipulation is provided by a teleoperated mechanism having multiple joints. The imaging device 628 and surgical tools 630a-c may be positioned and manipulated through an incision in the patient, maintaining a kinematic telecenter at the incision to minimize the size of the incision. Images of the surgical site may include images of the surgical tools 630a-c when their distal ends are positioned within the field of view of the imaging device 628. The imaging device 628 and surgical tools 630a-c may each be a therapeutic, diagnostic, or imaging instrument.

[0053] 8 is a perspective view of one embodiment of an operator input system 616 in a surgeon's control console. In some examples, the operator input system 616 may be a simulated medicine system separate from the control of medical instruments or manipulative assemblies. The simulated medicine system may be coupled to the control system to access a catalog of parameterized previous medical procedures, user profiles, or other information used to generate and perform the simulated exercises.

[0054] In this example, operator input system 16 includes a left-eye display 632 and a right-eye display 634 for presenting surgeon S with a calibrated stereoscopic view of the surgical environment that enables depth perception. Left-eye display 632 and right-eye display 634 may be components of a display system 635. In other embodiments, display system 635 may include one or more other types of displays. In some embodiments, the image(s) displayed on display system 635 may be displayed separately or simultaneously on at least one display screen of auxiliary system 626.

[0055] The operator input system 616 further includes one or more input control devices 636 that allow the assembly 612 to manipulate one or more instruments of the endoscopic imaging system 615 and / or the medical instrument system 614. The input control devices 636 may provide the same Cartesian degrees of freedom as their associated instruments, providing the surgeon S with telepresence, i.e., the perception that the input control devices 636 are one with the instruments, such that the surgeon has a strong sense of directly controlling the instruments. Thus, the degrees of freedom of each input control device 636 are mapped to the degrees of freedom of its associated instrument (e.g., one or more of the instruments of the endoscopic imaging system 615 and / or the medical instrument system 614). To this end, position, force, and tactile feedback sensors (not shown) may be employed to communicate position, force, and tactile sensations from the medical instruments, e.g., the surgical tools 630a-c or the imaging device 628, to the surgeon's hands via the input control devices 636. Additionally, the placement of the medical instruments may be mapped to the placement of the surgeon's hands and the view from the surgeon's eyes so that the surgeon has a strong sense of direct control over the instruments. The input control device 637 is a foot pedal that receives input from the user's foot. Aspects of the operator input system 616, assembly 612, and auxiliary system 626 may be adjustable and customizable to meet the physical needs, skill level, or preferences of the surgeon S.

[0056] Specific details are described herein to describe some embodiments. Numerous specific details are described to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that some embodiments can be practiced without some or all of these specific details. The specific embodiments disclosed herein are meant to be illustrative and not limiting. One skilled in the art may recognize other elements not specifically described herein that are within the scope and spirit of the present disclosure.

[0057] Elements described in detail with reference to one embodiment, implementation, or application may optionally, whenever practical, be included in other embodiments, implementations, or applications in which they are not specifically shown or described. For example, if an element is described in detail with reference to one embodiment and not with reference to a second embodiment, the element may still be claimed as being included in the second embodiment. Thus, to avoid unnecessary repetition in the following description, one or more elements illustrated and described in connection with one embodiment, implementation, or application may be incorporated in other embodiments, implementations, or aspects, unless specifically described otherwise, unless one or more elements render the embodiment or implementation non-functional, or unless two or more of the elements provide conflicting functionality. Not all illustrated processes are performed in all embodiments of the disclosed methods. Additionally, one or more processes not explicitly illustrated may be included before, after, between, or as part of the illustrated processes. In some embodiments, one or more of the processes may be executed by a control system or may be implemented, at least in part, in the form of executable code stored on a non-transitory, tangible, machine-readable medium that, when executed by one or more processors, can cause the one or more processors to perform one or more of the processes.

[0058] Any changes and further modifications to the described devices, instruments, methods, and any further applications of the principles of the present disclosure are fully contemplated as would normally occur to a person skilled in the art to which the present disclosure pertains.In addition, the dimensions provided herein are for specific examples, and it is contemplated that different sizes, dimensions, and / or ratios may be utilized to implement the concepts of the present disclosure.In order to avoid unnecessary repetition of description, one or more components or actions described according to one exemplary embodiment may be used or omitted as applicable from other exemplary embodiments.For brevity, multiple iterations of these combinations will not be described separately.For simplicity, in some instances, the same reference numbers are used throughout the drawings to refer to the same or similar parts.

[0059] The systems and methods described herein may be suitable for imaging any of a variety of anatomical systems, including the lungs, colon, intestines, stomach, liver, kidneys and calyces, brain, heart, circulatory system including vasculature, and / or the like. Although some embodiments are provided herein with respect to medical procedures, any reference to medical or surgical instruments and medical or surgical methods is non-limiting. For example, the instruments, systems, and methods described herein may be used for non-medical purposes, including industrial applications, general robotic applications, and sensing or manipulating non-tissue workpieces. Other exemplary applications include cosmetic enhancement, imaging of human or animal anatomy, collecting data from human or animal anatomy, and training medical or non-medical personnel. Further exemplary applications include use for performing procedures on tissues removed from human or animal anatomy (without returning to the human or animal anatomy), and performing procedures on human or animal cadavers. Additionally, these techniques may also be used for surgical and non-surgical medical therapeutic or diagnostic procedures.

[0060] One or more elements in the embodiments of the present disclosure may be implemented in software for execution on a processor of a computer system such as a control processing system. When implemented in software, the elements of the embodiments of the present disclosure may be code segments for performing various tasks. The programs or code segments may be stored in a processor-readable storage medium or device downloaded by a computer data signal embodied in a carrier wave over a transmission medium or communication link. The processor-readable storage device may include any medium capable of storing information, including optical, semiconductor, and / or magnetic media. Examples of the processor-readable storage device include electronic circuits, semiconductor elements, semiconductor memory devices, read-only memories (ROMs), flash memories, erasable programmable read-only memories (EPROMs), floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. The code segments may be downloaded over a computer network such as the Internet, an intranet, or the like. Any of a wide variety of centralized or distributed data processing architectures may be employed. The programmed instructions may be implemented as several separate programs or subroutines, or may be integrated into several other aspects of the system described herein. In some examples, the control system may support wireless communication protocols such as Bluetooth, Infrared Data Association (IrDA), HomeRF, IEEE 802.11, Digital Enhanced Cordless Telecommunications (DECT), Ultra Broadband (UWB), ZigBee, and Wireless Telemetry.

[0061] It should be noted that the processes and displays presented may not inherently relate to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct a more specialized apparatus to perform the operations described. The required structure for a variety of these systems will be set forth as elements of the claims. Additionally, embodiments of the present invention are not described with reference to any particular programming language. It will be understood that a variety of programming languages ​​may be used to implement the teachings of the present invention as described herein.

[0062] The present disclosure describes various instruments, parts of instruments, and anatomical structures with respect to their status in three-dimensional space. As used herein, the term position refers to the location of an object or part of an object in three-dimensional space (e.g., three translational degrees of freedom along Cartesian x, y, and z coordinates). As used herein, the term orientation refers to the rotational disposition of an object or part of an object (e.g., in one or more rotational degrees of freedom such as roll, pitch, and / or yaw). As used herein, the term pose refers to the position of an object or part of an object in at least one translational degree of freedom and the orientation of that object or part of an object in at least one rotational degree of freedom (e.g., up to six degrees of freedom in total). As used herein, the term shape refers to a set of poses, positions, or orientations measured along an object.

[0063] While certain exemplary embodiments of the present invention have been described and illustrated in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of the broad invention and not limiting thereof, and since various other modifications may occur to those skilled in the art, the embodiments of the present invention are not limited to the specific constructions and arrangements shown and described.

Claims

1. 1. A health care system comprising: A display system; An operator input device; a control system in communication with the display system and the operator input device; The control system comprises: A processor; Memory and The memory, when executed by the processor, provides the control system with: Accessing user experience factors; Referencing a parameterized set of previous actions; identifying a parameterized prior action associated with the experience factor from the set of parameterized prior actions; generating a simulation exercise including a plurality of parameters from the parameterized previous exercise; determining a model input to the operator input device associated with the plurality of parameters; 16. A medical system comprising machine-readable instructions for:

2. The medical system of claim 1 , wherein the experience factors include skill development subjects.

3. The medical system of claim 1 , wherein the experience factors include a type of future procedure scheduled for the user.

4. The medical system of claim 1 , wherein the experience factor is determined from previous procedures performed by the user.

5. The medical system of claim 1 , wherein the experience factors include anatomical parameters.

6. The medical system of claim 1 , wherein the experience factors are determined from a user profile.

7. The medical system of claim 1 , wherein the machine-readable instructions, when executed by the processor, further cause the control system to compare inputs received at the operator input device to the model inputs.

8. The medical system of claim 1 , wherein the machine-readable instructions, when executed by the processor, further cause the control system to scale the parameterized previous treatment.

9. The medical system of claim 1 , wherein the machine-readable instructions, when executed by the processor, further cause the control system to generate guidance for executing the model inputs.

10. The medical system of claim 1 , wherein accessing the experience factors comprises comparing a user skill profile to an expected procedure schedule.

11. The medical system of claim 1 , wherein accessing the experience factors comprises evaluating a user skill rating against an objective performance indicator.

12. The medical system of claim 1 , wherein the set of parameterized previous actions includes parameterized previous actions performed by an operator of the operator input device.

13. The medical system of claim 1 , wherein the set of parameterized previous procedures includes parameterized previous procedures performed by a trainer.

14. The medical system of claim 1 , wherein the plurality of parameters are determined from at least one of robotic system records, sensor data, in-vivo imaging data, ex-vivo imaging data, patient monitoring data, or patient outcome records.

15. The medical system of claim 1 , wherein the plurality of parameters comprises a position, an orientation, or a motion of a medical instrument.

16. The medical system of claim 1 , wherein the plurality of parameters comprises a position, an orientation, or a motion of an imaging instrument.

17. The medical system of claim 1 , wherein the plurality of parameters comprises a position, orientation, or movement of an operator input device.

18. The medical system of claim 1 , wherein the plurality of parameters comprises an operator position, orientation, or movement of an operator input device.

19. The medical system of claim 1 , wherein the plurality of parameters includes a position, orientation, or movement of a team member.

20. The medical system of claim 1 , wherein the plurality of parameters includes an initial manipulator assembly setup configuration.

21. The medical system of claim 1 , wherein the plurality of parameters includes an initial anatomical port placement.

22. The medical system of claim 1 , wherein the plurality of parameters includes location, frequency, or amount of energy delivered.

23. 1. A method for generating a customized medical simulation exercise, comprising: accessing a user's experience factors; Referencing a parameterized set of previous actions; identifying a parameterized prior action associated with the experience factor from the set of parameterized prior actions; generating a simulation exercise including a plurality of parameters from the parameterized previous action; determining a model input to an operator input device associated with the plurality of parameters; The method includes:

24. The method of claim 23 , wherein the experience factors include skill development subjects.

25. 24. The method of claim 23, wherein the experience factors include a type of future procedure scheduled for the user.

26. The method of claim 23 , wherein the experience factor is determined from previous actions taken by the user.

27. The method of claim 23 , wherein the experience factors include anatomical parameters.

28. The method of claim 23 , wherein the experience factors are determined from a user profile.

29. 24. The method of claim 23, further comprising the step of comparing input received at an operator input device with the model input.

30. 24. The method of claim 23, further comprising the step of scaling the parameterized previous actions.

31. 24. The method of claim 23, further comprising generating guidance for executing the model inputs.

32. 24. The method of claim 23, wherein the step of accessing the experience factors comprises comparing a user skill profile to an expected treatment schedule.

33. 24. The method of claim 23, wherein the step of accessing experience factors comprises evaluating user skill ratings against objective performance indicators.

34. The method of claim 23 , wherein the set of parameterized previous actions includes parameterized previous actions performed by an operator of the operator input device.

35. 24. The method of claim 23, wherein the set of parameterized previous actions includes parameterized previous actions performed by a trainer.

36. 24. The method of claim 23, wherein the plurality of parameters are determined from at least one of robotic system records, sensor data, in-vivo imaging data, ex-vivo imaging data, patient monitoring data, or patient outcome records.

37. The method of claim 23 , wherein the plurality of parameters comprises a position, an orientation, or a movement of a medical instrument.

38. The method of claim 23 , wherein the plurality of parameters comprises a position, an orientation, or a movement of an imaging instrument.

39. The method of claim 23 , wherein the plurality of parameters includes a position, an orientation, or a movement of an operator input device.

40. The method of claim 23 , wherein the plurality of parameters includes an operator position, orientation, or movement of an operator input device.

41. The method of claim 23 , wherein the plurality of parameters includes a position, orientation, or movement of a team member.

42. The method of claim 23 , wherein the plurality of parameters includes an initial manipulator assembly setup configuration.

43. The method of claim 23 , wherein the plurality of parameters includes an initial anatomical port placement.

44. 24. The method of claim 23, wherein the plurality of parameters includes location, frequency, or amount of energy delivered.

45. 1. A method for generating a customized medical simulation exercise, comprising: receiving a plurality of parameters from a parameterized current procedure; determining an interval for the current treatment; generating a simulation exercise based on a set of parameters of the plurality of parameters associated with the interval of the current procedure; The method includes:

46. 1. A method for generating a customized medical simulation exercise, comprising: receiving a plurality of parameters from a parameterized current procedure; identifying one or more parameterized previous procedures that correspond to the plurality of parameters from the parameterized current procedure; generating a simulation exercise for an anticipated treatment segment of the current treatment based on the plurality of parameters from the current treatment and the one or more parameterized previous treatments; The method includes:

Images