System and method for generating customized medical simulation
By using processors and memory to access user experience factors and previous routine data in medical systems, customized medical simulation exercises are solved, and the existing system cannot meet the needs of clinicians' personalized skills development, achieving high targeting simulation training effect.
Patent Information
- Application Number
- CN202380026620.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-16
- Filing Date
- 2023-03-14
- Publication Date
- 2024-11-08
AI Technical Summary
The existing medical simulation system is difficult to provide customized medical simulation exercises, cannot effectively meet the needs of clinicians' personalized skill development, and it is difficult to generate targeted simulation exercises based on user experience factors and previous routine data.
By introducing processors and memory in a medical system, accessing user experience factors and parameterized sets of previous routines with machine-readable instructions, identifying parameterized previous routines associated with experience factors, and generating a multi-parameterized one that includes multiple parameters Simulation exercises, identifying the model inputs associated with multiple parameters by the operator input device, thereby creating a customized medical simulation exercise.
It has achieved the generation of customized simulation teaching syllabus based on the user's skill development needs, enhanced the user's medical skills and ability, met the personalized development needs of clinicians, and improved the pertinence and effectiveness of simulation exercises.
Smart Images

Figure CN118922891A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims U.S. Provisional Application No. 63 / 320,553, filed on March 16, 2022, entitled “SYSTEMS AND METHODS FOR GENERATING CUSTOMISED MEDICAL SIMULATIONS,” which is incorporated herein by reference in its entirety. Technical Field
[0003] The present disclosure relates to systems and methods for generating customized medical simulations. Background Art
[0004] Minimally invasive medical techniques are intended to reduce the amount of tissue damaged during a diagnostic or surgical procedure, thereby reducing recovery time, discomfort, and harmful side effects for the patient. Such minimally invasive techniques can be performed through one or more surgical incisions or through natural orifices in the patient's anatomy. Through these incisions or natural orifices, clinicians can insert minimally invasive medical instruments to perform medical routines either manually or through actuation of robotically assisted instruments. Customized medical simulations can be developed to improve medical routines, train clinicians, and / or evaluate the effectiveness of medical routines. Summary of the invention
[0005] Examples of the invention are summarized by the claims following the description. Consistent with some examples, a medical system may include a display system, an operator input device, and a control system in communication with the display system and the operator input device. The control system may include a processor and a memory, the memory including machine-readable instructions that, when executed by the processor, cause the control system to access an experience factor of a user and reference a set of parameterized prior routines. The instructions may also cause the control system to identify a parameterized prior routine associated with the experience factor from the set of parameterized prior routines and generate a simulation exercise including a plurality of parameters from the parameterized prior routine. A model input associated with the plurality of parameters to the operator input device may be determined.
[0006] Consistent with some examples, a method for generating a customized medical simulation exercise may include accessing an experience factor of a user, referencing a set of parameterized previous routines, and identifying a parameterized previous routine associated with the experience factor from the set of parameterized previous routines. The method may also include generating a simulation exercise that includes a plurality of parameters from the parameterized previous routines, and determining a model input to an operator input device associated with the plurality of parameters.
[0007] It should be understood that the above general description and the following detailed description are exemplary and explanatory in nature, and are intended to provide an understanding of the present disclosure without limiting the scope of the present disclosure. In this regard, additional aspects, features and advantages of the present disclosure will be apparent to those skilled in the art through the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 is a method of customizing a simulation exercise according to some embodiments.
[0009] Figure 2 User profiles are shown in accordance with some embodiments.
[0010] Figure 3 is a schematic diagram of a parameterized medical routine record in accordance with some embodiments.
[0011] Figure 4 is a flow chart illustrating a measurement and recording system for parameterizing a medical routine.
[0012] Figure 5A A method of generating a customized simulation exercise according to some embodiments.
[0013] Figure 5B A method of generating a customized simulation exercise according to some embodiments.
[0014] Figure 5C A method of generating a customized simulation exercise according to some embodiments.
[0015] Figure 6 is a schematic diagram of a robot-assisted medical system according to some embodiments.
[0016] Figure 7 is a perspective view of a manipulator assembly according to some embodiments.
[0017] Figure 8 is a perspective view of an operator input system according to some embodiments.
[0018] Embodiments of the present disclosure and their advantages may be best understood by referring to the following detailed description.It should be understood that for purposes of illustrating but not limiting embodiments of the present disclosure, like reference numerals are used to identify like elements illustrated in one or more of the figures. DETAILED DESCRIPTION
[0019] A medical skill development system can identify a user's skill development needs and can generate a customized simulation syllabus including one or more simulation exercises built from prior routine data. The simulation exercises can strengthen the user's skill capabilities. Systems and methods for generating customized medical routine simulations are provided. Clinician profiles and parameterized prior routines can be used to create simulation exercises to meet the clinician's development needs.
[0020] Figure 1 A method 100 of generating a simulation exercise according to some embodiments is shown. The method 100 is shown as a set of operations or a process. Figure 1 The process shown in can be Figure 1 The method 100 may be performed in a different order than the order shown in the figure, and one or more of the processes shown may not be performed in some embodiments of the method 100. In addition, Figure 1 One or more processes not explicitly shown in the method 100 may be included before, after, in between, or as part of the process shown. In some embodiments, one or more processes of method 100 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which, when executed by one or more processors (e.g., a processor of a control system), causes the one or more processors to perform the one or more processes. At process 102, user information may be received. At process 104, a directory, database, or collection of one or more parameterized previous routines may be referenced. At process 106, a simulation exercise may be determined based on the user information and the referenced parameterized previous routines.
[0021] In more detail, at process 102, user information can be received from a user profile. User experience factors can be determined by the user profile and can include skill development topics for the user to improve clinical skills, or can include the type of future routines scheduled for the user to allow the user to prepare for future experiences. For example, the user information can be received at a processor (e.g., processor 620 of medical system 610). In some examples, a medical routine record can be received from a memory device (e.g., memory 624 of medical system 610). Figure 2An example of a user profile 150 including previous routine information 152 and future or expected routine information 154 is shown. For example, the user may be a surgeon, a medical device operator, a surgical assistant, or other clinician or operator involved in a medical routine. In some examples, the user may be a synthetic user with artificial intelligence that can be trained using the methods and systems described herein. Previous routine information 152 may include, for example, information about patients involved in previous routines performed by the user. Patient information or characteristics may include data from the patient's electronic medical record, gender, height, weight, body mass index, medical image data (e.g., CT images, ultrasound images), medical measurement data (e.g., blood pressure, electrocardiogram results), and / or other information related to patients involved in the user's previous routines. Previous routine information 152 may additionally or alternatively include, for example, team characteristics of previous medical routines performed by the user. Team characteristics may include, for example, the roles of team members involved in previous routines, the experience level of each team member involved in previous routines, and the previous experience of each team member in collaborating with the user. The previous routine information 152 may additionally or alternatively include, for example, information about a surgical system or instrument used in a previous medical routine performed by the user. The surgical system or instrument may include, for example, a robotic-assisted medical system, a laparoscopic instrument, an open routine instrument, or any combination thereof. The previous routine information 152 may additionally or alternatively include, for example, routine type information of a previous medical routine performed by the user. The type of routine may include, for example, abdominal, cardiac, colorectal, gynecological, head / neck, lung, chest, and / or urological surgical routines. The previous routine information 152 may additionally or alternatively include, for example, segment information about a previous medical routine performed by the user. The segment information may include, for example, difficulty information about a previous routine or a segment of a previous routine. The previous routine information 152 may additionally or alternatively include, for example, a user skill profile including judgement information from a previous medical routine performed by the user. The judgement information may include a performance evaluation based on an objective performance indicator. The critique information may include an evaluation of technical aspects of prior routines, including information regarding errors, inefficiencies, suboptimal patient outcomes, or other positive or negative indicators of the user's technical proficiency. The critique information may also or alternatively include an evaluation of non-technical aspects of prior routines, including information regarding communication effectiveness, leadership, stress, or other positive or negative indicators of the user's non-technical proficiency. For example, prior routine information may be generated from a routine performed on a patient during a training session using a synthetic tissue structure or during a prior computer-generated simulation.
[0022] Future routine information 154 may include information about scheduled, planned, or otherwise known or anticipated future routines to be performed by the user. For each future routine, various information may be received including, for example, patient information, team composition information, surgical system information of the systems and equipment to be used, routine type, routine segment information, and / or expected difficulty level.
[0023] Reference again Figure 1 , at process 104, a parameterized previous routine directory, database, or other information collection may be referenced or queried. During a medical routine, a sensor system, a measurement system, and / or a recording system may capture data records and store them in a medical routine record. In some examples, a medical routine record may capture information about a medical routine performed using a robotic-assisted medical system, using a laparoscopic medical system, using a manual medical device, or using a combination of systems and devices. Each medical routine record may capture information about a medical routine performed on a patient by a single clinician or a team of medical professionals. As described in U.S. Provisional Patent Application No. 63 / 320,538, entitled “Systems and Methods for Parameterized Medical Routines,” filed on March 16, 2022, the medical routine may be parameterized, which is incorporated herein by reference in its entirety. Parameters may be associated with characteristic actions of a medical routine and may be based on data records generated during a previous medical routine. A catalog, database, or collection of parameterized routines may include parameterized routines associated with a particular clinician; parameterized routines from guidelines, trainers, or other experts; parameterized routines from other users (e.g., peers, trainees); and / or parameterized routines from synthetic routines performed virtually or on a synthetic patient.
[0024] Figure 3200 is a schematic diagram of a parameterized medical routine record 200, which can be, for example, stored in a set of parameterized previous routines. The parameterized medical routine record 200 can include routine information 202 and data records AG from the medical routine associated with the routine information 202. The routine information 202 can include various input parameters for the routine. For example, operator information 210 can include clinician identification information, training history, experience, preferences and / or other information related to one or more clinicians involved in the medical routine. The operator can be, for example, a surgeon, a surgical intern, an expert or a guiding medical practitioner. The routine information 202 can include patient characteristics 212, including patient age, sex, height, weight, medical history and / or other information related to the only patient being performed the medical routine. The routine information can include team features 214 of team members and support personnel participating in the medical routine, including identification information, roles in the routine, training history, experience, preferences or other information related to the personnel performing the routine. The routine information 202 may also include system information 216 about the system or systems used to perform the routine. The system may include a sensor or recording system that captures information such as set or sensed parameters of the system during the routine. The system may include a robotic-assisted medical system or components thereof, a laparoscopic medical system, a manual medical device, and / or a support or peripheral system. The system information 216 may include the manufacturer, model, serial number, time of use, time since last maintenance, use cycle count, maintenance history, calibration information, or other information related to the system used to perform the medical routine. The routine information 202 may also include a routine type 218, which includes information about the type of medical routine. The medical routine type may include abdominal, cardiac, colorectal, gynecological, neurological, head / neck, pulmonary, thoracic, urological, or other categories or subcategories of anatomical systems involved in the medical routine. The routine information 202 may also include segment information 220. The segment information 220 may include information about subdivided portions of the medical routine. For example, a routine segment may include a sequence or group of actions associated with ablation, stapling, suturing, dissection, tissue resection, anastomosis, camera control, instrument wrist control, setting change, or tool change that occurs one or more times during a medical routine. In some examples, the routine information may also include other information about the routine, including, for example, the date the routine occurred, the time and duration of the routine, and / or the location and facility identification at which the routine occurred.
[0025] The medical routine record 200 also includes a record of data AG captured during the medical routine. In some examples, such as Figure 4As shown, various measurement and recording systems 400 may be used to document a medical routine 300 to generate a data record 402 (e.g., data record AG) associated with the routine 300 (e.g., a previous medical routine). The measurement and recording system 400 may include, for example, a robot-assisted medical system (e.g., Figure 6 The measuring and recording system 400 may also or alternatively include one or more sensor systems 410 associated with a monitored medical device used in the medical routine 300. In some examples, the sensor system 410 may include a position, orientation, motion, and / or displacement sensor system for a manipulator or instrument coupled to a manipulator of the robotic-assisted medical system. In some examples, the sensor system 410 may include a force sensor system, a clock, a motor encoder, an energy usage sensor, a user eye tracking sensor, or other sensor systems that measure and / or record data about the manipulator or instrument. The measuring and recording system 400 may also or alternatively include one or more sensor systems 412 associated with a monitored medical device used in the medical routine 300. In some examples, the sensor system 412 may track the position, orientation, motion, displacement, force, energy usage, duration of use, and / or other measurements associated with the medical device.
[0026] The measurement and recording system 400 may also or alternatively include one or more imaging systems 414 used during the medical routine 300. In some examples, the imaging system 414 may be an in vivo imaging system, such as an endoscopic imaging system or an ultrasound imaging system used during the routine 300. In some examples, the imaging system 414 may be an in vitro imaging system for the patient's anatomy, such as a computed tomography (CT) imaging system, a magnetic resonance imaging (MRI) imaging system, or a functional near infrared spectroscopy (fNIRS) imaging system used during the routine 300. In some examples, the imaging system 414 may be an environmental imaging system, such as an optical imaging system, that tracks the position and movement of manipulators, instruments, equipment, and / or personnel in the patient's environment during the routine 300.
[0027] The measurement and recording system 400 may also or alternatively include one or more audio systems 416 used during the medical routine 300. The audio system may capture and record audio from personnel in the medical area of the routine 300, an operator performing the routine 300, a patient, and / or equipment in the medical area of the routine 300. The measurement and recording system 400 may also or alternatively include one or more in-routine patient monitoring systems 418 used during the medical routine 300. The patient monitoring systems 418 may include, for example, respiratory, cardiac, blood pressure, anesthesia, insufflation, and / or patient / table orientation monitoring systems. The measurement and recording system 400 may also or alternatively include one or more patient outcome recording systems 420 that may be referenced after the routine 300 is completed. The patient outcome recording systems 420 may record information about post-routine (i.e., post-operative) hospital stay duration, complications, positive results, negative results, mortality, or other post-operative information about the patient. The measurement and recording system 400 may also or alternatively include one or more routine skill recording systems 422 that capture and record objective performance indicators of a clinician performing the routine 300 .
[0028] Data record 402 may include data generated by measurement and recording system 400. For example, data record 430 may record the position, orientation, movement, and / or displacement of an instrument (e.g., instrument 614) controlled by a robot-assisted manipulator or by manual operation. In some examples, data record 432 may record the position, orientation, movement, and / or displacement of a robot-assisted manipulator assembly (e.g., 612) of any arm including the manipulator during routine 300. In some examples, data record 434 may record the position, orientation, movement, and / or displacement of an imaging system (such as an endoscope or other in vivo or in vitro imaging system) during routine 300. In some examples, data record 436 may record the position, orientation, movement, and / or displacement of an operator input device (e.g., 636) during routine 300. In some examples, data record 438 may record the position, orientation, movement, and / or displacement of an operator (e.g., surgeon S) directing control of the instrument during routine 300. For example, data record 438 may record the movement of an operator's hand or the movement of a tracking head disengaging from an operator console. In some examples, data record 440 may record the position, orientation, movement, and / or displacement of one or more members of the medical team participating in routine 300. In some examples, data record 442 may record various aspects of the initial setup of routine 300, including the position and arrangement of robot-assisted manipulator components, the placement of patient ports, and the location of peripheral devices. In some examples, data record 444 may include a record of the position, frequency, and amount of energy provided to or delivered by an instrument (e.g., an ablation instrument) during routine 300. In some examples, data record 446 may include a record of instrument changes during routine 300. In some examples, data record 448 may include a time-based record that captures dwell time, idle time, and / or duration or speed of actions during routine 300. In some examples, data record 450 may capture various aspects of the workflow, including the number and / or sequence of actions during routine 300. For example, data record 450 may include a sequence of positions, orientations, movements, and / or displacements associated with discrete activities. In some examples, data record 452 can capture errors, difficulties, conflicting events during routine 300 or other unplanned events such as robotic arm collisions, conditions that led to a switch from robotic-assisted surgery to open surgery during a routine, conditions that led to a switch from robotic-assisted surgery to laparoscopic surgery during a routine, or conditions that led to a switch from laparoscopic surgery to open surgery during a routine. In some examples, data record 454 can capture aspects of the anatomical environment, including the size of the organ, the incision, and / or the treatment delivery area. Other aspects of the anatomical environment that can be recorded include the width of the pelvis, the distance between anatomical structures, and / or the location of the vasculature.In some examples, data record 456 may include measures of intervention consequences, such as bleeding, smoke, tissue movement, and / or tissue color change. In some examples, data record 458 may include a catalog of key skills for performing routine 300, relevant objective performance indicators of experienced clinicians performing the same type of routine, and objective performance indicators of the clinician performing routine 300.
[0029] Reference again Figure 3 In a parameterized medical routine record 200, data records AG (e.g., data records 430-456) have been associated with actions 244, 246, 248 performed during the medical routine. These actions have been associated with parameters 250, 252, 254, 256 by, for example, the parameterized method described in U.S. Provisional Patent Application No. 63 / 320,538. In this example, the actions and parameters have optionally been further grouped into segments 240, 242. For example, a routine segment may include a sequence or group of actions associated with ablation, stapling, suturing, dissection, tissue resection, anastomosis, camera control, instrument wrist control, setting change, or tool change, which occurs one or more times during the medical routine. In this example, a routine may include segment 240 and segment 242. Segment 240 may include actions 244 and 246. Action 244 may include two parameters 250 and parameter 252. Parameter 250 is or is determined by a single data record A. Parameter 252 is or is determined by three data records BD. Action 246 may include a single parameter 254 which is or is determined by data record E. Segment 242 includes a single action 248, and action 248 includes a single parameter 256. Parameter 256 is or is determined by data record F and data record G.
[0030] As an example, the routine record 200 may include a segment 240 that is a tissue resection segment and a segment 242 that is an ablation segment. The suturing segment 240 may include the actions of cutting tissue at action 244 and moving the cut tissue at action 246. The cutting action 244 may include parameters 250, including data record A, data record A including identification information of the instrument used to perform the cutting. The cutting action 244 may include parameters 252, including data record B, data record C, and data record D, data record B including the position and orientation of the end effector of the instrument performing the cutting at the beginning of the cutting, data record C including the position and orientation of the end effector of the instrument performing the cutting at the end of the cutting, and data record D including the duration between the beginning and the end of the cutting. The tissue moving action 246 may include parameters 254, including data record E, data record E including the distance the tissue is moved. Ablation segment 242 includes a single action 248 of ablating tissue, and single action 248 includes parameters 256 associated with data record F of power level and data record G of duration.
[0031] Reference again Figure 1 , at process 106 , a simulation exercise may be determined based on a user profile and a parameterized catalog, database, or collection of previous routines. Figure 5A An example of a method 500 for customizing a simulation routine is provided. For example, a customized simulation routine may allow a user to experience a difficult case previously performed by an expert. The experience may be customized to include guidance or other assistance based on the user's experience. The experience may also be customized, adjusted, or otherwise adapted to the user's plans for an upcoming routine. In some examples, the experience may be customized based on the user's previous performance judgment to include remedial exercises for correcting performance, or to include increasingly difficult exercises for expanding the user's capabilities. Figure 5A The process shown in can be Figure 5A The method 500 may be performed in a different order than the order shown in the figure, and one or more of the processes shown may not be performed in some embodiments of the method 500. Figure 5A One or more processes not explicitly shown in the method 500 may be included before, after, in between, or as part of the process shown. In some embodiments, one or more processes of method 500 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which, when executed by one or more processors (e.g., processors of a control system), may cause the one or more processors to perform the one or more processes. Figure 1Process 106 may be manual, in which case a clinician or (one or more) other human operators determine the simulated exercises; may be fully automated, in which case a control system determines the simulated exercises; or may be semi-automated, in which case certain portions of process 106 are performed by a human operator and other portions are performed by a control system.
[0032] At process 502, user experience factors may be accessed or otherwise determined. User experience factors may include skill development topics for users to improve their clinical skills, or may include the types of future routines scheduled for the user to allow the user to prepare for future experiences. Additionally or alternatively, experience factors may include factors associated with past or upcoming routines, such as the sequence of routine workflow steps or anatomical parameters, such as patient size or condition. Experience factors may be accessed or determined based on information in a user profile, information received from a user or other operator (e.g., a trainer), or any other source of information regarding skill development needs.
[0033] In an example where the user experience factor includes one or more topics or areas for skill development, the experience factor can be determined based on the user profile and / or parameterized previous routine information. For example, the previous routine information 152 from the user profile 150 can be compared with the surgical objective performance indicators 458 from one or more previous routines 300 to judge the user's skill level and identify the required skill development areas. For example, the user skill metric can be compared with the objective performance indicator or performance benchmark to determine weak areas or opportunities to strengthen skills. For example, a benchmark or indicator for a technique of tissue ablation can include a measurement of the amount of smoke produced, a measurement of the amount of tissue color change, a measurement of the time to complete the routine, and / or a measurement of the number of clamping actions performed. The user's skill metric compared to these benchmark skills can indicate that the user should complete simulation training to improve ablation skills. As another example, the user's skill metric can be compared to the skills required for the upcoming routine type based on the user's plan for the upcoming or expected routine. Comparing the user's skill metric with the skills required for the upcoming routine type can indicate that the user should complete simulation training to acquire or improve the skills required for those upcoming routine types. As another example, a user skill metric may be compared to the difficulty of an upcoming routine based on the user's plan for the upcoming routine. Comparing the user's skill metric to the difficulty of the upcoming routine may indicate that the user should complete simulation training to obtain or improve the skills required for the upcoming difficulty level. As another example, the user's patient history may be compared to the characteristics of the patient scheduled for the upcoming routine. Comparing the patient history to the characteristics of the patient scheduled for the upcoming routine type may indicate that the user should complete simulation training to obtain or improve the skills required to match the scheduled patient size, gender, pathology type, comorbidities, or other characteristics. As another example, the user's team composition history may be compared to the team scheduled for the user's upcoming routine. Comparing the user's team composition history to the team composition, skill level, and experience level of the upcoming routine may indicate that the user should complete simulation training to obtain or improve the skills required to work effectively with the planned team. As another example, the user's skill metric may be evaluated to determine the next level of skills for incremental skill growth. As another example, the user may self-identify areas that require growth or improvement based on the user's profile or based on parameterized previous routines. As another example, a trainer, key operations leader, expert, or other mentor figure may identify areas for growth or improvement for a user based on a profile of the user or based on parameterized prior routines.
[0034] At process 504, one or more parameterized previous routines may be identified from the set of parameterized previous routines to address the identified skill development needs. For example, if the upcoming routine involves a patient with a potential condition that the user has not previously experienced, a parameterized previous routine involving a real or simulated patient with the same potential condition may be identified. As another example, if the upcoming routine involves the use of a new robotic-assisted medical system that is different from the system previously used by the user, a parameterized previous routine using the new system may be identified. In some examples, the parameterized previous routine may be a routine associated with an efficient, minimally error-prone, or otherwise successful previous routine of an expert, key operational leader, trainer, peer, or even user. In some examples, the parameterized previous routine may be a routine of a user or other person that includes errors, suboptimal performance, inefficiency, or other problems that may be studied or exited to improve results.
[0035] At optional process 506, various aspects of the identified parameterized previous routine may be modified. For example, segments, actions, or parameters from the identified previous routine may be omitted or combined with segments, actions, or parameters from other identified previous routines. In some examples, the identified parameterized previous routine may be adjusted based on patient size, gender, age, or other conditions corresponding to the user's skill development needs. In some examples, the identified parameterized previous routine may be modified to allow additional team members to participate in the simulation. In some examples, the identified parameterized previous routine may be modified to include synthetic team members. In some examples, port placement for a parameterized previous robotic-assisted medical routine may be modified. In some examples, the identified parameterized previous routine may be modified to include different systems, devices, or instruments.
[0036] At process 508, a simulation exercise can be generated. The simulation exercise can allow a user to virtually experience a parameterized previous routine, including a simulated user interface that provides a visual, audio, and / or tactile simulation of the identified previous routine. Based on the data record obtained during the previous routine, the simulation exercise can include some or all parameters from the parameterized previous routine. The simulation exercise can simulate a robot-assisted medical system and instrument, a laparoscopic instrument, and / or an open routine instrument used in the identified previous routine. Image data, audio data, force transmission data, patient monitoring data, and / or patient outcome data recorded or collected and parameterized from the identified previous routine can be included in the simulation exercise. In some examples, image data, audio data, force transmission data, patient monitoring data, and / or patient outcome data can be artificially generated and included in the simulation exercise to create a synthetic or mixed synthetic recording environment and experience. The simulation exercise can be interactive and responsive to user input. In some examples, the simulation exercise can be presented to the user at a simulation user console, which includes a user interface component of an operator input system (e.g., an operator input system 616), which includes a display system, an audio system, and a user input control device. In some examples, the simulation exercise can be presented to the user at an actual user console (e.g., an operator input system 616) operating in simulation mode. In some examples, the simulation exercise can be suitable for being presented to the user on a laptop, tablet computer, phone, or other user input device, which may include a display, a user input control device, a control system, a memory, and / or other components that support visual, audio, and / or tactile user experience. In some examples, the simulation can be dynamically adjusted based on user preferences. For example, after starting the simulation with a first instrument used by a first guiding clinician in a first previous routine, the user can choose to change the simulation to use a second instrument used by a second guiding clinician in a second previous routine. In some examples, the simulation can include an inanimate anatomical model or a synthetic tissue model customized and constructed for the simulation. For example, a synthetic model can include a customized anatomical defect or a customized instrument port placement relative to the defect.
[0037] At optional process 510, the model operator input for performing simulation exercise can be determined according to parameterized previous routine. The model operator input can include hand, arm, body, eye, head, foot or other motion or behavior, which is used to generate the data record based on the parameters of simulation exercise or otherwise associated with these data records. For example, the parameter associated with the action of suturing tissue in the previous routine is determined according to the data record associated with a series of steps of performing suturing action or the routine information record. Therefore, the model operator input associated with suturing based on the parameters of the previous routine can include: select the instrument with the same characteristics to be used to grab the suture filament, select the same manipulator arm used to control the instrument, apply the same or similar force to grab the filament, rotate the wrist joint of the instrument by the same or similar amount, complete the rotation in the same or similar duration, release the filament at the same or similar position and orientation of the instrument. The model operator input can generate the data record identical with the corresponding data record of the previous routine or within the predetermined range of the corresponding data record. In some examples, user input in a simulation may be evaluated against or compared to model operator input to provide guidance, error reporting, success confirmation reporting, or other indications that the user input is the same, similar, or different than the model operator input.
[0038] At optional process 512, instructions for performing model inputs may be generated. For example, the instructions may include a simulation graphic for visual display to a user during a simulation exercise to explain or demonstrate the model inputs. The simulation graphic may include a ghosting tool illustration. The instructions may also include tactile forces or audio instructions delivered to an operator input device. The instructions may include a hand handover demonstration that allows the user to follow the instructed hand movements. The instructions may include pop-up graphics, text information, and / or highlighted or emphasized graphics. The instructions may include a graphic indicator, a picture-in-picture display, and / or a composite or pre-recorded video of an alternative technology. The instructions may include the ability to rewind or fast-forward the instructions. The instructions may be responsive to measured or sensed user input or team member input.
[0039] In some examples, the methods described herein can be used to generate customized camera control simulations. By parameterized previous routines, a camera target sequence (e.g., the position in the field of view where the camera is focused or the orientation of the camera) can be determined, and a simulation can be generated that prompts the user to take actions that follow the same sequence of camera movements. In some examples, the methods described herein can be used to generate customized suturing simulations. By parameterized previous routines, a sequence of needle positions, orientations, and movements can be determined, and a simulation can be generated that guides the user to take actions that follow the same or similar sequence of positions, orientations, and movements. In some examples, the methods described herein can be used to generate customized peeling simulations. By parameterized previous routines, a sequence of instrument positions, orientations, and movements can be determined, and a simulation can be generated that guides the user to take actions that follow the same or similar sequence of positions, orientations, and movements. In some examples, the methods described herein can be used to generate customized energy delivery. By parameterized previous routines, the position and amount of ablation energy can be determined, and a simulation can be generated that teaches the user to deliver energy with the same parameters. In some examples, the methods described herein can be used to perform a setup routine for port placement including robotic-assisted instruments. From the parameterized prior routines, an initial manipulator assembly setup or optimized positions for arm arrangement and / or port placement may be determined, and a simulation may be generated that guides the user to select the same or similar manipulator assembly setup or port placement positions.
[0040] In another example of process 106, Figure 5B Flowchart of illustrates a method 520 for customizing a simulation exercise. In this example, the customized simulation exercise can be based on a current medical routine (e.g., a diagnostic, therapeutic, or surgical routine being performed on a patient in real time). In some examples, the simulation experience can be experienced simultaneously with the current medical routine, or can be experienced after the current medical routine is completed. For example, the simulation experience can allow a clinician performing the current routine to virtually re-experience or repeat a segment of the current routine, or can allow a surgical trainee to virtually experience the segment simultaneously with the current routine.
[0041] As the current medical routine is performed, a parameterized medical routine record (e.g., record 200) of the current routine may be generated in real time. At process 522, a plurality of parameters for the parameterized current routine may be received. For example, the parameters from the medical record of the current routine may include routine information (e.g., routine information 202) or parameters associated with segments and actions of the routine.
[0042] At process 524, the interval of the current routine can be determined. The interval can be determined or identified in any of a variety of ways. For example, the interval can have a predetermined fixed duration relative to a duration trigger (e.g., two minutes of the current routine before engaging a trigger switch on a user interface or control device). In other examples, the interval of the current routine can have a user-defined duration (e.g., the previous 90 seconds before the trigger). In other examples, the interval of the current routine can be marked by the user engaging the trigger switch at the start time and end time of the interval. In other examples, the interval of the current routine can be marked by reversing the video recording of the current routine from the interval end frame to the interval start frame. In other examples, the interval can be determined by image analysis of the trigger action of the tool or tissue in the endoscopic image associated with the start or stop action of the interval. In other examples, the interval can be determined by kinematic identification of a trigger event, such as a motion or command to move a robotic arm or tool attached to a robotic arm.
[0043] At process 526, a simulation exercise can be generated based on multiple parameters associated with the interval of the current routine. The simulation exercise can be an augmented reality or virtual reality simulation that allows a user to experience the interval of the current routine in a simulated environment. For example, a simulation exercise can be provided to a trainee who has viewed the current routine on a secondary operator's console and wishes to now perform the witnessed routine segment in a simulated environment. The simulation exercise can be performed by the trainee while the current routine is continued by the original clinician, or can be performed at a later time (including repeatedly) after the current routine ends. In some examples, the simulation exercise can be provided to the original clinician who performs the current routine after the current routine ends, to allow the clinician to practice the identified part of the routine once or multiple times after the current routine. As an example, if the current routine includes a stripping segment, the interval of the current routine including the stripping segment can be identified. The generated simulation will allow the trainee or the original clinician to virtually experience the identified interval including the stripping segment under the same parameters as the original routine. The virtual experience may include guidance (e.g., tactile guidance, visual guidance) that encourages the user to repeat the same hand movements and tool manipulations as in the original routine. As another example, if the current routine includes camera adjustments, intervals of the current routine that include camera adjustments may be identified. The generated simulation will allow the trainee or original clinician to virtually experience the identified intervals that include camera adjustments under the same parameters as the original routine. The virtual experience may include guidance (e.g., tactile guidance, visual guidance) that encourages the user to repeat the same hand movements and camera manipulations as in the original routine.
[0044] In another example of process 106, Figure 5CFlowchart 530 shows a method for customizing a simulation exercise. In this example, the customized simulation exercise can be based on a current medical routine (e.g., a diagnostic, therapeutic, or surgical routine being performed on a patient in real time), but allows the user to preview an expected or future segment of the current routine. The simulation experience can be experienced simultaneously with the current medical routine, such as during a pause in the current medical routine. For example, the simulation experience can allow a clinician performing the current routine to virtually experience a predictable segment of the current routine determined and modeled by the same or a different clinician based on a previous routine of the same type.
[0045] As the current medical routine is performed, a parameterized medical routine record (e.g., record 200) of the current routine may be generated in real time. At process 532, a plurality of parameters for the parameterized current routine may be received. For example, the parameters from the medical record of the current routine may include routine information (e.g., routine information 202) or parameters associated with segments and actions of the routine.
[0046] At process 534, one or more previous routines corresponding to the plurality of parameters from the parameterized current routine may be identified. The identified previous routines may have the same or similar parameters, including, for example, the same tools used, the same robotic manipulator setup configuration, the same instrument settings, and similar patient characteristics (e.g., BMI, stage of disease progression).
[0047] At process 536, a simulation exercise can be generated for the expected segment of the current routine based on multiple parameters from the current routine and one or more parameterized previous routines. The simulation exercise can be an augmented reality or virtual reality simulation that allows the clinician performing the current routine to experience the expected segment of the current routine in a simulated environment. The simulation exercise can include guidance (e.g., tactile guidance, visual guidance) that encourages the user to practice or repeat the same hand movements and tool manipulations as in the previous routine with similar parameters. In some examples, the simulation exercise can allow the clinician to preview segments where instrument or arm collisions are prone to occur, and train using the previous routine parameters to learn to avoid collisions. In some examples, the simulation exercise can allow the clinician to practice camera movements in the expected upcoming routine segments, which are preferably performed at different camera angles. In some examples, the simulation exercise can allow the clinician to practice tool wrist movements in the expected upcoming segments of the current routine. The simulation exercise can be performed during the pause of the current routine, using the same user control device removed from the instrument follow mode so that the movement of the control device does not activate the surgical tool in the patient. The simulation exercise may allow the user to perform a segment simulation at a time close to the subsequent performance of the actual routine segment, using the same tools, the same setup configuration, and the same patient characteristics.
[0048] The medical routines and simulations described herein may be performed utilizing a variety of manual or robotic-assisted techniques. Figures 6 to 8 Together, an overview of a robotic-assisted medical system 610 is provided, which can be used for medical routines or simulations, including, for example, diagnosis, treatment, or surgery. The medical system 610 is located in a medical environment 611. Figure 6 , medical environment 611 is depicted as an operating room. In other embodiments, medical environment 611 may be an emergency room, a medical training environment, a medical laboratory, or some other type of environment in which any number of medical routines or medical training routines may occur. In other embodiments, medical environment 611 may include an operating room and a control area located outside of the operating room.
[0049] In one or more embodiments, the medical system 610 may be a teleoperated medical system under the teleoperated control of a surgeon. In alternative embodiments, the medical system 610 may be under partial control of a computer programmed to perform a medical routine or subroutine. In still further alternative embodiments, the medical system 610 may be a fully automated medical system under the full control of a computer programmed to perform a medical routine or subroutine using the medical system 610. One example of a medical system 610 that may be used to implement the systems and techniques described in the present disclosure is a da Surgical system, manufactured by Intuitive Surgical Operating Inc., Sunnyvale, California.
[0050] like Figure 6 As shown, the medical system 610 generally includes a component 612, which can be mounted on an operating table O where a patient P is positioned or positioned near the operating table O. The component 612 can be referred to as a patient side cart, a surgical cart, or a surgical robot. In one or more embodiments, the component 612 can be a remotely operated component. The remotely operated component can be referred to as, for example, a manipulation system and / or a remotely operated arm cart. A medical instrument system 614 and an endoscopic imaging system 615 are operably coupled to the component 612. An operator input system 616 allows a surgeon S or other type of clinician to view an image of a surgical site or an image representing a surgical site and to control the operation of the medical instrument system 614 and / or the endoscopic imaging system 615.
[0051] The medical device system 614 may include one or more medical devices. In embodiments where the medical device system 614 includes multiple medical devices, the multiple medical devices may include multiple identical medical devices and / or multiple different medical devices. Similarly, the endoscopic imaging system 615 may include one or more endoscopes. In the case of multiple endoscopes, the multiple endoscopes may include multiple identical endoscopes and / or multiple different endoscopes.
[0052] The operator input system 616 may be located at the surgeon's console, which may be located in the same room as the operating table O. In one or more embodiments, the operator input system 614 may be referred to as a user control system. In some embodiments, the surgeon S and the operator input system 616 may be located in a different room from the patient P or in a completely different building. The operator input system 614 typically includes one or more control devices, which may be referred to as input control devices, for controlling the medical instrument system 614 or the imaging system 615. The control device may include one or more of any number of various input devices, such as a handle, a joystick, a trackball, a data glove, a trigger gun, a foot pedal, a hand controller, a voice recognition device, a touch screen, a body motion or presence sensor, and other types of input devices.
[0053] In some embodiments, the control device(s) will be configured with the same Cartesian degrees of freedom as the medical device(s) of the medical device system 614 to provide telepresence to the surgeon S, which is the feeling that the control device(s) are integral to the instrument, giving the surgeon a strong sense of directly controlling the instrument as if at the surgical site. In other embodiments, the control device(s) may have more or fewer degrees of freedom than the associated medical device and still provide telepresence to the surgeon S. In some embodiments, the control device(s) are manual input devices that move in six degrees of freedom, and they may also include an actuatable handle for actuating the instrument (e.g., for closing the gripping jaw end effector, applying potentials to electrodes, delivering medication, and actuating other types of instruments). Thus, the degrees of freedom and actuation capabilities of the control device(s) are mapped to the available degrees of freedom and range of motion of the medical device(s).
[0054] When the surgeon S views the surgical site through the operator input system 616, the component 612 supports and manipulates the medical instrument system 614. The image of the surgical site can be obtained by the endoscopic imaging system 615, which can be manipulated by the component 612. The component 612 may include multiple endoscopic imaging systems 615, and may also similarly include multiple medical instrument systems 614. The number of medical instrument systems 614 used at one time generally depends on factors such as the diagnostic or surgical routine to be performed and the space limitations in the operating room. The component 612 may include one or more non-servo-controlled links (e.g., one or more links that can be manually positioned and locked in place, generally referred to as manipulator support structures) and the kinematic structure of the manipulator. When the manipulator takes the form of a remotely operated manipulator, the component 612 is a remotely operated component. The component 612 includes multiple motors that drive inputs on the medical instrument system 614. In an embodiment, these motors move in response to commands from a control system (e.g., control system 620). The motor includes a drive system that, when coupled to the medical device system 614, can advance the medical device into a natural or surgically created anatomical hole. Other motor drive systems can move the distal end of the medical device in multiple degrees of freedom, which can include three linear motions (e.g., linear motions along the X, Y, Z Cartesian axes) and three rotational motions (e.g., rotations around the X, Y, Z Cartesian axes). In addition, the motor can be used to actuate an articulated end effector of the medical device for grasping tissue in the jaws of a biopsy device, etc. The medical device of the medical device system 614 may include an end effector having a single working member, such as a surgical scalpel, a blunt blade, an optical fiber, or an electrode. Other end effectors may include, for example, forceps, a grasper, scissors, or a clip applier.
[0055] The medical system 610 also includes a control system 620. The control system 620 includes at least one memory 624 and at least one processor 622 (which can be part of a processing unit) for implementing control between the medical device system 614, the operator input system 616, and other auxiliary systems 626, which may include, for example, imaging systems, audio systems, fluid delivery systems, display systems, lighting systems, steering control systems, irrigation systems, and / or suction systems. Clinicians can travel within the medical environment 611 and, for example, can access the component 612 during a setup routine or view the display of the auxiliary system 626 from the patient's bedside. In some embodiments, the auxiliary system 626 may include a display screen separate from the operator input system 616. In some examples, the display screen may be an independent screen that can be moved around the medical environment 611. The display screen may be oriented so that the surgeon S and one or more other clinicians or assistants can view the display screen simultaneously.
[0056] Despite Figure 6 612, in some embodiments, the control system 620 may be completely contained within the component 611. The control system 620 also includes programmed instructions (e.g., stored on a non-transitory computer-readable medium) to implement some or all of the methods described in accordance with the various aspects disclosed herein. Figure 6 612, another portion of the processing is performed at the operator input system 616, and so on.
[0057] Any of a variety of centralized or distributed data processing architectures may be employed. Similarly, the programmed instructions may be implemented as a plurality of separate programs or subroutines, or they may be integrated into the systems described herein, including a plurality of other aspects of the remote operating system. In one embodiment, the control system supports wireless communication protocols such as Bluetooth, infrared data communication, i.e., IrDA, home radio frequency technology, i.e., HomeRF, IEEE 802.11, digital enhanced wireless communication system, i.e., DECT, and wireless telemetry.
[0058] The control system 620 communicates with a database 627, which can store one or more medical routine records. The database 627 can be stored in the memory 624 and can be dynamically updated. Additionally or alternatively, the database 627 can be stored on a device such as a server or a portable storage device, which can be accessed by the control system 620 via an internal network (e.g., a secure network of a medical institution or a remote operating system provider) or an external network (e.g., the Internet). The database 627 can be distributed throughout two or more locations. For example, the database 627 can exist on multiple devices, which can include devices and / or cloud servers of different entities. Additionally or alternatively, the database 627 can be stored on a portable user distribution device, such as a computer, a mobile device, a smart phone, a laptop computer, an electronic badge, a tablet computer, a pager, and other similar user devices.
[0059] In some embodiments, the control system 620 may include one or more servo controllers that receive force and / or torque feedback from the medical device system 614. In response to the feedback, the servo controller transmits a signal to the operator input system 616. The (one or more) servo controllers may also send signals instructing the assembly 612 to move the (one or more) medical device systems 14 and / or the endoscopic imaging system 615, which extend to an internal surgical site in the patient's body via an opening in the body. Any suitable conventional or dedicated servo controller may be used. The servo controller may be separate from or integrated in the assembly 612. In some embodiments, the servo controller and assembly 612 are provided as part of a teleoperated arm cart positioned adjacent to the patient's body.
[0060] The control system 620 can be coupled to the endoscopic imaging system 615 and can include a processor to process the captured images for subsequent display, such as display to the surgeon on the surgeon's console, or display on another suitable display located locally and / or remotely. For example, in the case of using a stereoscopic endoscope, the control system 620 can process the captured images to present the surgeon with a coordinated stereoscopic image of the surgical site as a field of view image. Such coordination can include alignment between relative images, and can include adjusting the stereo working distance of the stereoscopic endoscope.
[0061] In alternative embodiments, the medical system 610 may include more than one component 612 and / or more than one operator input system 616. The exact number of components 612 will depend on factors such as the surgical routine and space limitations within the operating room. The operator input systems 616 may be collocated, or they may be located in separate locations. Multiple operator input systems 616 allow more than one operator to control one or more components 612 in various combinations. The medical system 610 may also be used to train and practice medical routines.
[0062] Figure 76 is a perspective view of an embodiment of a manipulator assembly 612, which may be referred to as a manipulator system, a patient side cart, a surgical cart, a teleoperated arm cart, or a surgical robot. The assembly 612 shown provides manipulation of three surgical tools 630a, 630b, and 630c (e.g., a medical instrument system 614) and an imaging device 628 (e.g., an endoscopic imaging system 615), such as a stereo endoscope for capturing images of a surgical site. The imaging device 628 may send a signal to the control system 620 via a cable 656. Manipulation is provided by a teleoperated mechanism having multiple joints. The imaging device 628 and the surgical tools 630a-c can be positioned and manipulated through an incision in the patient so that a remote center of motion is maintained at the incision to minimize the size of the incision. The image of the surgical site can include images of the distal ends of the surgical tools 630a-c when they are positioned within the field of view of the imaging device 628. Imaging device 628 and surgical tools 630a-c may each be a therapeutic, diagnostic, or imaging instrument.
[0063] Figure 8 6 is a perspective view of an embodiment of an operator input system 616 at a surgeon's console. In some examples, the operator input system 616 can be a simulated medical system that is decoupled from the controls of the medical instrument or manipulation assembly. The simulated medical system can be coupled to a control system to access a catalog of parameterized previous medical routines, user profiles, or other information used to generate and deliver simulation exercises.
[0064] In this example, the operator input system 16 includes a left eye display 632 and a right eye display 634 for presenting a coordinated stereoscopic view of the surgical environment that enables depth perception to the surgeon S. The left eye display 632 and the right eye display 634 can be components of a display system 635. In other embodiments, the display system 635 can include one or more other types of displays. In some embodiments, the image(s) displayed on the display system 635 can be displayed separately or simultaneously on at least one display screen of the auxiliary system 626.
[0065] The operator input system 616 also includes one or more input control devices 636, which in turn enable the assembly 612 to manipulate one or more instruments of the endoscopic imaging system 615 and / or the medical instrument system 614. The input control devices 636 can provide the same Cartesian degrees of freedom as the instruments with which they are associated to provide the surgeon S with a sense of telepresence, or integration of the input control device 638 with the instruments, so that the surgeon has a strong sense of direct control of the instruments. Thus, the degrees of freedom of each input control device 636 are mapped to the degrees of freedom of the associated instrument of each input control device 636 (e.g., one or more of the instruments of the endoscopic imaging system 615 and / or the medical instrument system 614). To this end, position, force, and tactile feedback sensors (not shown) can be used to transmit position, force, and tactile sensations from the medical instruments, such as surgical tools 630a-c or imaging devices 628, back to the surgeon's hands through the input control devices 636. In addition, the arrangement of the medical instruments can be mapped to the arrangement of the surgeon's hands and the view from the surgeon's eyes, so that the surgeon has a strong sense of direct control of the instruments. The input control device 637 is a foot pedal that receives input from the user's foot.Aspects of the operator input system 616, assembly 612, and auxiliary system 626 may be adjustable and customizable to meet the surgeon's S physical needs, skill level, or preferences.
[0066] In the description, the specific details of describing some embodiments have been set forth. Many specific details have been set forth in order to provide a thorough understanding of these embodiments. However, it will be apparent to those skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to illustrate but are not limited thereto. Those skilled in the art may implement other elements, although not specifically described herein, within the scope and spirit of the present disclosure.
[0067] As long as feasible, the elements described in detail with reference to an embodiment, implementation or application may be optionally included in other embodiments, implementations or applications that are not specifically shown or described. For example, if an element is described in detail with reference to an embodiment, but not described with reference to a second embodiment, the element can still be claimed to be included in the second embodiment. Therefore, in order to avoid unnecessary repetition in the following description, one or more elements shown and described in association with an embodiment, implementation or application may be incorporated into other embodiments, implementations or aspects, unless otherwise specifically described, unless the one or more elements will make the embodiment or implementation inoperative, or unless two or more elements provide conflicting functions. Not all of the processes shown can be executed in all embodiments of the disclosed method. In addition, one or more processes not explicitly shown may be included before, after, in the middle or as part of the process shown. In some embodiments, one or more of the processes may be executed by a control system or implemented at least in part in the form of executable code stored on a non-temporary, tangible, machine-readable medium, and when the executable code is run by one or more processors, one or more of the processes may be executed by one or more processors.
[0068] Any changes and further modifications to the described devices, apparatuses, methods, and any further applications of the principles of the present disclosure are generally contemplated by those skilled in the art to which the present disclosure relates. In addition, the dimensions provided herein are for specific examples, and it is conceivable that different sizes, dimensions, and / or ratios may be used to implement the concepts of the present disclosure. In order to avoid unnecessary repetition of description, one or more components or actions described according to one illustrative embodiment can be used or omitted when applicable to other illustrative embodiments. For the sake of brevity, multiple iterations of these combinations will not be described separately. For simplicity, in some cases, the same reference numerals are used throughout the accompanying drawings to refer to the same or similar parts.
[0069] The systems and methods described herein can be applied to imaging any of various anatomical systems, including lungs, colon, intestines, stomach, liver, kidneys and calyces, brain, heart, circulatory system including vascular system, etc. Although some embodiments of medical routines are provided herein, any reference to medical or surgical instruments and medical or surgical methods is non-restrictive. For example, the instruments, systems and methods described herein can be used for non-medical purposes, including industrial uses, general robotic uses, and sensing or manipulating non-tissue artifacts. Other example applications relate to cosmetic improvements, imaging of human or animal anatomical structures, collecting data from human or animal anatomy, and training medical or non-medical personnel. Additional example applications include routines for tissues removed from human or animal anatomical structures (not returned to human or animal anatomical structures) and routines performed on human or animal corpses. In addition, these technologies can also be used for surgical and non-surgical medical or diagnostic routines.
[0070] One or more elements in the embodiments of the present disclosure may be implemented in software to be executed on a processor of a computer system such as a control processing system. When implemented in software, the elements of the embodiments of the present disclosure may be code snippets for performing various tasks. Routines or code snippets can be stored in a processor-readable storage medium or device, which can be downloaded via a computer data signal contained in a carrier wave on a transmission medium or a communication link. Processor-readable storage devices can include any medium capable of storing information, including optical media, semiconductor media, and / or magnetic media. Examples of processor-readable storage devices include electronic circuits; semiconductor devices, semiconductor memory devices, read-only memories (ROMs), flash memories, erasable programmable read-only memories (EPROMs); floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. Code snippets can be downloaded via computer networks such as the Internet, intranets, etc. Any of a variety of centralized or distributed data processing architectures can be used. Programming instructions can be implemented as multiple separate programs or subroutines, or they can be integrated into multiple other aspects of the system described herein. In some examples, the control system can support wireless communication protocols such as Bluetooth, Infrared Data Association (IrDA), HomeRF, IEEE 802.11, Digital Enhanced Cordless Communications (DECT), Ultra-Wideband (UWB), ZigBee, and wireless telemetry.
[0071] Note that the processes and displays presented may not be inherently related to any particular computer or other device. Various multi-purpose systems may be used with the routines taught herein, or it may prove convenient to construct more specialized devices to perform the described operations. The structures required for various of these systems will appear in the claims as elements. In addition, embodiments of the present invention are described without reference to any particular programming language. It should be understood that a variety of programming languages may be used to implement the teachings of the present invention as described herein.
[0072] The present disclosure describes various apparatuses, parts of apparatuses, and anatomical structures from the perspective of the state of apparatuses in three-dimensional space. As used herein, the term position refers to the position of an object or a part of an object in three-dimensional space (e.g., three translational degrees of freedom along Cartesian x, y, and z coordinates). As used herein, the term orientation refers to the rotational placement of an object or a part of an object (e.g., with one or more rotational degrees of freedom, such as rolling, pitching, and / or yaw). As used herein, the term posture refers to the position of an object or a part of an object on at least one translational degree of freedom, and the orientation of the object or the part of an object on at least one rotational degree of freedom (e.g., up to a total of 6 degrees of freedom). As used herein, the term shape refers to a group of postures, positions, or orientations measured along an object.
[0073] Although certain illustrative embodiments of the present invention have been described and shown in the accompanying drawings, it should be understood that such embodiments are merely illustrative of the broad invention and not restrictive, and that the embodiments of the present invention are not limited to the specific constructions and arrangements shown and described, since various other modifications may be readily conceived by those skilled in the art.
Claims
1. A medical system comprising: Display system; operator input devices; and a control system in communication with the display system and the operator input device, wherein the control system comprises: Processor; and a memory comprising machine-readable instructions that, when executed by the processor, cause the control system to: Access experience factors for users; A collection of reference-parameterized previous routines; identifying, from the set of parameterized previous routines, a parameterized previous routine associated with the experience factor; generating a simulation exercise including a plurality of parameters from said parameterized previous routine; and A model input associated with the plurality of parameters is determined to the operator input device.
2. The medical system according to claim 1, wherein: The experiential factors include the topic of skill development.
3. The medical system according to claim 1, wherein: The experience factors include the type of future routines scheduled for the user.
4. The medical system according to claim 1, wherein: The experience factor is determined based on previous routines performed by the user.
5. The medical system according to claim 1, wherein: The experience factors include anatomical parameters.
6. The medical system according to claim 1, wherein: The experience factor is determined based on the user profile.
7. The medical system according to claim 1, wherein: The machine readable instructions, when executed by the processor, further cause the control system to compare input received at the operator input device to the model input.
8. The medical system according to claim 1, wherein: The machine readable instructions, when executed by the processor, further cause the control system to adjust a previous routine of the parameterization.
9. The medical system according to claim 1, wherein: The machine readable instructions, when executed by the processor, further cause the control system to generate instructions for executing the model inputs.
10. The medical system according to claim 1, wherein: Accessing the experience factors includes comparing the user skill profile to the schedule of expected routines.
11. The medical system according to claim 1, wherein: Accessing the experience factors includes evaluating user skill judgments against objective performance indicators.
12. The medical system according to claim 1, wherein: The set of parameterized previous routines includes parameterized previous routines executed by an operator of the operator input device.
13. The medical system according to claim 1, wherein: The set of parameterized previous routines includes parameterized previous routines executed by the trainer.
14. The medical system of claim 1, wherein: The plurality of parameters are determined from at least one of robotic system records, sensor data, in vivo imaging data, in vitro imaging data, patient monitoring data, or patient outcome records.
15. The medical system of claim 1, wherein: The plurality of parameters may include a position, orientation or movement of a medical device.
16. The medical system of claim 1, wherein: The plurality of parameters may include a position, orientation or movement of an imaging device.
17. The medical system of claim 1, wherein: The plurality of parameters includes a position, orientation, or movement of an operator input device.
18. The medical system of claim 1, wherein: The plurality of parameters includes a position, orientation, or movement of an operator of an operator input device.
19. The medical system of claim 1, wherein: The plurality of parameters may include the position, orientation or movement of team members.
20. The medical system of claim 1, wherein: The plurality of parameters includes an initial manipulator assembly setup arrangement.
21. The medical system of claim 1, wherein: The plurality of parameters includes initial anatomical port placement.
22. The medical system of claim 1, wherein: The plurality of parameters include the location, frequency or amount of energy delivered.
23. A method for generating customized medical simulation exercises, comprising: Access experience factors for users; A collection of reference-parameterized previous routines; identifying, from the set of parameterized previous routines, a parameterized previous routine associated with the experience factor; generating a simulation exercise including a plurality of parameters from said parameterized previous routine; and A model input associated with the plurality of parameters is determined to the operator input device.
24. The method according to claim 23, wherein: The experiential factors include the topic of skill development.
25. The method according to claim 23, wherein: The experience factors include the type of future routines scheduled for the user.
26. The method of claim 23, wherein: The experience factor is determined based on previous routines performed by the user.
27. The method according to claim 23, wherein: The experience factors include anatomical parameters.
28. The method of claim 23, wherein: The experience factor is determined based on the user profile.
29. The method of claim 23, further comprising comparing input received at an operator input device to the model input.
30. The method of claim 23, further comprising adjusting a previous routine of the parameterization.
31. The method of claim 23, further comprising generating instructions for performing the model input.
32. The method of claim 23, wherein: Accessing the experience factors includes comparing the user skill profile to the schedule of expected routines.
33. The method of claim 23, wherein: Accessing the experience factors includes evaluating user skill judgments against objective performance indicators.
34. The method of claim 23, wherein: The set of parameterized previous routines includes parameterized previous routines executed by an operator of the operator input device.
35. The method of claim 23, wherein: The set of parameterized previous routines includes parameterized previous routines executed by the trainer.
36. The method of claim 23, wherein: The plurality of parameters are determined from at least one of robotic system records, sensor data, in vivo imaging data, in vitro imaging data, patient monitoring data, or patient outcome records.
37. The method of claim 23, wherein: The plurality of parameters may include a position, orientation or movement of a medical device.
38. The method of claim 23, wherein the plurality of parameters comprises a position, orientation, or movement of an imaging modality.
39. The method of claim 23, wherein: The plurality of parameters includes a position, orientation, or movement of an operator input device.
40. The method of claim 23, wherein: The plurality of parameters includes a position, orientation, or movement of an operator of an operator input device.
41. The method of claim 23, wherein: The plurality of parameters may include the position, orientation or movement of team members.
42. The method of claim 23, wherein: The plurality of parameters includes an initial manipulator assembly setup arrangement.
43. The method of claim 23, wherein: The plurality of parameters includes initial anatomical port placement.
44. The method of claim 23, wherein: The plurality of parameters include the location, frequency or amount of energy delivered.
45. A method for generating a customized medical simulation exercise, comprising: Receive multiple parameters from the parameterized current routine; determining an interval of the current routine; and A simulated exercise is generated based on a set of parameters of the plurality of parameters associated with the interval of the current routine.
46. A method for generating a customized medical simulation exercise, comprising: Receive multiple parameters from the parameterized current routine; identifying one or more parameterized previous routines corresponding to the plurality of parameters from the parameterized current routine; and A simulated exercise of an intended routine segment for the current routine is generated based on the plurality of parameters from the current routine and one or more parameterized previous routines.
