Systems and methods for parameterizing medical procedures

Abstract

A method may include receiving a plurality of medical procedure records. Each of the plurality of medical procedure records may be generated during a medical procedure performed with the robotically assisted medical system. The method also includes identifying a common set of characteristic actions in the plurality of medical procedure records and determining at least one parameter for each characteristic action in the set of characteristic actions.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 320,538, filed on March 16, 2022, and entitled “SYSTEMS AND METHODS FOR PARAMETERING MEDICAL PROCEDURES,” which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to systems and methods for parameterizing medical procedures. Background Art

[0004] Minimally invasive medical techniques are intended to reduce the amount of tissue damaged during a diagnostic or surgical procedure, thereby reducing patient recovery time, discomfort, and harmful side effects. Such minimally invasive techniques can be performed through one or more surgical incisions or through natural orifices in the patient's anatomy. Through these incisions or natural orifices, a clinician can insert a minimally invasive medical instrument to perform a medical procedure through manual or robotic-assisted instrument actuation. Systems and methods are needed to determine parameters associated with characteristic actions of a medical procedure. Summary of the invention

[0005] Examples of the invention are summarized by the claims following the description. Consistent with some examples, a method may include receiving a plurality of medical procedure records. Each of the plurality of medical procedure records may be generated during a medical procedure performed with a robotic-assisted medical system. The method also includes identifying a set of characteristic actions common to the plurality of medical procedure records and determining at least one parameter for each characteristic action in the set of characteristic actions.

[0006] In some examples, a robotic-assisted medical system may include a manipulator arm, an operator console, a medical instrument coupled to the manipulator arm, and a control system in communication with the operator console and the manipulator arm. The control system may include a processor and a memory, the memory including machine-readable instructions that, when executed by the processor, cause the control system to receive a plurality of medical procedure records and identify a set of characteristic actions common to the plurality of medical procedure records. The instructions may also cause the control system to determine at least one parameter for each characteristic action in the set of characteristic actions.

[0007] In some examples, a method may include receiving a medical procedure record, identifying one or more actions in the medical procedure record, and determining at least one parameter for each of the one or more actions.

[0008] It should be understood that both the above general description and the following detailed description are illustrative and explanatory in nature, and are intended to provide an understanding of the present disclosure without limiting the scope of the present disclosure. In this regard, additional aspects, features and advantages of the present disclosure will be apparent to those skilled in the art through the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of a medical procedure record according to some embodiments.

[0010] Figure 2 is a flow chart illustrating a measurement and recording system for parameterizing medical procedures.

[0011] Figure 3 is a flow chart illustrating a method for parameterizing a medical procedure according to some embodiments.

[0012] Figure 4 is a schematic diagram of a parameterized medical procedure record in accordance with some embodiments.

[0013] Figure 5 is a flow chart illustrating a method for parameterizing a medical procedure according to some embodiments.

[0014] Figure 6 is a schematic diagram of a robot-assisted medical system according to some embodiments.

[0015] Figure 7 is a perspective view of a manipulator assembly according to some embodiments.

[0016] Figure 8 is a perspective view of an operator input system according to some embodiments.

[0017] Examples of the present disclosure and its advantages are best understood by referring to the following detailed description. It should be understood that like reference numerals are used to identify like elements shown in one or more of the accompanying drawings for purposes of illustrating examples of the present disclosure, rather than limiting the present disclosure. DETAILED DESCRIPTION

[0018] Systems and methods for parameterizing medical procedures are provided. From an aggregated medical procedure record, characteristic actions from the medical procedure record can be identified. Parameters associated with the characteristic actions can also be determined from the procedure record. Parameters can be measured or recorded at various aspects of a procedure, including pre-interventional, interventional, and post-interventional phases of the procedure. Measured and recorded parameters for multiple procedures of the same type can be aggregated and associated with procedure outcomes.

[0019] During a medical procedure, the sensor system, measurement system, and / or recording system can capture data records and store them in a medical procedure record. In some examples, the medical procedure record can capture information about a medical procedure performed with a robotic-assisted medical system, a laparoscopic medical system, a manual medical device, or a combination of systems and devices. Each medical procedure record can capture information about a medical procedure performed on a patient by a single clinician or a team of medical professionals. For example, a medical procedure record can be generated from a procedure performed on a patient, during a training period with a synthetic tissue construct, or during a previous computer-generated simulation.

[0020] In some examples, such as Figure 1As shown in the schematic diagram of , the medical procedure record 100 includes procedure information 102 and data record 104 from the medical procedure. The procedure information 102 may include operator information 110, which includes clinician identification information, training history, experience, preferences and / or other information related to one or more clinicians involved in the medical procedure. The operator may be, for example, a surgeon, a surgical intern, a specialist, or a supervising physician. The procedure information 102 may include patient characteristics 112, which include patient age, gender, height, weight, medical history, and / or other information related to the only patient on whom the medical procedure is performed. Patient characteristics may be specific to the procedure, such as tissue inflammation level, abnormal tissue thickness or toughness, hernia size. Patient characteristics may also include abnormalities, such as cysts, additional lymph nodes, and tumor size, location, or complexity. The procedure information may include team characteristics 114 of team members and support staff involved in the medical procedure, which include identification information, roles in the procedure, training history, experience, preferences, or other information related to the person performing the procedure. The procedure information 102 may also include system information 116 about one or more systems used to perform the procedure. These systems may include sensors or recording systems that capture information during the procedure, such as settings or sensed parameters of the system. The system may include a robotic-assisted medical system or components thereof, a laparoscopic medical system, a manual medical device, and / or a support or peripheral system. The system information 116 may include the manufacturer, model, serial number, time of use, time since last maintenance, count of use cycles, maintenance history, calibration information, or other information related to the system used to perform the medical procedure. The procedure information 102 may also include a procedure type 118, which includes information about the type of medical procedure. The medical procedure type may include abdominal, cardiac, colorectal, gynecological, neurological, head / neck, pulmonary, thoracic, urological, or other categories or subcategories of anatomical systems involved in the medical procedure. The procedure information 102 may also include segmentation information 120. The segmentation information 120 may include information about subdivided portions of the medical procedure. For example, a procedure segment may include a sequence or group of actions associated with ablation, stapling, suturing, dissection, tissue resection, stapling, camera control, instrument wrist control, setting change, or tool change occurring one or more times during a medical procedure. In some examples, the procedure information may also include other information about the procedure including, for example, the date the procedure occurred, the time and duration of the procedure, and / or the location and facility identification at which the procedure occurred.

[0021] Medical procedure record 100 also includes data record 104 captured during the medical procedure. In this example, data record 104 may include data record AG. In some examples, such as Figure 2As shown, various measurement and recording systems 202 may be used to document the medical procedure 200 to generate a data record 204 (e.g., data record 104) associated with the procedure 200. The measurement and recording system 202 may include, for example, a robot-assisted medical system (e.g., Figure 6 The measuring and recording system 202 may also or alternatively include one or more sensor systems 210 associated with a monitored medical device used in the medical procedure 200. In some examples, the sensor system 210 may include a position, orientation, motion, and / or displacement sensor system for a manipulator or an instrument coupled to a manipulator of the robotic-assisted medical system. In some examples, the sensor system 210 may include a force sensor system, a clock, a motor encoder, an energy usage sensor, a user eye tracking sensor, or other sensor system that measures and / or records data about the manipulator or instrument. The measuring and recording system 202 may also or alternatively include one or more sensor systems 212 associated with a monitored medical device used in the medical procedure 200. In some examples, the sensor system 212 may track the position, orientation, motion, displacement, force, energy usage, duration of use, and / or other metrics associated with the medical device.

[0022] The measurement and recording system 202 may also or alternatively include one or more imaging systems 214 used during the medical procedure 200. In some examples, the imaging system 214 may be an in vivo imaging system, such as an endoscopic imaging system or an ultrasound imaging system used during the procedure 200. In some examples, the imaging system 214 may be an in vitro imaging system for the patient's anatomy, such as a computed tomography (CT) imaging system, a magnetic resonance imaging (MRI) imaging system, or a functional near infrared spectroscopy (fNIRS) imaging system used during the procedure 200. In some examples, the imaging system 214 may be an environmental imaging system, such as an optical imaging system, that tracks the position and movement of manipulators, instruments, equipment, and / or personnel in the patient's environment during the procedure 200.

[0023] The measurement and recording system 202 may also or alternatively include one or more audio systems 216 used during the medical procedure 200. The audio system may capture and record audio from personnel in the medical area of ​​the procedure 200, operators performing the procedure 200, patients, and / or equipment in the medical area of ​​the procedure. The measurement and recording system 202 may also or alternatively include one or more in-procedure patient monitoring systems 218 used during the medical procedure 200. The patient monitoring systems 218 may include, for example, respiratory, cardiac, blood pressure, anesthesia, insufflation, and / or patient / operating table orientation monitoring systems. The measurement and recording system 202 may also or alternatively include one or more patient outcome recording systems 220 that may be referenced after the procedure 200 is completed. The patient outcome recording systems 220 may record information about post-procedure hospitalization duration, complications, positive results, negative results, mortality, or other post-procedure information about the patient. The measurement and recording system 202 may also or alternatively include one or more procedure skill recording systems 222 that capture and record objective performance indicators of a clinician performing the procedure 200 .

[0024] The data record 204 may include data generated by the measurement and recording system 202. For example, the data record 230 may record the position, orientation, movement, and / or displacement of an instrument (e.g., instrument 614) controlled by a robot-assisted manipulator or manual operation. In some examples, the data record 232 may record the position, orientation, movement, and / or displacement of a robot-assisted manipulator assembly (e.g., 612) including any arm of the manipulator during the procedure 200. In some examples, the data record 234 may record the position, orientation, movement, and / or displacement of an imaging system (such as an endoscope or other in vivo or in vitro imaging system) during the procedure 200. In some examples, the data record 236 may record the position, orientation, movement, and / or displacement of an operator input device (e.g., 636) during the procedure 200. In some examples, the data record 238 may record the position, orientation, movement, and / or displacement of an operator (e.g., surgeon S) directing instrument control during the procedure 200. For example, the data record 238 may record the movement of the operator's hand or track the detachment of the head from the operator console. In some examples, data record 240 may record the position, orientation, movement, and / or displacement of one or more members of the medical team associated with procedure 200. In some examples, data record 242 may record various aspects of the initial setup of procedure 200, including the position and arrangement of robot-assisted manipulators, patient port placement, and the position of peripheral equipment. In some examples, data record 244 may include a record of the position, frequency, and amount of energy provided to or delivered by an instrument (e.g., an ablation instrument) during procedure 200. In some examples, data record 246 may include a record of instrument changes during procedure 200. In some examples, data record 248 may include a time-based record that captures dwell time, idle time, and / or duration or speed of actions during procedure 200. In some examples, data record 250 may capture various aspects of the workflow, including the number and / or sequence of actions during procedure 200. For example, data record 250 may include a sequence of positions, orientations, movements, and / or displacements associated with discrete activities. In some examples, data record 252 can capture errors, difficulties, unexpected events, or other unplanned events during procedure 200, such as manipulator arm collisions, conditions that led to conversion of surgery from robotic-assisted surgery to open surgery during the procedure, conditions that led to conversion from robotic-assisted surgery to laparoscopic surgery during the procedure, or conditions that led to conversion from laparoscopic surgery to open surgery during the procedure. In some examples, data record 254 can capture various aspects of the anatomical environment, including the size of organs, incisions, and / or treatment delivery areas. Other aspects of the anatomical environment that can be recorded include pelvic width, distances between anatomical structures, and / or the location of the vasculature.In some examples, data record 256 may include measures of interventional consequences, such as bleeding, smoke, tissue movement, and / or tissue color change. In some examples, data record 258 may include a catalog of key skills for performing procedure 200, associated target performance indicators for experienced clinicians performing the same type of procedure, and target performance indicators for clinicians performing procedure 200.

[0025] Figure 3 A method 300 is shown for parameterizing a medical procedure. The method 300 is shown as a set of operations or processes. Figure 3 The process shown in Figure 3 The method 300 may be performed in a different order than the order shown in the figure, and one or more of the processes shown may not be performed in some embodiments of the method 300. Figure 3 One or more processes not explicitly shown in the method 300 may be included before, after, in between, or as part of the process shown. In some embodiments, one or more of the processes of method 300 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which instructions, when executed by one or more processors (e.g., processors of a control system), may cause the one or more processors to perform one or more of the processes.

[0026] At process 302, a medical procedure record can be received. For example, the medical procedure record can be received at a processor (e.g., processor 620 of medical system 610). In some examples, the medical procedure record can be received from a memory device (e.g., memory 624 of medical system 610). The received medical record can be, for example, medical record 100.

[0027] In process 304, one or more actions in a medical procedure can be identified by analyzing a medical procedure record (e.g., procedure information 102 and / or data record 104). In some examples, the identification of the action can be determined by computer analysis. Additionally or alternatively, a human analyst can identify one or more actions. In some examples, the action of the procedure can include a single action, such as grabbing tissue with a clamping instrument. In some examples, the action of the procedure can include a series of actions or behaviors. For example, the action of creating a suture silk may involve clamping the suture silk with an instrument, rotating the wrist joint of the instrument that clamps the suture silk, and releasing the movement of the suture silk.

[0028] At least one parameter can be determined for each action of the medical procedure at process 306. For example, the parameter can be determined from one or more of a data record (e.g., data records 104, 204) or a procedure information record (e.g., procedure information 102) from a medical procedure record. Multiple parameters can be determined for each action, and thus a medical procedure with multiple actions can include many parameters. In some examples, parameters associated with an action of grasping tissue can include the identity of an instrument used to grasp the tissue, the identity of a manipulator arm used to control the instrument, the force applied by the instrument, the duration of application of the force, and / or a measure of the consequences of the intervention such as a change in tissue color.

[0029] In some examples, parameters associated with the action of suturing tissue can be determined from a data record or a procedural information record associated with a series of steps in performing the suturing action. For example, the parameters associated with suturing can include the identity of the instrument used to grab the suture wire, the identity of the manipulator arm used to control the instrument, the force applied to grab the wire, the rotation of the instrument wrist, the duration of the rotation, the position and orientation of the instrument when the wire is released, and / or a measure of the consequences of intervention such as bleeding.

[0030] In some examples, parameters associated with the act of delivering energy to tissue can be determined from a data record or a procedure information record associated with the act of delivering energy. For example, parameters associated with delivering energy to tissue can include the identity of an instrument used to ablate tissue, the identity of a manipulator arm used to control the ablation instrument, a measure of energy delivered to tissue, movement of the instrument, duration to complete ablation, number of locations to deliver energy, and / or measures of interventional consequences such as smoke generated during the act.

[0031] In some examples, parameters associated with the action of the camera control can be determined from a data record or a protocol information record associated with the action of the camera control or the camera position and orientation. For example, parameters associated with the camera control can include the movement of other instruments immediately before or after the endoscope is moved (e.g., the instrument movement can indicate why a change in the camera is needed), a change in the position and / or orientation of the distal end of the endoscope, and / or a change in the focus of the endoscope.

[0032] In some examples, parameters associated with the action of the instrument movement can be determined from a data record or a procedure information record associated with the instrument movement. For example, parameters associated with moving the instrument can include a measure of the position, orientation, speed, and / or displacement of the instrument end effector. Parameters can also include the position, orientation, speed, and / or displacement of a manipulator arm coupled to the instrument. Parameters can also include error indicators or other measures of arm collisions.

[0033] In some examples, parameters associated with the operator's actions can be determined from data records or procedure information records associated with the operator console. For example, parameters associated with operator engagement can include data records associated with eye gaze, head engagement with the operator console, duration of performing an action, movement of the hand and / or operator input device used to control instrument movement.

[0034] In some examples, the determined parameters may be clinically relevant parameters such as duration or motion data that provide information about the speed or efficiency of the motion and may provide an indication of operator skill. In some examples, the determined parameters may be demarcation parameters that provide an indication of a stage or point in a procedure. For example, a data record of a foot pedal input may be a demarcation parameter indicating the start of a camera control action in which the angle or position of the endoscope is adjusted. A subsequent second data record of a foot pedal input may be a demarcation parameter indicating the end of the camera control action and a return to instrument follow mode.

[0035] In optional process 308, the determined parameters can be used for various purposes, including training, evaluation, procedure improvement evaluation or other development or analysis purposes. For example, the determined parameters can be used to conduct training procedures to strengthen the operator's skill capabilities. In some examples, one or more parameterized procedures can be used to create simulation exercises that meet the development needs of clinicians. The simulation exercise can be customized, scaled or otherwise adapted to the schedule of the operator's upcoming procedures. In some examples, the experience can be customized to include remedial exercises to correct performance, or include increasingly difficult exercises to expand the user's capabilities. The simulation exercise can allow the user to virtually experience one or more aspects of a single parameterized procedure or a mixture of more than one parameterized procedure. The simulation exercise can include a simulation user interface that provides a visual, audio and / or tactile simulation of the procedure. Based on the data records obtained during the procedure, the simulation exercise can include some or all parameters from the parameterized procedure. The simulation exercise can simulate the robot-assisted medical system and instruments, laparoscopic instruments and / or open procedure instruments used in the parameterized procedure. Image data, audio data, force transfer data, patient monitoring data and / or patient outcome data recorded or collected and parameterized from the procedure may be included in the simulation exercise. In some examples, image data, audio data, force transfer data, patient monitoring data and / or patient outcome data may be artificially generated and included in the simulation exercise to create an environment and experience of a synthetic or mixed synthetic record. The simulation exercise may be interactive and responsive to user input. In some examples, the simulation exercise may be presented to the user at a simulated user console, which includes a user interface component of an operator input system (e.g., operator input system 616), which includes a display system, an audio system and a user input control device. In some examples, the simulation exercise may be presented to the user at an actual user console (e.g., operator input system 616) operating in a simulated mode. In some examples, the simulation exercise may be suitable for presentation to the user on a laptop, tablet computer, phone or other user input device, which may include a display, a user input control device, a control system, a memory and / or other components supporting a visual, audio and / or tactile user experience. In some examples, the simulation may be dynamically adapted based on user preferences. In some examples, the simulation can include an inanimate anatomical model or a synthetic tissue model that is customized and constructed for the simulation. For example, the synthetic model can include a customized anatomical defect or a customized instrument port location relative to the defect.

[0036] In some examples, the determined parameters can be used to evaluate operators. Such evaluation can include comparing comparable parameters from different users, experts, or standards for the same or similar actions or procedures. In some examples, the determined parameters can be used to identify inefficiencies, delays, equipment issues, environmental issues, surgical team issues, or other issues that can be addressed to improve future procedures.

[0037] Figure 4 4 is a schematic diagram of a parameterized medical procedure record 400 based on an initial medical procedure record 100. The parameterized medical procedure record 400 may include procedure information 102 and a data record AG from a medical procedure. In the parameterized medical procedure record 400, the data record AG has been associated with actions 402, 404, 406 and parameters 410, 412, 414, 416 by the parameterization method 300. In this example, the actions and parameters are optionally further divided into segments 420, 422. For example, a surgical segment may include a sequence or group of actions associated with ablation, pinning, suturing, dissection, tissue resection, anastomosis, camera control, instrument wrist control, setting changes, or tool changes that occur one or more times during a medical procedure. In this example, the procedure may include segments 420 and 422. Segment 420 may include actions 402 and 404. Action 402 may include two parameters 410 and 412. Parameter 410 is a single data record A or is determined from it. Parameter 412 is or is determined from three data records BD. Action 404 may include a single parameter 414, which is or is determined from data record E. Segment 422 includes a single action 406, and action 406 includes a single parameter 416. Parameter 413 is or is determined from data record F and data record G.

[0038] In some examples, the procedure record 400 may include a segment 420 as a tissue resection segment and a segment 422 as an ablation segment. The suturing segment 420 may include the actions of cutting tissue in action 402 and moving the cut tissue in action 404. The cutting action 402 may include parameters 410, which include data record A, which includes identification information of the cutting instrument. The cutting action 402 may include parameters 412, which include data record B, data record C, and data record D, where data record B includes the position and orientation of the end effector of the cutting instrument at the beginning of the cutting, data record C includes the position and orientation of the end effector of the cutting instrument at the end of the cutting, and data record D includes the duration between the beginning and the end of the cutting. The tissue movement action 404 may include parameters 414, which include data record E, which includes the distance the tissue is moved. The ablation segment 422 includes a single action 406 of ablating tissue, which includes parameters 416 associated with data record F of power level and data record G of duration.

[0039] Figure 5 A method 500 is shown for parameterizing a medical procedure type from a plurality of medical procedures. The method 500 is shown as a set of operations or processes. Figure 5 The process shown in Figure 5 The method 500 may be performed in a different order than the order shown in the figure, and one or more of the processes shown may not be performed in some embodiments of the method 500. Figure 5 One or more processes not explicitly shown in the method 500 may be included before, after, in between, or as part of the process shown. In some embodiments, one or more of the processes of method 500 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which, when executed by one or more processors (e.g., a processor of a control system), may cause the one or more processors to perform one or more of the processes.

[0040] At process 502, a plurality of medical procedure records may be received. For example, the medical procedure records may be received at a processor (e.g., processor 620 of medical system 610). In some examples, the medical procedure records may be received from a memory device (e.g., memory 624 of medical system 610). In some examples, as few as one medical record (e.g., medical record 100) may be received, but in other examples, tens, hundreds, thousands, or millions of medical procedure records including procedure information and data records may be received.

[0041] In process 504, a set of characteristic actions common to multiple medical procedure records may be identified by analyzing procedure information (e.g., procedure information 102), data records (e.g., data record 104), segmentation information (e.g., segmentation information 120, 420, 422), action information (e.g., actions 402, 404, 406) from multiple medical procedure records, or parameter information (e.g., parameters 410, 412, 414, 416) from multiple medical procedure records. Characteristic actions may include actions of instruments or instrument categories, instrument manipulators or instrument manipulator categories, operator console components or operator console component categories, or other systems, devices, or components that are common to multiple medical procedure records. For example, a retractor instrument may be a category or type of instrument utilized within one or more of the multiple medical procedure records, although not the exact instrument. Within multiple medical records, a characteristic action may include a single characteristic action, such as grasping tissue with a gripping instrument. Additionally or alternatively, a characteristic action may include a sequence of actions or behaviors. For example, a common characteristic action found in multiple medical procedure records may be an action to create a stitch, which may involve clamping a suture wire with an instrument, rotating the wrist joint of the instrument that clamps the suture wire, and releasing the suture wire. In other examples, characteristic actions may include, but are not limited to, clamping the jaws of an end effector, delivering energy through an ablation instrument, changing the posture of an endoscopic imaging instrument, or moving an operator input device at an operator console.

[0042] At least one parameter may be determined for each characteristic action in the set of characteristic actions at process 506. For example, the parameter may be determined from one or more of a data record (e.g., data record 104, 204) or a procedure information record (e.g., procedure information 202) from a plurality of medical procedure records.

[0043] In some examples, parameters associated with characteristic actions of grasping tissue may include identification of various instruments used to grasp tissue in multiple medical procedure records, identification of manipulator arms used to control the instruments in multiple medical procedure records, forces applied by the instruments in multiple medical procedure records, duration of applied forces, and / or measures of consequences of the intervention, such as tissue color changes in multiple medical procedure records.

[0044] In some examples, parameters associated with the characteristic action of suturing tissue can be determined from a data record or a procedure information record associated with a series of steps in performing the suturing action. For example, the parameters associated with suturing can include the identification of the instrument used to grab the suture wire in multiple medical procedure records, the identification of the manipulator arm used to control the instrument in multiple medical procedure records, the force applied to grab the wire, the rotation of the wrist joint of the instrument in multiple medical procedure records, the duration of the rotation to complete in multiple medical procedure records, the position and orientation of the instrument when releasing the wire in multiple medical procedure records, and / or the measurement of the consequences of intervention such as bleeding in multiple medical procedure records.

[0045] In some examples, parameters associated with the act of delivering energy to tissue can be determined from a data record or a procedure information record related to the characteristic act of delivering energy. For example, parameters associated with delivering energy to tissue can include identification of an instrument used to ablate tissue in multiple medical procedure records, identification of a manipulator arm used to control the ablation instrument in multiple medical procedure records, measurements of energy delivered to tissue in multiple medical procedure records, movement of the instrument, time to complete ablation in multiple medical procedure records, number of locations where energy was delivered in multiple medical procedure records, and / or measurements of consequences of intervention such as smoke generated during the act in multiple medical procedure records.

[0046] In some examples, parameters associated with a characteristic action of camera control can be determined from a data record or a procedure information record associated with the camera control action. For example, parameters associated with camera control can include movement of other instruments immediately before or after movement of the endoscope in multiple medical procedure records, changes in the position and / or orientation of the distal end of the endoscope in multiple medical procedure records, and / or changes in the focus of the endoscope in multiple medical procedure records.

[0047] In some examples, parameters associated with characteristic actions of the instrument movement can be determined from a data record or a procedure information record associated with the instrument movement. For example, parameters associated with moving an instrument can include measurements of the position, orientation, velocity, and / or displacement of an end effector of the instrument in multiple medical procedure records. Parameters can also include the position, orientation, velocity, and / or displacement of a manipulator arm coupled to the instrument in multiple medical procedure records. Parameters can also include error indicators or other measurements of arm collisions in multiple medical procedure records.

[0048] In some examples, parameters associated with characteristic actions of an operator can be determined from data records associated with an operator console or a protocol information record. For example, parameters associated with operator engagement can include data records associated with eye gaze, head engagement with an operator console, duration of performing an action, hand used to control instrument movement, and / or movement of an operator input device in multiple medical protocol records.

[0049] In some examples, the determined parameters may be clinically relevant parameters, such as duration or motion data, which provide information about the speed or efficiency of the motion and may provide an indication of operator skill. In some examples, the determined parameters may be demarcation parameters that provide an indication of a stage or point in a procedure.

[0050] With the identification of characteristic actions and the parameterization of medical procedures, medical procedures, segments or actions can optionally be evaluated for quality, efficiency, patient outcomes or other evaluation criteria. For example, in optional process 508, the determined parameters can be related to outcomes or performance criteria. For example, parameters associated with low patient morbidity, low procedure duration, minimizing camera posture changes, minimizing surgical attendant intervention, minimizing manipulator arm collisions or quality markings can be identified through multiple medical procedure records. For example, these parameters can be used to evaluate and train operators and / or develop new or improved procedures. As described in International Application No. PCT / US2023 / 064324[P06470-WO], which is incorporated herein by reference in its entirety, parameterized medical procedures can be referenced in a medical skills development system to generate customized medical simulations.

[0051] The medical procedures described herein may be performed using a variety of manual or robotic-assisted techniques. Figures 6 to 8 Together, an overview of a robotic-assisted medical system 610 is provided, which can be used for medical procedures including, for example, diagnostic, therapeutic, or surgical procedures. The medical system 610 is located in a medical environment 611. Figure 6 , medical environment 611 is depicted as an operating room. In other embodiments, medical environment 611 may be an emergency room, a medical training environment, a medical laboratory, or some other type of environment in which any number of medical procedures or medical training procedures may be performed. In other embodiments, medical environment 611 may include an operating room and a control area located outside of the operating room.

[0052] In one or more embodiments, the medical system 610 may be a teleoperated controlled medical system under the teleoperated control of a surgeon. In alternative embodiments, the medical system 610 may be under partial control of a computer that is programmed to perform a medical procedure or sub-procedure. In other alternative embodiments, the medical system 610 may be a fully automated medical system under the full control of a computer that is programmed to perform a medical procedure or sub-procedure with the medical system 610. An example of a medical system 610 that may be used to implement the systems and techniques described in the present disclosure is the da Surgical system.

[0053] like Figure 6 As shown, the medical system 610 generally includes a component 612, which can be mounted on or positioned near an operating table O on which a patient P is positioned. The component 612 can be referred to as a patient side cart, a surgical cart, or a surgical robot. In one or more embodiments, the component 612 can be a teleoperated component. The teleoperated component can be referred to as, for example, a manipulation system and / or a teleoperated arm cart. A medical instrument system 614 and an endoscopic imaging system 615 are operably coupled to the component 612. An operator input system 616 allows a surgeon S or other type of clinician to view an image of the surgical site or an image representing the surgical site and to control the operation of the medical instrument system 614 and / or the endoscopic imaging system 615.

[0054] The medical device system 614 may include one or more medical devices. In embodiments where the medical device system 614 includes multiple medical devices, the multiple medical devices may include multiple identical medical devices and / or multiple different medical devices. Similarly, the endoscopic imaging system 615 may include one or more endoscopes. In the case of multiple endoscopes, the multiple endoscopes may include multiple identical endoscopes and / or multiple different endoscopes.

[0055] The operator input system 616 may be located at a surgeon's console, which may be located in the same room as the operating table O. In one or more embodiments, the operator input system 616 may be referred to as a user control system. In some embodiments, the surgeon S and the operator input system 616 may be located in a different room than the patient P or in a completely different building. The operator input system 616 typically includes one or more control devices, which may be referred to as input control devices, for controlling the medical instrument system 614 or the imaging system 615. The control devices may include one or more of any number of various input devices, such as a handle, a joystick, a trackball, a data glove, a trigger gun, a foot pedal, a manual controller, a voice recognition device, a touch screen, a body motion or presence sensor, and other types of input devices.

[0056] In some embodiments, the control device will have the same Cartesian degrees of freedom as the medical devices of the medical device system 614 to provide telepresence to the surgeon S, which is the sense that the control device is integrated with the device so that the surgeon has a strong sense of direct control of the device, as if present at the surgical site. In other embodiments, the control device may have more or fewer degrees of freedom than the associated medical device and still provide telepresence to the surgeon S. In some embodiments, the control device is a manual input device that moves in six degrees of freedom, and it may also include an actuatable handle for actuating the device (e.g., for closing the gripper end effector, applying potentials to electrodes, delivering drug therapy, and actuating other types of devices). Therefore, the degrees of freedom and actuation capabilities of the control device are mapped to the degrees of freedom and range of motion available to the medical device.

[0057] When the surgeon S views the surgical site through the operator input system 616, the assembly 612 supports and manipulates the medical instrument system 614. Images of the surgical site can be obtained by an endoscopic imaging system 615, which can be manipulated by the assembly 612. The assembly 612 may include multiple endoscopic imaging systems 615, and may also similarly include multiple medical instrument systems 614. The number of medical instrument systems 614 used at one time generally depends on factors such as the diagnostic or surgical procedures to be performed and the space limitations in the operating room. The assembly 612 may include one or more non-servo-controlled links (e.g., one or more links that can be manually positioned and locked in place, generally referred to as manipulator support structures) and the kinematic structure of the manipulator. When the manipulator takes the form of a teleoperated manipulator, the assembly 612 is a teleoperated assembly. The assembly 612 includes multiple motors that drive inputs on the medical instrument system 614. In an embodiment, these motors move in response to commands from a control system (e.g., control system 620). The motor includes a drive system that, when coupled to the medical instrument system 614, can advance the medical instrument into a natural or surgically created anatomical hole. Other electric drive systems can move the distal end of the medical device in multiple degrees of freedom, which can include three degrees of freedom for linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and three degrees of freedom for rotational motion (e.g., rotation around the X, Y, and Z Cartesian axes). In addition, the motor can be used to actuate an articulated end effector of the medical device for grasping tissue in the jaws of a biopsy device, etc. The medical device of the medical device system 614 can include an end effector having a single working member, such as a scalpel, a blunt blade, an optical fiber, or an electrode. Other end effectors can include, for example, forceps, a grasper, scissors, or a clip applier.

[0058] The medical system 10 also includes a control system 620. The control system 620 includes at least one memory 624 and at least one processor 622 (which can be part of a processing unit) for implementing control between the medical device system 614, the operator input system 616, and other auxiliary systems 626, which may include, for example, imaging systems, audio systems, fluid delivery systems, display systems, lighting systems, steering control systems, flushing systems, and / or suction systems. The clinician can circulate within the medical environment 611 and can access, for example, the component 612 during the setting procedure, or view the display of the auxiliary system 626 from the patient's bedside. In some embodiments, the auxiliary system 626 may include a display screen separate from the operator input system 616. In some examples, the display screen may be an independent screen that can be moved around the medical environment 611. The display screen may be oriented so that the surgeon S and one or more other clinicians or assistants can view the display screen at the same time.

[0059] Despite Figure 6 612, in some embodiments, the control system 620 may be completely contained within the component 612. The control system 620 also includes programming instructions (e.g., stored on a non-transitory computer-readable medium) to implement some or all of the methods described in accordance with aspects disclosed herein. Figure 6 612, another portion of the processing is performed at the operator input system 616, and so on.

[0060] Any of a variety of centralized or distributed data processing architectures may be employed. Similarly, the programming instructions may be implemented as a plurality of separate programs or subroutines, or they may be integrated into a variety of other aspects of the systems described herein, including teleoperation systems. In one embodiment, the control system 620 supports wireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and wireless telemetry.

[0061] The control system 620 communicates with a database 627, which can store one or more medical procedure records. The database 627 can be stored in the memory 624 and can be dynamically updated. Additionally or alternatively, the database 627 can be stored on a device such as a server or a portable storage device, which can be accessed by the control system 620 via an internal network (e.g., a secure network of a medical facility or teleoperation system provider) or an external network (e.g., the Internet). The database 627 can be distributed throughout two or more locations. For example, the database 627 can exist on multiple devices, which can include devices and / or cloud servers of different entities. Additionally or alternatively, the database 627 can be stored on a portable user signature device, such as a computer, a mobile device, a smart phone, a laptop computer, an electronic badge, a tablet computer, a pager, and other similar user devices.

[0062] In some embodiments, the control system 620 may include one or more servo controllers that receive force and / or torque feedback from the medical device system 614. In response to the feedback, the servo controller transmits a signal to the operator input system 616. The servo controller may also transmit a signal that instructs the assembly 612 to move the medical device system 14 and / or the endoscopic imaging system 615, which extend to an internal surgical site within the patient's body via an opening in the body. Any suitable conventional or specialized servo controller may be used. The servo controller may be separate from or integrated with the assembly 612. In some embodiments, the servo controller and assembly 612 are provided as part of a teleoperated arm cart positioned near the patient's body.

[0063] The control system 620 can be coupled to the endoscopic imaging system 615 and can include a processor to process the captured images for subsequent display, such as on the surgeon's console or on another suitable display located locally and / or remotely to the surgeon. For example, where a stereoscopic endoscope is used, the control system 620 can process the captured images to present the surgeon with a coordinated stereoscopic image of the surgical site as a field of view image. Such coordination can include alignment between the relative images and can include adjusting the stereo working distance of the stereoscopic endoscope.

[0064] In alternative embodiments, the medical system 610 may include more than one component 612 and / or more than one operator input system 616. The exact number of components 612 will depend on the surgical procedure and space limitations within the operating room, among other factors. The operator input systems 616 may be collocated, or they may be located in separate locations. Multiple operator input systems 616 allow more than one operator to control one or more components 612 in various combinations. The medical system 610 may also be used to train and rehearse medical procedures.

[0065] Figure 7 612, which may be referred to as a manipulation system, a patient side cart, a surgical cart, a teleoperated arm cart, or a surgical robot. The illustrated assembly 612 provides manipulation of three surgical tools 630a, 630b, and 630c (e.g., a medical instrument system 614) and an imaging device 628 (e.g., an endoscopic imaging system 615), such as a stereo endoscope for capturing images of a surgical site. The imaging device 628 may transmit signals to the control system 620 via a cable 656. Manipulation is provided by a teleoperated mechanism having multiple joints. The imaging device 628 and the surgical tools 630a-c may be positioned and manipulated through an incision in the patient's body so that a kinematic remote center is maintained at the incision to minimize the size of the incision. When the surgical tools 630a-c are positioned within the field of view of the imaging device 628, the image of the surgical site may include images of the distal ends of these surgical tools. The imaging device 628 and the surgical tools 630a-c may each be a therapeutic, diagnostic, or imaging instrument.

[0066] Figure 86 is a perspective view of an embodiment of an operator input system 616 at a surgeon's console. The operator input system 16 includes a left eye display 632 and a right eye display 634 for presenting a coordinated stereoscopic view of the surgical environment to the surgeon S that enables depth perception. The left eye display 632 and the right eye display 634 can be components of a display system 635. In other embodiments, the display system 35 can include one or more other types of displays. In some embodiments, the images displayed on the display system 635 can be displayed separately or simultaneously on at least one display screen of the auxiliary system 626.

[0067] The operator input system 616 further includes one or more input control devices 636, which in turn cause the assembly 612 to manipulate one or more instruments of the endoscopic imaging system 615 and / or the medical instrument system 614. The input control device 636 can provide the same Cartesian degrees of freedom as the instrument with which it is associated to provide a telepresence to the surgeon S, or provide a sense that the input control device 638 is integrated with the instrument, so that the surgeon has a strong sense of direct control of the instrument. Therefore, the degrees of freedom of each input control device 636 are mapped to the degrees of freedom of the associated instrument of each input control device 636 (e.g., one or more of the instruments of the endoscopic imaging system 615 and / or the medical instrument system 614). To this end, position, force, and tactile feedback sensors (not shown) can be used to transmit position, force, and tactile sensations from the medical instrument (e.g., surgical tools 630a-c or imaging device 628) back to the surgeon's hand through the input control device 636. In addition, the arrangement of the medical instruments can be mapped to the arrangement of the surgeon's hands and the view of the surgeon's eyes, so that the surgeon has a strong sense of direct control of the instrument. Input control 637 is a foot pedal that receives input from the user's foot.Aspects of operator input system 616, assembly 612, and auxiliary system 626 may be adjustable and customizable to meet the surgeon's S physical needs, skill level, or preferences.

[0068] In the description, the specific details of describing some embodiments have been set forth. In order to provide a thorough understanding of the embodiments, many specific details have been set forth. However, it will be apparent to those skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to be illustrative rather than limiting. Those skilled in the art may realize other elements, although not specifically described here, within the scope and spirit of this disclosure.

[0069] As long as feasible, the elements described in detail with reference to one embodiment, implementation or application may be optionally included in other embodiments, implementations or applications that are not specifically shown or described. For example, if an element is described in detail with reference to one embodiment, and is not described with reference to a second embodiment, the element may still be required to be included in the second embodiment. Therefore, in order to avoid unnecessary repetition in the following description, one or more elements shown and described in association with one embodiment, implementation or application may be incorporated into other embodiments, implementations or aspects, unless otherwise specifically described, unless the one or more elements will make the embodiment or implementation inoperative, or unless two or more of the elements provide conflicting functions. Not all of the processes shown may be performed in all embodiments of the disclosed method. In addition, one or more processes not explicitly shown in the shown process may be included before, after, in the middle or as part of the shown process. In some embodiments, one or more of the processes may be performed by a control system, or may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which may enable one or more processors to perform one or more of the processes when run by one or more processors.

[0070] Any changes and further modifications to the described devices, apparatuses, methods, and any further applications of the principles of the present disclosure are generally contemplated by those skilled in the art to which the present disclosure relates. In addition, the dimensions provided herein are for specific examples, and it is contemplated that the concepts of the present disclosure may be implemented using different sizes, dimensions, and / or ratios. In order to avoid unnecessary descriptive repetitions, one or more components or actions described according to one illustrative embodiment may be used or omitted from other illustrative embodiments as appropriate. For the sake of brevity, multiple iterations of these combinations will not be described separately. For simplicity, in some cases, the same reference numerals are used throughout the drawings to refer to the same or similar parts.

[0071] The systems and methods described herein can be applied to imaging any of various anatomical systems, including lungs, colon, intestines, stomach, liver, kidneys and calyces, brain, heart, circulatory system including vascular system, etc. Although some embodiments of medical procedures are provided herein, any reference to medical or surgical instruments and medical or surgical methods is non-restrictive. For example, the instruments, systems and methods described herein can be used for non-medical purposes, including industrial uses, general robotic uses, and sensing or manipulating non-tissue artifacts. Other example applications relate to cosmetic improvements, imaging of human or animal anatomical structures, collecting data from human or animal anatomical structures, and training medical or non-medical personnel. Additional example applications include procedures for tissues removed from human or animal anatomical structures (not returned to human or animal anatomical structures) and procedures for human or animal corpses. In addition, these technologies can also be used for surgical and non-surgical medical treatment or diagnostic procedures.

[0072] One or more elements in the embodiments of the present disclosure may be implemented in software to be executed on a processor of a computer system such as a control processing system. When implemented in software, the elements of the examples of the present disclosure may be code segments that perform various tasks. The program or code segment may be stored in a processor-readable storage medium or device, which may be downloaded via a computer data signal embodied in a carrier wave on a transmission medium or a communication link. The processor-readable storage device may include any medium that may store information, including optical media, semiconductor media, and / or magnetic media. Examples of processor-readable storage devices include electronic circuits; semiconductor devices, semiconductor memory devices, read-only memories (ROMs), flash memories, erasable programmable read-only memories (EPROMs); floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. The code segments may be downloaded via computer networks such as the Internet, intranets, and the like. Any of a variety of centralized or distributed data processing architectures may be employed. Programming instructions may be implemented as multiple separate programs or subroutines, or they may be integrated into multiple other aspects of the systems described herein. In some examples, the control system may support wireless communication protocols such as Bluetooth, Infrared Data Association (IrDA), HomeRF, IEEE 802.11, Digital Enhanced Cordless Telecommunications (DECT), Ultra-Wideband (UWB), ZigBee, and wireless telemetry.

[0073] Note that the processes and displays presented may not be inherently related to any particular computer or other device. Various general purpose systems may be used with programs according to the teachings herein, or it may prove convenient to construct more specialized equipment to perform the described operations. The desired structures of various such systems will appear as elements in the claims. In addition, examples of the present invention are described without reference to any particular programming language. It should be understood that various programming languages ​​may be used to implement the teachings of the present invention described herein.

[0074] The present disclosure describes the states of various instruments, instrument parts, and anatomical structures in three-dimensional space. As used herein, the term "position" refers to the position of an object or a part of an object in three-dimensional space (e.g., three translational degrees of freedom along Cartesian x, y, and z coordinates). As used herein, the term "orientation" refers to the rotational placement of an object or a part of an object (e.g., in one or more rotational degrees of freedom, such as roll, pitch, and / or yaw). As used herein, the term "attitude" refers to the position of an object or a part of an object in at least one translational degree of freedom, and the orientation of the object or part of the object in at least one rotational degree of freedom (e.g., up to six total degrees of freedom). As used herein, the term "shape" refers to a set of attitudes, positions, or orientations measured along an object.

[0075] While certain illustrative embodiments of the invention have been described and shown in the drawings, it is to be understood that such embodiments are merely illustrative and not restrictive of the broad invention, and that the embodiments of the invention are not limited to the specific construction and arrangements shown and described, since various other modifications may occur to those of ordinary skill in the art.

Claims

1. A method comprising: receiving a plurality of medical procedure records, each of the plurality of medical procedure records being generated during a medical procedure performed with a robotic-assisted medical system; identifying a set of characteristic actions common to the plurality of medical procedure records; and At least one parameter is determined for each characteristic action in the set of characteristic actions. 2 . The method of claim 1 , wherein each of the plurality of medical procedure records includes medical procedure information for a medical procedure performed on a unique patient. 3 . The method of claim 1 , wherein each of the plurality of medical procedure records includes medical procedure information for a medical procedure performed by a common physician. 4 . The method of claim 1 , wherein each of the plurality of medical procedure records includes medical procedure information for a medical procedure performed by a supervising physician.

5. The method of claim 1, wherein the set of characteristic actions includes actions of a medical instrument coupled to the robotic-assisted medical system, the medical instrument being common in the plurality of medical procedure records. The method of claim 5 , wherein the motion of the medical device is a rotational movement of a wrist joint of the medical device.

7. The method of claim 6, wherein the at least one parameter comprises a measure of an amount of rotation of the wrist joint. The method of claim 6 , wherein the at least one parameter comprises a duration to complete the rotational movement.

9. The method of claim 6, wherein the at least one parameter comprises a measure of a consequence of the intervention.

10. The method of claim 9, wherein the consequence of the intervention is bleeding.

11. The method of claim 5, wherein the motion of the medical device is a clamping motion of an end effector of the medical device.

12. The method of claim 11, wherein the at least one parameter comprises a measure of a force applied by the clamping action of the end effector.

13. The method of claim 11, wherein the at least one parameter includes a duration to complete the clamping action. The method of claim 11 , wherein the at least one parameter comprises a measure of a consequence of the intervention.

15. The method of claim 14, wherein the consequence of the intervention is a tissue color change.

16. The method of claim 5, wherein the action of the medical device is delivery of energy by the medical device.

17. The method of claim 16, wherein the at least one parameter comprises a measure of energy applied by the medical device.

18. The method of claim 16, wherein the at least one parameter comprises a duration for which the medical device delivers energy.

19. The method of claim 16, wherein the at least one parameter comprises a measure of a consequence of the intervention.

20. The method of claim 19, wherein the consequence of intervention is smoke.

21. The method of claim 5, wherein the medical device is an imaging device, and the motion of the medical device is a posture change of the imaging device.

22. The method of claim 1, wherein the set of characteristic motions comprises motions of a manipulator arm that are common among the plurality of medical procedure records.

23. The method of claim 22, wherein the at least one parameter comprises a measure of the amount of movement of the manipulator arm.

24. The method of claim 22, wherein the at least one parameter comprises a pose of the manipulator arm during an arm collision event.

25. The method of claim 1, wherein the set of characteristic actions includes actions at an operator console that is common among the plurality of medical procedure records.

26. The method of claim 25, wherein the at least one parameter includes a duration that the head is detached from the operator console.

27. The method of claim 25, wherein the operator console includes an operator input device and the at least one parameter includes a measure of movement of the operator input device.

28. The method of claim 1, wherein the at least one parameter is recorded by a position sensor system.

29. The method of claim 1, wherein the at least one parameter is recorded by a force sensor system.

30. The method of claim 1, wherein the at least one parameter is recorded by an imaging system.

31. The method of claim 1, wherein the at least one parameter is recorded by a patient monitoring system.

32. A robot-assisted medical system, comprising: Manipulator arm; Operator console; a medical device coupled to the manipulator arm; as well as a control system in communication with the operator console and the manipulator arm, wherein the control system comprises: processor; as well as a memory including machine-readable instructions that, when executed by the processor, cause the control system to: Receive multiple medical procedure records; identifying a set of characteristic actions common to the plurality of medical procedure records; and At least one parameter is determined for each characteristic action in the set of characteristic actions.

33. A method comprising: receiving medical procedure records; identifying one or more actions in the medical procedure record; as well as At least one parameter is determined for each of the one or more actions.

Images