Autonomous Instrument-Assisted Surgical System Operation

Abstract

Systems, methods, and means for autonomous operation of a surgical device within a pre-defined boundary are described herein. Discrete signals associated with clamp control (e.g., closure of a clamp jaw) may be received by the surgical device. The discrete signals may be triggered by a medical professional or may be activated autonomously. In response to the discrete signals, the surgical device may generate a continuous signal based on an algorithm to cause a continuous application of force or the deployment of an action. The surgical device may determine a safety adjustment associated with the operation of the surgical device based at least on a measurement associated with one of tissue, inrush current, or the distance between a smart energy device and a smart grasper.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application is related to the following applications filed simultaneously, the contents of each of which are incorporated herein by reference. ·U.S. Patent Application entitled "METHOD OF CONTROLLING AUTONOMOUS OPERATIONS IN A SURGICAL SYSTEM" filed together with this specification and having Attorney Docket No. END9430USNP1. ·U.S. Patent Application entitled "AUTONOMOUS INTRA - INSTRUMENT SURGICAL SYSTEM ACTUATION" filed together with this specification and having Attorney Docket No. END9430USNP6. ·U.S. Patent Application entitled "AUTONOMOUS SURGICAL SYSTEM INSTRUMENT ACTUATION" filed together with this specification and having Attorney Docket No. END9430USNP7.

Background Art

[0002] Surgical procedures performed using a surgical system or surgical device may rely on medical professionals to control all aspects of the surgical system or surgical device during the surgical procedure. Current surgical systems and / or surgical devices may not be suitable for performing surgical procedures autonomously.

Summary of the Invention

Means for Solving the Problems

[0003] Systems, methods, and means for autonomous operation of surgical devices are described herein. For example, the surgical device may be a surgical cutting device or a surgical energy device. A first discrete signal associated with clamp control may be received by the surgical device. The first discrete signal may be associated with the initiation of closure of the clamp jaw. The first discrete signal may be triggered by a medical professional or may be activated autonomously. The surgical device may generate a first continuous signal to cause a continuous application of force based on a first autonomous control algorithm in response to the first discrete signal. For example, the continuous application of force may be adjusted autonomously based on at least a first measurement (e.g., a measurement associated with tissue).

[0004] A second discrete signal associated with clamp control may be received by the surgical device. The second discrete signal may be associated with the initiation of a firing sequence. The second discrete signal may be triggered by a medical professional or may be activated autonomously. The deployment operation may be the advancement and retraction of a cutting member. The surgical device may generate a second continuous signal to cause a deployment operation based on a second autonomous control algorithm in response to the second discrete signal. The second measurement may be the ratio of collagen to elastin within the tissue. The deployment operation may be adjusted autonomously based on at least the second measurement.

[0005] Systems, methods, and means for autonomous operation of a surgical device within a predefined boundary are described herein. For example, the surgical device may be a smart grasper, a smart surgical stapler, or a smart energy device. The predefined boundary may be a virtual movement boundary associated with a surgical task. The predefined boundary may be a field of view defined by a scope device.

[0006] A surgical device, such as a smart grasper, can determine a safety adjustment for the operation of the smart grasper based at least on the condition that the tissue tension measurement associated with the smart grasper is greater than or equal to the maximum tissue tension. The safety adjustment may be a reduction in the grasping force. A surgical device, such as a smart stapler, can determine a safety adjustment for the operation of the smart stapler based at least on the condition that the measured inrush current is less than a minimum threshold. The safety adjustment may be to stop the firing sequence. A surgical device, such as a smart energy device, may determine a safety adjustment for the movement of the smart energy device based on one or more position data and orientation data based at least on the condition that the distance between the smart energy device and the smart grasper is less than a threshold.

[0007] A smart surgical device is described. The smart surgical device includes a processor. The processor is configured to generate a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm in response to receiving a first discrete signal associated with clamp control. The continuous application of force is autonomously adjusted based at least on a first measurement value. The processor is configured to generate a second continuous signal for causing a deployment operation based on a second autonomous control algorithm in response to receiving a second discrete signal associated with the deployment operation. The deployment operation is autonomously adjusted based at least on a second measurement value. Requiring two discrete signals to cause the autonomous clamp operation and the autonomous deployment operation provides the advantage of allowing these steps to be separated for control purposes. Thus, a medical professional can indicate whether the clamp operation or the deployment operation is to be performed manually or autonomously, providing greater flexibility with respect to autonomous control. For example, a medical professional can manually perform the clamp and then indicate that the deployment operation should be performed autonomously.

[0008] The processor may be configured to generate a second continuous signal in response to the reception of a second discrete signal, regardless of whether the first discrete signal has been received. Advantageously, the clamping operation and the deployment operation may be independent such that the deployment operation can be performed autonomously regardless of whether the clamping operation has been performed manually or autonomously.

[0009] The first discrete signal may be triggered by a medical professional. Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by the medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or wish to have the device perform it autonomously.

[0010] The first discrete signal may be triggered by a medical professional actuating and then releasing an actuation control trigger.

[0011] The second discrete signal may be triggered by a medical professional or autonomously. Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by the medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or wish to have the device perform it autonomously.

[0012] The second discrete signal may be triggered by a medical professional activating and then releasing an actuation control trigger, or may be triggered by the completion of the first autonomous control algorithm. Advantageously, the autonomous control of the deployment operation can be indicated in several ways. For example, a medical professional can activate and then release an actuation trigger associated with the deployment operation to provide an autonomous deployment operation after a manual clamping operation or an autonomous clamping operation. Alternatively, if the clamping operation is performed autonomously, the trigger to initiate the autonomous control of the deployment operation may be the completion of the clamping operation. For example, a medical professional may momentarily activate a clamp control trigger and then release it. In this example, the clamp control operation and the firing control operation can be autonomous. When the clamp control operation is completed, a second signal is autonomously triggered and the firing control operation is initiated. Further, a medical professional may switch from manual control to autonomous control by releasing the holding of an actuation control trigger (e.g., a firing control trigger). In such a case, a second discrete signal is triggered and the deployment operation may transition from manual mode to autonomous mode.

[0013] A smart surgical device is described. The smart surgical device includes a first actuation trigger associated with clamp control, a second actuation trigger associated with a deployment operation, and a processor. The processor is configured to generate a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm in response to receiving a first discrete signal associated with the first actuation trigger. The continuous application of force is autonomously adjusted based at least on a first measurement value. The processor is further configured to generate a second continuous signal for causing a deployment operation based on a second autonomous control algorithm in response to receiving a second discrete signal associated with the second actuation trigger. The deployment operation is autonomously adjusted based at least on a second measurement value. In response to a first user-initiated continuous signal associated with the first actuation trigger, the device enables the clamp control to be manually executed. In response to a second user-initiated continuous signal associated with the second actuation trigger, the device enables the deployment operation to be manually executed. Requiring two discrete signals to cause the autonomous clamp operation and the autonomous deployment operation provides the advantage of allowing these steps to be separated for control. Thus, a medical professional can indicate whether the clamp operation or the deployment operation is to be manually or autonomously executed by the actuation of those first and second control triggers, providing greater flexibility with respect to autonomous control. For example, a medical professional can manually execute the clamp and then indicate that the deployment operation should be autonomously performed.

[0014] The processor can be configured to generate the second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal has been received. Advantageously, the clamp operation and the deployment operation can be independent such that the deployment operation can be autonomously executed regardless of whether the clamp operation has been manually or autonomously performed.

[0015] The first discrete signal can be triggered by a medical professional activating and then releasing a first activation trigger. Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by the medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or have the device perform it autonomously.

[0016] The second discrete signal may be triggered by a medical professional activating and then releasing a second activation trigger, or may be triggered by the completion of a first autonomous control algorithm. Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by the medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or have the device perform it autonomously. Advantageously, the autonomous control of the deployment operation can be indicated in several ways. For example, the medical professional can activate and then release an activation trigger associated with the deployment operation to provide an autonomous deployment operation after a manual clamping operation or an autonomous clamping operation. Alternatively, if the clamping operation is performed autonomously, the trigger to initiate the autonomous control of the deployment operation may be the completion of the clamping operation. For example, the medical professional may momentarily activate a clamping control trigger and then release it. In this example, the clamping control operation and the firing control operation can be autonomous. When the clamping control operation is completed, a second signal is triggered autonomously and the firing control operation is initiated. Further, the medical professional may switch from manual control to autonomous control by releasing the hold of an activation control trigger (e.g., a firing control trigger). In such a case, the second discrete signal is triggered and the deployment operation can shift from manual mode to autonomous mode.

[0017] The smart surgical device can be a smart surgical cutting device or a smart surgical energy device. The first discrete signal can be associated with initiating the closure of the clamp jaw.

[0018] The smart surgical device may be a smart surgical cutting device. The continuous application of force may be applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release.

[0019] The smart surgical device may be a smart surgical cutting device. The first measurement value may be one of the loads on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement value indicating the presence of a rigid object.

[0020] The smart surgical device may be a smart surgical cutting device. The deployment operation may be the advancement and retraction of the cutting member.

[0021] The smart surgical device may be a smart surgical energy device. The second discrete signal may be associated with the start of the emission sequence.

[0022] The smart surgical device may be a smart surgical energy device. The deployment operation may be the generation of energy.

[0023] The smart surgical device may be a smart surgical energy device. The continuous application of force to the tissue may be applied during one or more of the following clamp controls: initial contact, clamp down, standby, or pressure maintenance.

[0024] The smart surgical device may be a smart surgical energy device. The first measurement value may be the position of the tissue between the clamp arm and the energy blade. The second measurement value may be the ratio of collagen to elastin in the tissue.

[0025] A computer-implemented method is described. The method includes receiving a first discrete signal associated with clamp control. The method further includes generating, in response to the first discrete signal, a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm, wherein the continuous application of force is autonomously adjusted based at least on a first measurement value. The method further includes receiving a second discrete signal associated with a deployment operation. The method further includes generating, in response to the second discrete signal, a second continuous signal for causing a deployment operation based on a second autonomous control algorithm, wherein the deployment operation is autonomously adjusted based at least on a second measurement value. A method that requires two discrete signals to cause an autonomous clamping operation and an autonomous deployment operation provides the advantage of allowing these steps to be separated for control. Thus, a medical professional can indicate whether the clamping operation or the deployment operation is to be performed manually or autonomously, providing greater flexibility with respect to autonomous control.

[0026] The method may be performed on a smart surgical device for performing a surgical task or on a hub in a state where the hub provides commands to the smart surgical device.

[0027] The method may include generating a second continuous signal in response to receiving the second discrete signal, regardless of whether the first discrete signal has been received. Advantageously, the clamping operation and the deployment operation may be independent such that the deployment operation may be autonomously performed regardless of whether the clamping operation has been performed manually or autonomously.

[0028] The first discrete signal may be triggered by a medical professional activating and then releasing an activation control trigger. Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by a medical professional, enabling the medical professional to indicate whether the medical professional desires to perform the task manually or desires the device to perform it autonomously.

[0029] The second discrete signal can be triggered by a medical professional or autonomously. Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by a medical professional, enabling the medical professional to indicate whether they desire to perform the task manually or desire to have the device perform it autonomously.

[0030] The second discrete signal can be triggered by a medical professional actuating and then releasing an actuation control trigger. Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by a medical professional, enabling the medical professional to indicate whether they desire to perform the task manually or desire to have the device perform it autonomously. Advantageously, the autonomous control of the deployment operation can be indicated in several ways. For example, a medical professional can actuate and then release an actuation trigger associated with the deployment operation to provide an autonomous deployment operation after a manual clamp operation or an autonomous clamp operation. Alternatively, if the clamp operation is performed autonomously, the trigger to initiate the autonomous control of the deployment operation may be the completion of the clamp operation. For example, a medical professional may momentarily actuate a clamp control trigger and then release it. In this example, the clamp control operation and the firing control operation can be autonomous. When the clamp control operation is completed, the second signal is triggered autonomously and the firing control operation is initiated. Further, a medical professional may switch from manual control to autonomous control by releasing the hold of an actuation control trigger (e.g., a firing control trigger). In such a case, the second discrete signal is triggered and the deployment operation can transition from manual mode to autonomous mode.

[0031] The smart surgical device can be a smart surgical cutting device or a smart surgical energy device. The first discrete signal can be associated with the start of the closure of the clamp jaw.

[0032] The smart surgical device may be a smart surgical cutting device. The continuous application of force may be applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release.

[0033] The smart surgical device may be a smart surgical cutting device. The first measurement value may be one of the load on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement value indicating the presence of a rigid object.

[0034] The smart surgical device may be a smart surgical cutting device. The deployment operation may be the advancement and retraction of the cutting member.

[0035] The smart surgical device may be a smart surgical energy device. The second discrete signal may be associated with the start of the emission sequence.

[0036] The smart surgical device may be a smart surgical energy device. The deployment operation may be the generation of energy.

[0037] The smart surgical device may be a smart surgical energy device. The continuous application of force to the tissue may be applied during one or more of the following clamp controls: initial contact, clamp down, standby, or pressure maintenance.

[0038] The smart surgical device may be a smart surgical energy device. The first measurement value may be the position of the tissue between the clamp arm and the energy blade. The second measurement value may be the ratio of collagen to elastin in the tissue.

[0039] A computer program including instructions, which, when the program is executed by a controller, cause the controller to execute any of the foregoing methods, is described.

[0040] A computer-readable medium including instructions that, when executed by a controller, cause the controller to perform any of the foregoing methods is described.

[0041] A computing device is described. The computing device includes a processor. The processor is configured to control a surgical device to operate autonomously within a predefined boundary. The processor is further configured to determine a safety adjustment for an operation based on a condition being met. The processor is further configured to control the surgical device to operate based on the safety adjustment. The computing device provides the advantage of controlling the surgical device to enhance its safety during operation, thereby reducing the risk of any harmful outcomes or complications associated with the operation of the surgical device during a surgical task.

[0042] The predefined boundary may be a virtual movement boundary associated with a surgical task. By controlling the surgical device within the virtual movement boundary, it is possible to increase the safety of the surgical task, for example, by restricting the movement and / or articulation of the surgical device to prevent any unintended contact with tissues or other areas of the surgical device.

[0043] The condition may be met when a measurement associated with the surgical device or the surgical task is above or below a preset maximum / minimum threshold.

[0044] The surgical device may be a smart grasper. The condition may be that a tissue tension measurement associated with the smart grasper is above a maximum tissue tension. The safety adjustment may be a reduction in the grasping force. Controlling the surgical device to operate based on the safety adjustment may include sending a control signal to the surgical device to cause a reduction in the grasping force.

[0045] The surgical device may be a smart surgical stapler. The condition may be that the measured inrush current is less than a minimum threshold. The safety adjustment may be to stop the firing sequence. Controlling the surgical device to operate based on the safety adjustment may include stopping the transmission of a control signal to the surgical device to stop the firing sequence.

[0046] The surgical device may be a smart energy device. The processor may be further configured to receive first placement data associated with a first trocar and second placement data associated with a second trocar. The first trocar may be associated with a smart grasper, and the second trocar may be associated with a smart energy device. The processor may be further configured to determine first position data associated with the smart grasper based on the first placement data and to determine second position data associated with the smart energy device based on the second placement data.

[0047] The processor may be further configured to receive third position data associated with the patient's body and first orientation data associated with the patient's body. The condition may be that the distance between the smart energy device and the smart grasper is less than a threshold. The safety adjustment may be an adjustment of the movement of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data.

[0048] The pre - defined boundary may be the field of view defined by a scope device. Advantageously, a surgical device outside the current scope field of view may have its autonomous operation prohibited, reducing the risk that an unseen portion of the surgical device contacts unintended tissue or another device.

[0049] The computing device may be a robotic system.

[0050] A computer-implemented method is described. The method includes controlling a surgical device to operate autonomously within a pre-defined boundary. The method includes determining a safety adjustment for an operation based on a condition being met. The method further includes controlling the surgical device to operate based on the safety adjustment. The method provides the advantage of controlling the surgical device to increase its safety during operation, thereby reducing the risk of any adverse outcomes or complications associated with the operation of the surgical device during a surgical task.

[0051] The pre-defined boundary may be a virtual movement boundary associated with a surgical task. The method may include restricting the movement of the surgical device according to the virtual movement boundary. By controlling the surgical device within the virtual movement boundary, it is possible to increase the safety of the surgical task by restricting the movement and / or articulation of the surgical device to prevent any unintended contact with tissue or other areas of the surgical device.

[0052] The condition may be met when a measurement associated with the surgical device or the surgical task is above / below a pre-set maximum / minimum threshold.

[0053] The surgical device may be a smart grasper. The condition may be that a tissue tension measurement associated with the smart grasper is above a maximum tissue tension. The safety adjustment may be a reduction in the grasping force. Controlling the surgical device to operate based on the safety adjustment may include sending a control signal to the surgical device to cause a reduction in the grasping force.

[0054] The surgical device may be a smart surgical stapler. The condition may be that an inrush current measurement is below a minimum threshold. The safety adjustment may be to stop the firing sequence. Controlling the surgical device to operate based on the safety adjustment may include stopping the transmission of a control signal to the surgical device to stop the firing sequence.

[0055] The surgical device may be a smart energy device. The predefined boundary may be a virtual movement boundary associated with a surgical task. The method may further include receiving first placement data associated with a first trocar and second placement data associated with a second trocar. The first trocar may be associated with a smart grasper, and the second trocar may be associated with a smart energy device. The method further includes determining first position data associated with the smart grasper based on the first placement data and determining second position data associated with the smart energy device based on the second placement data.

[0056] The method may further include receiving third position data associated with the patient's body and first orientation data associated with the patient's body. The condition may be that the distance between the smart energy device and the smart grasper is less than a threshold value. The safety adjustment may be an adjustment of the movement of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data.

[0057] The predefined boundary may be the field of view defined by a scope device. Advantageously, a surgical device outside the current scope field of view may be prohibited from autonomous operation, reducing the risk of the unseen part of the surgical device contacting unintended tissue or another device.

[0058] A computer program including instructions, which, when the program is executed by a controller, cause the controller to execute any of the aforementioned methods, is described.

[0059] A computer-readable medium including instructions, which, when executed by a controller, cause the controller to execute any of the aforementioned methods, is described.

Brief Description of the Drawings

[0060]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11A

Figure 11B

Figure 12

Figure 13A

Figure 13B

Figure 13C

Figure 14

Mode for Carrying Out the Invention

[0061] FIG. 1 is a block diagram of a computer-implemented surgical system 20000. An exemplary surgical system such as surgical system 20000 can include one or more surgical systems (e.g., surgical subsystems) 20002, 20003, and 20004. For example, each surgical system 20002 can include a computer-implemented bidirectional surgical system. For example, surgical system 20002 may include a surgical hub 20006 and / or a computing device 20016 that communicate with a cloud computing system 20008, as described, for example, in FIG. 2. The cloud computing system 20008 can include at least one remote cloud server 20009 and at least one remote cloud storage unit 20010. Exemplary surgical systems 20002, 20003, or 20004 may include a wearable sensing system 20011, an environmental sensing system 20015, a robotic system 20013, one or more intelligent instruments 20014, a human interface system 20012, etc. The human interface system is also referred to herein as a human interface device. The wearable sensing system 20011 may include one or more HCP sensing systems and / or one or more patient sensing systems. The environmental sensing system 20015 may include, for example, one or more devices used to measure one or more environmental attributes, as further described in FIG. 2. The robotic system 20013 may include, for example, a plurality of devices used to perform a surgical operation, as further described in FIG. 2.

[0062] The surgical system 20002 can communicate with a remote server 20009 that can be part of a cloud computing system 20008. In one example, the surgical system 20002 may communicate with the remote server 20009 via a cable / FIOS networking node of an Internet service provider. In one example, the patient sensing system may communicate directly with the remote server 20009. The surgical system 20002 and / or its components may use one or more of the cellular protocols of GSM / GPRS / EDGE (2G), UMTS / HSPA (3G), long term evolution (LTE) or 4G, LTE-Advanced (LTE-A), new radio (NR) or 5G to communicate with the remote server 20009 via a cellular transmission / reception point (TRP) or base station.

[0063] The surgical hub 20006 can have a collaborative interaction with one of more means for displaying images from the laparoscope scope as well as information from one or more other smart devices and one or more sensing systems 20011. The surgical hub 20006 can interact with one or more sensing systems 20011, one or more smart devices, and multiple displays. The surgical hub 20006 can be configured to collect measurement data from one or more sensing systems 20011 and send notification or control messages to one or more sensing systems 20011. The surgical hub 20006 can transmit and / or receive information, including notification information, to and from a human interface system 20012. The human interface system 20012 can include one or more human interface devices (HID). The surgical hub 20006 can transmit and / or receive acoustic devices, notification or control information to the display, and / or control information to various devices that communicate with the surgical hub.

[0064] For example, as discussed in FIG. 1, the sensing system 20001 may include a wearable sensing system 20011 (which may include one or more HCP sensing systems and one or more patient sensing systems) and an environmental sensing system 20015. One or more sensing systems 20001 may measure data regarding various biomarkers. One or more sensing systems 20001 may use one or more sensors, such as optical sensors (e.g., photodiodes, photoreceptors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, thermoelectric sensors, infrared sensors, etc., to measure biomarkers. One or more sensors may use one of more of the sensing techniques such as photoplethysmography, electrocardiogram examination, electroencephalogram examination, colorimetric analysis, impedancemetry, potential difference measurement, current measurement, etc., to measure biomarkers as described herein.

[0065] Biomarkers measured by one or more sensing systems 20001 may include, but are not limited to, sleep, core body temperature, maximal oxygen consumption, physical activity, alcohol intake, respiratory rate, oxygen saturation, blood pressure, blood glucose, heart rate variability, blood potential of hydrogen, hydration status, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, cough and sneeze, gastrointestinal motility, gastrointestinal imaging, airway bacteria, edema, mental aspect, sweat, circulating tumor cells, autonomic nervous tension, circadian rhythm, and / or menstrual cycle.

[0066] Biomarkers may be related to physiological systems, including but not limited to the behavioral and psychological, cardiovascular, renal, skin, nervous, gastrointestinal, respiratory, endocrine, immune, tumor, musculoskeletal, and / or reproductive systems. Information from biomarkers may be determined and / or used, for example, by a computer-implemented patient and surgical system 20000. Information from biomarkers may be determined and / or used by a computer-implemented patient and surgical system 20000 to, for example, improve the above systems and / or improve patient outcomes. One or more sensing systems 20001, biomarkers 20005, and physiological systems are described in detail in U.S. Patent Application No. 17 / 156,287, filed January 22, 2021, entitled "METHOD OF ADJUSTING A SURGICAL PARAMETER BASED ON BIOMARKER MEASUREMENTS" (Attorney Docket No. END9290USNP1), the disclosure of which is incorporated herein by reference in its entirety.

[0067] Figure 2 shows an example of a surgical system 20002 in an operating room. As illustrated in Figure 2, a patient is operated on by one or more healthcare professionals (HCPs). The HCPs are monitored by one or more HCP sensing systems 20020 worn by the HCPs. The HCPs, and the environment surrounding the HCPs, may also be monitored by one or more environmental sensing systems, including, for example, a set of cameras 20021, a set of microphones 20022, and other sensors deployed in the operating room. The HCP sensing systems 20020 and the environmental sensing systems communicate with a surgical hub 20006 and may further communicate with one or more cloud servers 20009 of a cloud computing system 20008, as shown in Figure 1. The environmental sensing system may be used to measure one or more environmental attributes, such as the location of HCPs in the operating room, HCP movement, ambient noise in the operating room, temperature / humidity in the operating room, and the like.

[0068] As illustrated in FIG. 2, the main display 20023 and one or more audio output devices (e.g., speaker 20019) are placed within the sterile field so as to be visible to the operator on the operating table 20024. Additionally, the visualization / notification tower 20026 is placed outside the sterile field. The visualization / notification tower 20026 may include a first non-sterile human interface device (HID) 20027 and a second non-sterile HID 20029 that face opposite each other. The HID may be a display, or may be a display having a touch screen that enables a human to directly interface with the HID. The human interface system guided by the surgical hub 20006 may be configured to utilize the HIDs 20027, 20029, and 20023 to regulate the flow of information to the operators inside and outside the sterile field. In one example, the surgical hub 20006 may cause the HID (e.g., the main HID 20023) to display notifications and / or information regarding the patient and / or surgical steps. In one example, the surgical hub 20006 may prompt and / or receive input from a person within the sterile field or non-sterile area. In one example, the surgical hub 20006 may cause the HID to display a snapshot of the surgical site recorded by the imaging device 20030 on the non-sterile HID 20027 or 20029 while maintaining a live video of the surgical site on the main HID 20023. The snapshot on the non-sterile display 20027 or 20029 may, for example, permit a non-sterile operator to perform diagnostic steps related to the surgery.

[0069] In one aspect, the surgical hub 20006 may be configured to send diagnostic input or feedback entered by a non-sterile operator at the visualization tower 20026 to the main display 20023 within the sterile field so that it can be viewed by the sterile operator on the operating table. In one example, the input may be in the form of a modification to a snapshot displayed on the non-sterile display 20027 or 20029 that can be sent by the surgical hub 20006 to the main display 20023.

[0070] Referring to FIG. 2, the surgical instrument 20031 is used as part of a surgical system 20002 in a surgical operation. The hub 20006 can also be configured to regulate the information flow to the display of the surgical instrument 20031. For example, in U.S. Patent Application Publication No. 2019 / 0200844 (A1) (U.S. Patent Application No. 16 / 209,385) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY", filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety. Diagnostic inputs or feedback entered by a non-sterile operator at the visualization tower 20026 are sent by the hub 20006 to the surgical instrument display within the sterile field, where they can be viewed by the operator of the surgical instrument 20031. Exemplary surgical instruments suitable for use with the surgical system 20002 are described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019 / 0200844 (A1) (U.S. Patent Application No. 16 / 209,385) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY", filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0071] FIG. 2 illustrates an example of a surgical system 20002 used to perform surgery on a patient lying on an operating table 20024 within an operating room 20035. A robotic system 20034 can be used as part of the surgical system 20002 in a surgical procedure. The robotic system 20034 can include a surgeon's console 20036, a patient-side cart 20032 (surgical robot), and a surgical robot hub 20033. While the surgeon views the surgical site through the surgeon's console 20036, the patient-side cart 20032 can manipulate at least one removably coupled surgical tool 20037 through a minimally invasive incision in the patient's body. An image of the surgical site can be obtained by a medical imaging device 20030 that can be manipulated by the patient-side cart 20032 to change the orientation of the imaging device 20030. The robotic hub 20033 can be used to process the image of the surgical site and then display it to the surgeon through the surgeon's console 20036.

[0072] Other types of robotic systems can be readily adapted to be used with the surgical system 20002. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019 / 0201137 (A1) (U.S. Patent Application No. 16 / 209,407), entitled "METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL," filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0073] Various examples of cloud-based analysis methods implemented by a cloud computing system 20008 and suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0206569 (A1) (U.S. Patent Application No. 16 / 209,403), entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB," filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0074]

[0074] In various embodiments, the imaging device 20030 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, Charge-Coupled Device (CCD) sensors and Complementary Metal-Oxide Semiconductor (CMOS) sensors.

[0075] The optical components of the imaging device 20030 may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate a portion of the surgical field. The one or more image sensors may be capable of receiving light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0076] The one or more illumination sources may be configured to emit electromagnetic energy within the visible spectrum as well as the invisible spectrum. The visible spectrum, which is sometimes also referred to as the optical spectrum or emission spectrum, is a portion of the electromagnetic spectrum that is visible to the human eye (i.e., detectable by the human eye) and may be referred to as visible light or simply light. Typically, the human eye responds to wavelengths of approximately 380 nm to approximately 750 nm in air.

[0077]

[0075] The invisible spectrum (e.g., non-emission spectrum) is a portion of the electromagnetic spectrum that is located below and above the visible spectrum (i.e., wavelengths less than approximately 380 nm and greater than approximately 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than approximately 750 nm are longer than the red visible spectrum and become invisible infrared (IR), microwaves, and radio electromagnetic radiation. Wavelengths less than approximately 380 nm are shorter than the violet spectrum and become invisible ultraviolet, x-rays, and gamma ray electromagnetic radiation.

[0078] In various aspects, the imaging device 20030 is configured for use in minimally invasive procedures. Examples of imaging devices suitable for use with the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, choledochoscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophagogastroduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngo-neproscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0079] The imaging device may employ multispectral monitoring to distinguish topography from the underlying structure. A multispectral image captures image data within a specific wavelength range from across the electromagnetic spectrum. The wavelengths can be separated by a filter or by using an instrument having sensitivity to specific wavelengths including frequencies beyond the visible light range, e.g., IR, and light from ultraviolet. Spectral imaging enables extraction of additional information that cannot be captured by the red, green, and blue receptors of the human eye. The use of multispectral imaging is detailed under the heading “Advanced Imaging Acquisition Module” in U.S. Patent Application Publication No. 2019 / 0200844(A1) (U.S. Patent Application No. 16 / 209,385) filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. Multispectral monitoring can be a useful tool for repositioning the surgical field after a surgical task for performing one or more of the above-described tests on the treated tissue has been completed. It is understood that strict sterilization of the operating room and surgical instruments is required during any surgical procedure. The strict hygiene and sterilization conditions required in the “operating room,” i.e., the operating or treatment room, require the highest possible sterility of all medical devices and instruments. Part of the sterilization process is the need to sterilize anything that comes into contact with the patient or enters the sterile field, including the imaging device 20030 and its accessories and components. It will be understood that the sterile field can be considered a specific area considered to be free of microorganisms, such as within a tray or on a sterile towel, or the sterile field can be considered the area immediately surrounding a patient prepared for surgery. The sterile field can include properly attired and scrubbed team members, as well as all equipment and fixtures within that area.

[0080] The wearable sensing system 20011 shown in FIG. 1 may include one or more sensing systems, such as the HCP sensing system 20020 as shown in FIG. 2. The HCP sensing system 20020 may include a sensing system for monitoring and detecting a set of physical and / or physiological states of a healthcare provider (HCP). The HCP may generally be one or more healthcare providers assisting a surgeon or other healthcare service provider. In one example, the sensing system 20020 may measure a set of biomarkers to monitor the heart rate of the HCP. In one example, the sensing system 20020 (e.g., a watch or a wristband) worn on the wrist of a surgeon may use an accelerometer to detect hand movement and / or shake and determine the magnitude and frequency of tremors. The sensing system 20020 may transmit measurement data associated with the set of biomarkers and data associated with the physical state of the surgeon to the surgical hub 20006 for further processing. One or more environmental sensing devices may transmit environmental information to the surgical hub 20006. For example, the environmental sensing device may include a camera 20021 for detecting the position of the HCP's hand / body. The environmental sensing device may include a microphone 20022 for measuring ambient noise in the operating room. Other environmental sensing devices may include devices such as a thermometer for measuring temperature and a hygrometer for measuring the ambient humidity in the operating room. The surgical hub 20006 may, alone or in communication with a cloud computing system, use the surgeon biomarker measurement data and / or environmental sensing information to, for example, modify the control algorithm of a handheld instrument or the average latency of a robot interface to minimize tremors. In one example, the HCP sensing system 20020 may measure one or more surgeon biomarkers associated with the HCP and transmit measurement data associated with the surgeon biomarkers to the surgical hub 20006.The HCP awareness system 20020 may use one or more of the RF protocols of Bluetooth (registered trademark), Bluetooth Low-Energy (BLE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low-Power Wireless Personal Area Network (6LoWPAN), and Wi-Fi to communicate with the surgical hub 20006. The surgeon biomarkers may include one or more of stress, heart rate, etc. The environmental measurement values from the operating room may include the ambient noise level related to the movement of the surgeon or patient, surgeon and / or staff, the attention level of the surgeon and / or staff, etc.

[0081] The surgical hub 20006 may adaptively control one or more surgical instruments 20031 using the surgeon biomarker measurement data associated with the HCP. For example, the surgical hub 20006 may send a control program to the surgical instrument 20031 to control its actuator to limit or compensate for fatigue and the use of fine motor skills. The surgical hub 20006 may send the control program based on situation recognition and / or circumstances regarding the importance or criticality of the task. The control program may instruct the instrument to change its operation to provide more control when control is needed.

[0082] Figure 3 shows an exemplary surgical system 20002 having a surgical hub 20006. The surgical hub 20006 can be paired with a wearable sensing system 20011, an environmental sensing system 20015, a human interface system 20012, a robotic system 20013, and an intelligent instrument 20014 via a modular control unit. The hub 20006 includes a display 20048, an imaging module 20049, a generator module 20050, a communication module 20056, a processor module 20057, a storage array 20058, and an operating room mapping module 20059. In certain embodiments, as illustrated in FIG. 3, the hub 20006 further includes an exhaust smoke module 20054 and / or a suction / irrigation module 20055. The various modules and systems can be connected directly or via the communication module 20056 to the modular control unit via a router. The operating room devices can be coupled to cloud computing resources and data storage via the modular control unit. The human interface system 20012 can include a display subsystem and a notification subsystem.

[0083] The modular control unit may be connected to a non-contact sensor module. The non-contact sensor module may use ultrasonic, laser type, and / or similar non-contact measurement devices to measure the dimensions of the operating room and generate a map of the operating room. Other distance sensors can be used to determine the boundaries of the operating room. In U.S. Provisional Patent Application No. 62 / 611,341, filed on December 28, 2017, entitled "INTERACTIVE SURGICAL PLATFORM", which is hereby incorporated by reference in its entirety, as described under the heading "Surgical Hub Spatial Awareness Within an Operating Room" in the same document, an ultrasonic-based non-contact sensor module can scan the operating room by transmitting an ultrasonic burst and receiving an echo when the ultrasonic burst is reflected from the outer wall of the operating room. The sensor module may be configured to determine the size of the operating room and adjust the Bluetooth pairing distance limit. A laser-based non-contact sensor module can scan the operating room, for example, by transmitting a laser light pulse, receive the laser light pulse reflected from the outer wall of the operating room, compare the phase of the transmitted pulse with the received pulse to determine the size of the operating room, and adjust the Bluetooth pairing distance limit.

[0084] During surgery, applying energy to tissue for sealing and / or cutting is generally associated with smoke evacuation, aspiration of excess fluid, and / or perfusion of tissue. Fluid lines, power lines, and / or data lines from different sources often become entangled during surgery. Valuable time may be lost in addressing this problem during surgery. To untangle the lines, it may be necessary to remove the lines from their corresponding modules, and for this, it may be necessary to reset the modules. The hub module type enclosure 20060 provides an integrated environment for managing power lines, data lines, and fluid lines, reducing the frequency of such entanglements between lines. Aspects of the present disclosure present a surgical hub 20006 for use in surgeries involving applying energy to tissue at the surgical site. The surgical hub 20006 includes a hub enclosure 20060 and a combined generator module slidably receivable within the docking station of the hub enclosure 20060. The docking station includes data contacts and power contacts. The combined generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component housed within a single unit. In one aspect, the combined generator module also includes a smoke evacuation component, at least one energy supply cable for connecting the combined generator module to a surgical instrument, at least one smoke evacuation component configured to discharge smoke, fluid, and / or particulates generated by applying therapeutic energy to tissue, and a fluid line extending from a remote surgical site to the smoke evacuation component. In one aspect, the fluid line may be a first fluid line, and a second fluid line may extend from a remote surgical site to a suction and perfusion module 20055 slidably receivable within the hub enclosure 20060. In one aspect, the hub enclosure 20060 may include a fluid interface. Certain surgeries may require applying two or more energy types to tissue.One type of energy may be more beneficial for cutting tissue, while a different type of energy may be more beneficial for sealing tissue. For example, a bipolar generator can be used to seal tissue, while an ultrasonic generator can be used to cut the sealed tissue. Aspects of the present disclosure present a solution where a hub module enclosure 20060 is configured to house different generators and facilitate two-way communication between them. One advantage of the hub module enclosure 20060 is that it allows for the quick removal and / or replacement of various modules. Aspects of the present disclosure present a modular surgical enclosure for use in surgical procedures involving the application of energy to tissue. The modular surgical enclosure includes a first energy generator module configured to generate a first energy for application to tissue, and a first docking station having a first docking port including a first data and power contact, wherein the first energy generator module is slidably movable to electrically engage with the power and data contacts, and the first energy generator module is also slidably movable to disengage from the electrical engagement with the first power and data contacts. In addition to the above, the modular surgical enclosure also includes a second energy generator module configured to generate a second energy different from the first energy for application to tissue, and a second docking station having a second docking port including a second data contact and a second power contact, wherein the second energy generator module is slidably movable to electrically engage with the power and data contacts, and the second energy generator module is also slidably movable to disengage from the electrical engagement with the second power and data contacts. Additionally, the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port configured to facilitate communication between the first energy generator module and the second energy generator module.Referring to FIG. 3, aspects of the present disclosure are presented regarding a generator module 20050, a smoke exhaust module 20054, and a hub module type enclosure 20060 that enables modular integration of the suction / irrigation module 20055. The hub module type enclosure 20060 further facilitates two-way communication between the module 20059, the module 20054, and the module 20055. The generator module 20050 may include an integrated monopolar component, a bipolar component, and an ultrasonic component supported within a single housing unit slidably insertable into the hub's module type enclosure 20060. The generator module 20050 may be configured to connect to a monopolar device 20051, a bipolar device 20052, and an ultrasonic device 20053. Alternatively, the generator module 20050 may include a series of monopolar generator modules, bipolar generator modules, and / or ultrasonic generator modules that interact via the hub module type enclosure 20060. The hub module type enclosure 20060 can be configured to facilitate the insertion of multiple generators and two-way communication between the generators docked to the hub module type enclosure 20060 such that the multiple generators function as a single generator.

[0085] FIG. 4 illustrates a surgical data network having a set of communication hubs configured to connect to a cloud, a set of sensing systems, an environmental sensing system, and a set of other modular devices disposed in one or more operating rooms, patient recovery rooms, or rooms within a medical facility specially equipped for surgery within a medical facility, according to at least one aspect of the present disclosure.

[0086] As illustrated in FIG. 4, the surgical hub system 20060 may include a modular communication hub 20065 configured to connect modular devices disposed within a medical facility to a cloud-based system (e.g., a cloud computing system 20064 that may include a remote server 20067 connected to a remote storage 20068). The modular communication hub 20065 and the devices may be connected in a room within a medical facility specially equipped for surgical procedures. In one aspect, the modular communication hub 20065 may include a network hub 20061 and / or a network switch 20062 that communicate with a network router 20066. The modular communication hub 20065 may also be coupled to a local computer system 20063 to provide local computer processing and data manipulation.

[0087] The computer system 20063 may include a processor and a network interface 20100. The processor may be connected via a system bus to a communication module, storage, memory, non-volatile memory, and an input / output (I / O) interface. The system bus may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures, examples of which include a 9-bit bus, Industrial Standard Architecture (ISA), Micro-Charmel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), USB, Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), Small Computer Systems Interface (SCSI), or any other proprietary bus, but is not limited thereto.

[0088] The processor may be any single-core or multi-core processor, such as those known by the trade name ARM Cortex by Texas Instruments. In one aspect, the processor may be, for example, the LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments. This processor core includes on-chip memory of 256KB single-cycle flash memory or other non-volatile memory up to 40MHz, a prefetch buffer for improving performance beyond 40MHz, 32KB of single-cycle serial random access memory (SRAM), an internal read-only memory (ROM) with StellarisWare® software, 2KB of electrically erasable programmable read-only memory (EEPROM) and / or one or more pulse width modulation (PWM) modules, one or more quadrature encoder input (QEI) analogs, and one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, the details of which are available in the product datasheet.

[0089] In one example, the processor may include a safety controller with two controller-based families such as TMS570 and RM4x, also known by the trade name Hercules ARM Cortex R4 from Texas Instruments. The safety controller may be configured specifically for safety-critical applications of IEC61508 and ISO26262, among others, while providing scalable performance, connectivity, and memory options, and providing a high degree of integrated safety mechanisms.

[0090] It should be understood that the computer system 20063 may include software that functions as a medium between the described user and the basic computer resources in a suitable operating environment. Examples of such software include operating systems. An operating system that may be stored on disk storage may function to control and allocate the resources of the computer system. System applications may utilize the resource management by the operating system via program modules and program data stored either in system memory or on disk storage. It should be understood that the various components described herein can be implemented with various operating systems or combinations of operating systems.

[0091] A user can input commands or information into the computer system 20063 via an input device connected to the I / O interface. Examples of input devices include, but are not limited to, pointing devices such as a mouse, trackball, stylus, touchpad, keyboard, microphone, joystick, gamepad, satellite broadcast receiving antenna, scanner, TV tuner card, digital camera, digital video camera, webcam, etc. These and other input devices are connected to the processor 20102 through the system bus via an interface port. Examples of interface ports include serial ports, parallel ports, game ports, and USB. Output devices use some of the same types of ports as input devices. Thus, for example, a USB port may be used to provide input to the computer system 20063 and output information from the computer system 20063 to an output device. Output adapters may be provided to illustrate that among output devices that may require special adapters, there can be several output devices such as monitors, displays, speakers, and printers. Examples of output adapters include, but are not limited to, video and sound cards that provide connection means between the output device and the system bus. Note that other devices and / or systems of devices, such as remote computers, can provide both input and output functions.

[0092] The computer system 20063 can operate in a networked environment that uses logical connections to one or more remote computers, such as a cloud computer, or a local computer. The remote cloud computer can be, for example, a personal computer, a server, a router, a network PC, a workstation, a microprocessor-based device, a peer device, or other common network nodes, but typically includes many or all of the elements described with respect to the computer system. For simplicity, only a memory storage device is illustrated along with the remote computer. The remote computer can be logically connected to the computer system via a network interface and subsequently physically connected via a communication connection. The network interface can include communication networks such as a local area network (LAN) and a wide area network (WAN). Examples of LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet / IEEE802.3, Token Ring / IEEE802.5, etc. Examples of WAN technologies include circuit-switched networks such as point-to-point links, Integrated Services Digital Network (ISDN) and its variants, packet-switched networks, and Digital Subscriber Line (DSL), but are not limited thereto.

[0093] In various examples, computer system 20063 may include an image processor, an image processing engine, a media processor, or any special digital signal processor (DSP) used for processing digital images. The image processor can enhance speed and efficiency using parallel computing with single instruction, multiple data (SIMD), or multiple instruction, multiple data (MIMD) techniques. The digital image processing engine can perform various tasks. The image processor may be a system on a chip with a multi-core processor architecture.

[0094] The communication connection part may refer to the hardware / software used to connect a network interface to a bus. For exemplary clarity, the communication connection part is shown inside computer system 20063, but the communication connection part may be outside computer system 20063. For illustrative purposes only, the hardware / software required for connection to a network interface can include modems such as ordinary telephone grade modems, cable modems, fiber optic modems, and DSL modems, ISDN adapters, and internal and external technologies such as Ethernet cards. In some examples, the network interface may also be provided using an RF interface.

[0095] The surgical data network associated with the surgical hub system 20060 may be configured as passive, intelligent, or switching. A passive surgical data network functions as a conduit for data, enabling data to go from one device (or segment) to another device (or segment) and to cloud computing resources. An intelligent surgical data network enables traffic to pass through the surgical data network being monitored and includes additional features that configure each port within the network hub 20061 or network switch 20062. An intelligent surgical data network may be referred to as a manageable hub or switch. A switching hub reads the destination address of each packet and then forwards the packet to the correct port.

[0096] The modular devices 1a - 1n arranged in the operating room can be connected to the modular communication hub 20065. The network hub 20061 and / or the network switch 20062 can be connected to the network router 20066 to connect the devices 1a - 1n to the cloud computing system 20064 or the local computer system 20063. The data associated with the devices 1a - 1n may be transferred via the router to a cloud - based computer for remote data processing and operation. The data associated with the devices 1a - 1n can also be transferred to the local computer system 20063 for local data processing and operation. The modular devices 2a - 2m arranged in the same operating room may also be connected to the network switch 20062. The network switch 20062 can be connected to the network hub 20061 and / or the network router 20066 to connect the devices 2a - 2m to the cloud 20064. The data associated with the devices 2a - 2m can be transferred via the network router 20066 to the cloud computing system 20064 for data processing and operation. The data associated with the devices 2a - 2m may also be transferred to the local computer system 20063 for local data processing and operation.

[0097] The wearable sensing system 20011 may include one or more sensing systems 20069. The sensing system 20069 may include an HCP sensing system and / or a patient sensing system. One or more sensing systems 20069 may communicate with the computer system 20063 of the surgical hub system 20060 or the cloud server 20067 directly via one of the network routers 20066 or via a network hub 20061 or network switching 20062 that communicates with the network router 20066.

[0098] The sensing system 20069 can be coupled to a network router 20066 to connect the sensing system 20069 to a local computer system 20063 and / or a cloud computing system 20064. Data associated with the sensing system 20069 can be transferred via the network router 20066 to the cloud computing system 20064 for data processing and manipulation. Data associated with the sensing system 20069 may also be transferred to the local computer system 20063 for local data processing and manipulation.

[0099] As illustrated in FIG. 4, the surgical hub system 20060 can be expanded by interconnecting a plurality of network hubs 20061 and / or a plurality of network switches 20062 with a plurality of network routers 20066. The modular communication hub 20065 can be housed within a modular control tower configured to receive a plurality of devices 1a-1n / 2a-2m. The local computer system 20063 may also be housed within the modular control tower. The modular communication hub 20065 can be connected to a display 20068 to display images obtained by some of the devices 1a-1n / 2a-2m, for example, during a surgical procedure. In various aspects, the devices 1a-1n / 2a-2m can include various modules such as an imaging module coupled to an endoscope, a generator module coupled to an energy-based surgical device, a smoke evacuation module, a suction / irrigation module, a communication module, a processor module, a storage array, a surgical device coupled to a display, and / or a non-contact sensor module, among other modular devices that can be coupled to the modular communication hub 20065 of a surgical data network.

[0100] In one aspect, the surgical hub system 20060 illustrated in FIG. 4 may include a combination of a network hub(s), network switch, and network router(s) that connect devices 1a - 1n / 2a - 2m, or the sensing system 20069, to the cloud - based system 20064. One or more of the devices 1a - 1n / 2a - 2m or the sensing system 20069 connected to the network hub 20061 or the network switch 20062 may collect data in real - time and transfer the data to a cloud computer for data processing and operation. It will be understood that cloud computing relies on sharing computing resources rather than having local servers or personal devices to handle software applications. The term "cloud" may be used as a metaphor for the "Internet", but this term is not so limited. Thus, the term "cloud computing" can be used herein to refer to "one type of Internet - based computing", in which case various services such as servers, storage, and applications are distributed via the Internet to the modular communication hub 20065 and / or the computer system 20063 located in an operating room (e.g., a fixed, mobile, temporary, or on - site operating room or space), and to devices connected to the modular communication hub 20065 and / or the computer system 20063. The cloud infrastructure may be maintained by a cloud service provider. In this context, the cloud service provider may be an entity that coordinates the use and control of the devices 1a - 1n / 2a - 2m located in one or more operating rooms. Cloud computing services may perform a number of calculations based on data collected by smart surgical instruments, robots, sensing systems, and other computerized devices located in the operating room. The hub hardware enables multiple devices, sensing systems, and / or connections to connect to a computer that communicates with cloud computing resources and storage.

[0101] By applying cloud computing data processing technology to the data collected by devices 1a to 1n / 2a to 2m, the surgical data network can provide improvements in surgical outcomes, cost reduction, and patient satisfaction. After tissue sealing and cutting procedures, at least some of devices 1a to 1n / 2a to 2m can be used to observe the state of the tissue to evaluate leakage or perfusion of the sealed tissue. At least some of devices 1a to 1n / 2a to 2m can be used to examine data including images of samples of body tissue for diagnostic purposes to identify pathologies such as the effects of diseases. This can include tissue localization, margin confirmation, and phenotype. At least some of devices 1a to 1n / 2a to 2m can be used to identify the anatomical structure of the body using various sensors integrated with the imaging device and techniques such as overlaying images captured by multiple imaging devices. The data collected by devices 1a to 1n / 2a to 2m, including image data, can be transferred to a cloud computing system 20064 or a local computer system 20063 or both for data processing and operations including image processing and manipulation. The data may be analyzed to improve the results of surgery by determining whether further treatments such as endoscopic interventions, emerging technologies, targeted radiation, targeted interventions, and precision robots can be performed on tissue-specific sites and conditions. Such data analysis may further employ prognostic analysis processing, and using standardized methods can provide useful feedback either to confirm surgical treatment and surgeon behavior or to propose modifications to surgical treatment and surgeon behavior.

[0102] Applying cloud computer data processing technology to the measurement data collected by the sensing system 20069 can result in improved surgical outcomes, improved recovery outcomes, reduced costs, and improved patient satisfaction. At least some of the sensing system 20069 may be used to evaluate the physiological state of a surgeon performing surgery on a patient, a patient being prepared for surgery, or a patient recovering after surgery. The cloud-based computing system 20064 can monitor biomarkers associated with a surgeon or patient in real time, generate a surgical plan based at least on the measurement data collected before surgery, supply control signals to surgical instruments during surgery, and be used to notify a patient of complications during the postoperative period.

[0103] The operating room devices 1a to 1n can be connected to the modular communication hub 20065 via a wired channel or a wireless channel according to the configuration of the devices 1a to 1n with respect to the network hub 20061. In one aspect, the network hub 20061 may be implemented as a local network broadcast device that functions on the physical layer of the Open System Interconnection (OSI) model. The network hub can provide connectivity to the devices 1a to 1n located within the same operating room network. The network hub 20061 can collect data in the form of packets and transmit them to the router in half-duplex mode. The network hub 20061 cannot store any media access control / Internet Protocol (MAC / IP) for transferring device data. Only one of the devices 1a to 1n can transmit data at a time via the network hub 20061. The network hub 20061 cannot have a routing table or intelligence regarding the destination of information and broadcasts all network data across each connection and to the remote server 20067 of the cloud computing system 20064. The network hub 20061 can detect basic network errors such as collisions, but broadcasting all information to multiple ports can pose a security risk and cause bottlenecks.

[0104] The operating room devices 2a to 2m can be connected to the network switch 20062 via a wired channel or a wireless channel. The network switch 20062 functions within the data link layer of the OSI model. The network switch 20062 may be a multicast device for connecting the devices 2a to 2m arranged in the same operating room to the network. The network switch 20062 transmits data in the form of frames to the network router 20066 and can function in full-duplex mode. A plurality of devices 2a to 2m can transmit data simultaneously via the network switch 20062. The network switch 20062 stores and uses the MAC addresses of the devices 2a to 2m for transferring data.

[0105] The network hub 20061 and / or the network switch 20062 can be connected to the network router 20066 to connect to the cloud computing system 20064. The network router 20066 functions within the network layer of the OSI model. The network router 20066 creates a route for transmitting the data packets received from the network hub 20061 and / or the network switch 20062 to the cloud-based computer resources for further processing and operation of the data collected by any one or all of the devices 1a to 1n / 2a to 2m and the wearable sensing system 20011. The network router 20066 may be used to connect two or more different networks located at different positions, such as different operating rooms in the same medical facility or different networks in different operating rooms of different medical facilities. The network router 20066 transmits data in the form of packets to the cloud computing system 20064 and can function in full-duplex mode. A plurality of devices can transmit data simultaneously. The network router 20066 can use IP addresses for transferring data.

[0106] In one example, the network hub 20061 may be implemented as a USB hub that enables a plurality of USB devices to be connected to a host computer. The USB hub can expand a single USB port into several levels so that there are more available ports for connecting devices to the host system computer. The network hub 20061 may include a wired function or a wireless function for receiving information via a wired channel or a wireless channel. In one aspect, a wireless USB short-range high-bandwidth wireless communication protocol may be used for communication between devices 1a to 1n and devices 2a to 2m located within an operating room.

[0107] In an example, the operating room devices 1a-1n / 2a-2m and / or the sensing system 20069 may communicate with the modular communication hub 20065 via the Bluetooth wireless technology standard to exchange data over a short distance from fixed and mobile devices (using short wavelength UHF radio waves in the 2.4-2.485 GHz ISM band) and to construct a personal area network (PAN). The operating room devices 1a-1n / 2a-2m and / or the sensing system 20069 may communicate with the modular communication hub 20065 via some wireless communication standards or wired communication standards or protocols, such as Bluetooth, Low-Energy Bluetooth, near-field communication (NFC), Wi-Fi (IEEE802.11 family), WiMAX (IEEE802.16 family), IEEE802.20, New Radio (NR), Long Term Evolution (LTE), and Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, and Ethernet derivatives thereof, as well as 3G, 4G, 5G, and any other wireless protocols and wired protocols designated in the future, but not limited thereto. The computing module may include a plurality of communication modules. For example, the first communication module may be dedicated to short-range wireless communication such as Wi-Fi and Bluetooth, Low-Energy Bluetooth, Bluetooth Smart, etc., and the second communication module may be dedicated to long-range wireless communication such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, etc.

[0108] The modular communication hub 20065 functions as a central connection for one or more of the operating room devices 1a - 1n / 2a - 2m and / or the sensing system 20069 and can handle data types known as frames. Frames can carry data generated by the devices 1a - 1n / 2a - 2m and / or the sensing system 20069. When a frame is received by the modular communication hub 20065, the frame is amplified and sent to the network router 20066, which can transfer this data to the cloud computing system 20064 or the local computer system 20063 by using a number of wireless communication standards or wired communication standards or protocols as described herein.

[0109] The modular communication hub 20065 may be used as a stand - alone device or may be connected to compatible network hubs 20061 and network switches 20062 to form a larger network. The modular communication hub 20065 can be a good option for networking the operating room devices 1a - 1n / 2a - 2m because it is generally easy to install, configure, and maintain.

[0110] FIG. 5 illustrates a logic diagram of a control system 20220 for a surgical instrument or a surgical tool according to one or more aspects of the present disclosure. The surgical instrument or the surgical tool may be configurable. The surgical instrument may be at hand, such as an imaging device, a surgical stapler, an energy device, an end cutter device, and may include surgical supplies specific to the procedure. For example, the surgical instrument may include any of an electric stapler, an electric stapler generator, an energy device, a high-energy device, a high-energy Joe device, an end cutter clamp, an energy device generator, an in-operation room imaging system, a smoke exhaust device, a suction irrigation device, a pneumoperitoneum system, etc. The system 20220 may include a control circuit. The control circuit may include a microcontroller 20221 including a processor 20222 and a memory 20223. For example, one or more of sensors 20225, 20226, 20227 provide real-time feedback to the processor 20222. A motor 20230 driven by a motor driver 20229 is operably connected to a longitudinally movable displacement member to drive an I-beam knife element. A tracking system 20228 may be configured to determine the position of the longitudinally movable displacement member. The position information may be provided to a processor 20222 that may be programmed or configured to determine the position of the longitudinally movable drive member as well as the positions of the firing member, the firing bar, and the I-beam knife element. An additional motor may be provided to the tool driver interface to control the firing of the I-beam, the movement of the closure tube, the rotation of the shaft, and the articulation movement. A display 20224 may display various operating states of the instrument and may include a touch screen function for data input. The information displayed on the display 20224 may be overlaid with an image acquired via an endoscope imaging module.

[0111] The microcontroller 20221 may be any single-core or multi-core processor, such as those known by the product names of ARM Cortex from Texas Instruments. In one aspect, the main microcontroller 20221 may be, for example, an on-chip memory of 256KB single-cycle flash memory or other non-volatile memory with a maximum of 40MHz, the details of which are available in the product datasheet, a prefetch buffer for improving performance beyond 40MHz, 32KB single-cycle SRAM, an internal ROM with StellarisWare® software, 2KB EEPROM, one or more PWM modules, one or more QEI analogs, and / or one or more 12-bit ADCs with 12 analog input channels, which may be the LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments.

[0112] The microcontroller 20221 may also include a safety controller with two controller-based families such as TMS570 and RM4x known by the product names of Hercules ARM Cortex R4 from Texas Instruments. The safety controller may be configured specifically for safety-critical applications of IEC61508 and ISO26262, among others, to provide a scalable performance, connectivity, and memory options while providing a high-level integrated safety mechanism.

[0113] The microcontroller 20221 may be programmed to perform various functions such as precise control over the speed and position of the knife and the articulation motion system. In one aspect, the microcontroller 20221 may include a processor 20222 and a memory 20223. The electric motor 20230 may be a brushed direct current (DC) motor with a gearbox and a mechanical coupling to the articulation motion section or the knife system. In one aspect, the motor driver 20229 may be an A3941 available from Allegro Microsystems, Inc. Other motor drivers may be readily substituted for use in the tracking system 20228 with an absolute positioning system. A detailed description of the absolute positioning system is described in U.S. Patent Application Publication No. 2017 / 0296213, titled "SYSTEMS AND METHODS FOR CONTROLLING A SURGICAL STAPLING AND CUTTING INSTRUMENT", published on October 19, 2017, which is hereby incorporated by reference in its entirety.

[0114] The microcontroller 20221 may be programmed to provide accurate control over the speed and position of the displacement member and the articulation motion system. The microcontroller 20221 may be configured to calculate a response within the software of the microcontroller 20221. The calculated response may be compared to the measured response of the actual system to obtain an "observed" response, which is used for actual feedback decision-making. The observed response may be a suitably adjusted value that balances the smooth and continuous nature of the simulated response with the measured response, which can detect external influences on the system.

[0115] The motor 20230 may be controlled by a motor driver 20229 and can also be used by a surgical instrument or a tool firing system. In various forms, the motor 20230 may be a brushed DC drive motor having a maximum rotational speed of about 25,000 RPM. In some examples, the motor 20230 may include a brushless motor, a cordless motor, a synchronous motor, a stepper motor, or any other suitable electric motor. The motor driver 20229 may include, for example, an H-bridge driver comprising field effect transistors (FETs). The motor 20230 may be powered by a power supply assembly releasably attached to a handle assembly or a tool housing to supply control power to a surgical instrument or a tool. The power supply assembly may include a battery that may include a number of battery cells connected in series that can be used as a power source to power the surgical instrument or tool. In certain situations, the battery cells of the power supply assembly may be replaceable and / or rechargeable. In at least one example, the battery cells may be lithium-ion batteries that can be connectable to and separable from the power supply assembly.

[0116] The motor driver 20229 may be the A3941 available from Allegro Microsystems, Inc. The A3941 may be a full-bridge controller for use with an external N-channel power metal-oxide semiconductor field-effect transistor (MOSFET) specifically designed for inductive loads such as brushed DC motors. The driver 20229 may include an inherent charge pump regulator, which supplies a full (>10V) gate drive to a battery voltage up to 7V, enabling the A3941 to operate with a reduced gate drive up to 5.5V. A bootstrap capacitor may be used to supply the above battery supply voltage required for the N-channel MOSFET. The internal charge pump for high-side drive enables DC (100% duty cycle) operation. The full bridge can be driven in fast or slow decay mode using diodes or synchronous rectification. In slow decay mode, current recirculation is possible by either the high-side FET or the low-side FET. The power FET can be protected from shoot-through by a resistor-adjustable dead time. The integrated diagnostics indicate low voltage, overtemperature, and power bridge anomalies and can be configured to protect the power MOSFET under most short-circuit conditions. Other motor drivers may be easily substituted for use in the tracking system 20228 with an absolute positioning system.

[0117] Tracking system 20228 may comprise a controlled motor drive circuit configuration with a position sensor 20225 according to one aspect of the present disclosure. The position sensor 20225 for an absolute positioning system may supply a unique position signal corresponding to the position of the displacement member. In some examples, the displacement member may represent a longitudinally movable drive member having a rack of drive teeth for meshing engagement with a corresponding drive gear of a gear reduction assembly. In some examples, the displacement member may represent a firing member adapted and configured to include a rack of drive teeth. In some examples, the displacement member may represent a firing bar or an I-beam, each of which may be adapted and configured to include a rack of drive teeth. Thus, as used herein, the term displacement member may generally be used to refer to any movable member of a surgical instrument or tool, such as a drive member, a firing member, a firing bar, an I-beam, or any element that may be displaced. In one aspect, the longitudinally movable drive member may be coupled to the firing member, the firing bar, and the I-beam. Thus, the absolute positioning system may actually track the linear displacement of the I-beam by tracking the linear displacement of the longitudinally movable drive member. In various aspects, the displacement member may be coupled to any position sensor 20225 suitable for measuring linear displacement. Thus, the longitudinally movable drive member, the firing member, the firing bar, or the I-beam, or combinations thereof, may be coupled to any suitable linear displacement sensor. The linear displacement sensor may include a contact displacement sensor or a non-contact displacement sensor.The linear displacement sensor may include a linear variable differential transformer (LVDT), a differential variable reluctance transducer (DVRT), a slide potentiometer, a magnetic sensing system including a movable magnet and a series of linearly arranged Hall effect sensors, a magnetic sensing system including a fixed magnet and a series of linearly arranged movable Hall effect sensors, an optical detection system including a movable light source and a series of linearly arranged light diodes or photodetectors, an optical sensing system including a fixed light source and a series of linearly arranged movable photodiodes or photodetectors, or any combination thereof.

[0118] The electric motor 20230 may include a rotatable shaft that operably interfaces with a gear assembly attached to mesh with a set of drive teeth or a rack on the displacement member. The sensor element may be operably coupled to the gear assembly such that one rotation of the position sensor 20225 element corresponds to some linear longitudinal translation of the displacement member. The configuration of the gear ring and sensor may be connected to a linear actuator by a rack and pinion configuration or to a rotary actuator by spur gears or other connections. A power supply may supply power to the absolute positioning system, and an output indicator may display the output of the absolute positioning system. The displacement member may represent a longitudinally movable drive member having a rack of drive teeth formed thereon for meshing with a corresponding drive gear of a gear reduction assembly. The displacement member may represent a longitudinally movable firing member, firing bar, I-beam, or combination thereof.

[0119] One rotation of the sensor element associated with the position sensor 20225 can correspond to a longitudinal linear displacement d1 of the displacement member, where d1 is the longitudinal linear distance that the displacement member moves from point "a" to point "b" after one rotation of the sensor element connected to the displacement member. The sensor device can be connected via a gear reduction device in which the position sensor 20225 completes one or more rotations with respect to the full stroke of the displacement member. The position sensor 20225 may complete multiple rotations with respect to the full stroke of the displacement member.

[0120] To provide a unique position signal for two or more rotations of the position sensor 20225, a series of switches (where n is an integer greater than 1) may be used alone or in combination with a gear reduction device. The state of the switch can be fed back to the microcontroller 20221, which applies logic to determine a unique position signal corresponding to the longitudinal linear displacement d1 + d2 +... dn of the displacement member. The output of the position sensor 20225 is supplied to the microcontroller 20221. The position sensor 20225 of the sensor device may comprise a magnetic sensor, an analog rotational sensor such as a potentiometer, or an array of analog Hall effect elements that output a unique combination of position signals or values.

[0121] The position sensor 20225 may comprise any number of magnetic sensing elements such as, for example, a magnetic sensor classified by measuring the total magnetic field or a vector component of the magnetic field. The technologies used to produce both types of magnetic sensors can encompass many aspects of physics and electronics. Technologies used for sensing magnetic fields include, among others, search coils, flux gates, optical pumping, nuclear precession, SQUIDs, Hall effect, anisotropic magnetoresistance, giant magnetoresistance, magnetic tunnel junctions, giant magnetic impedance, magnetostrictive / piezoelectric composites, magnetic diodes, magnetic transistors, optical fibers, magneto-optics, and microelectromechanical system-based magnetic sensors.

[0122] The position sensor 20225 of the tracking system 20228 with an absolute positioning system may include a magnetic rotary absolute positioning system. The position sensor 20225 may be implemented as an AS5055EQFT single-chip magnetic rotary position sensor available from Austria Microsystems, AG. The position sensor 20225 is connected to the microcontroller 20221 to realize an absolute positioning system. The position sensor 20225 is a low-voltage and low-power component and may include four Hall effect elements in the area of the position sensor 20225 that can be arranged above the magnet. Also, a high-resolution ADC and a smart power management controller may be provided on the chip. A coordinate rotation digital computer (CORDIC) processor, also known as the digit-by-digit method and the border algorithm, may be provided to implement a concise and efficient algorithm for calculating hyperbolic and trigonometric functions that only require addition, subtraction, bit shifting, and table reference operations. The angular position, alarm bit, and magnetic field information may be transmitted to the microcontroller 20221 via a standard serial communication interface such as a serial peripheral interface (SPI) interface. The position sensor 20225 may provide a resolution of 12 bits or 14 bits. The position sensor 20225 may be an AS5055 chip provided in a small QFN16-pin 4×4×0.85 mm package.

[0123] Tracking system 20228 with an absolute positioning system may include and / or may be programmed to implement a feedback controller such as a PID, state feedback, and adaptive controller. The power supply converts a signal from the feedback controller into a physical input to the system, in this case a voltage. Other examples include PWM of voltage, current, and force. In addition to the position measured by position sensor 20225, other sensors (s) may be provided to measure physical parameters of the physical system. In some aspects, examples of other sensors (s) may include sensor arrangements such as those described in U.S. Patent No. 9,345,481, issued May 24, 2016, titled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is hereby incorporated by reference in its entirety, U.S. Patent Application Publication No. 2014 / 0263552, published September 18, 2014, titled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is hereby incorporated by reference in its entirety, and U.S. Patent Application No. 15 / 628,175, filed June 20, 2017, titled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," which is hereby incorporated by reference in its entirety. In a digital signal processing system, the absolute positioning system is coupled to a digital data acquisition system, where the output of the absolute positioning system has a finite resolution and sampling frequency. The absolute positioning system may include a comparison and combination circuit to combine the calculated response with the measured response using algorithms such as weighted averages and theoretical control loops that drive the calculated response towards the measured response. The calculated response of the physical system may take into account characteristics such as mass, inertia, viscous friction, inductive resistance, etc. to predict how the state and output of the physical system will be based on knowing the input.

[0124] An absolute positioning system can provide the absolute position of a displacement member upon power-up of the instrument without retracting or advancing the displacement member to a reset (zero or home) position, which may require a conventional rotary encoder that simply counts the number of forward or backward steps taken by a motor 20230 to estimate the position of a device actuator, drive bar, knife, etc.

[0125] For example, a sensor 20226, such as a strain gauge or a micro-strain gauge, may be configured to measure one or more parameters of an end effector, such as, for example, the closing force applied to an anvil, or the amplitude of the strain exerted on the anvil during a clamping operation. The measured strain can be converted into a digital signal and provided to a processor 20222. Instead of, or in addition to, the sensor 20226, a sensor 20227, such as a load cell, for example, may measure the closing force applied to the anvil by a closing drive system. For example, a sensor 20227, such as a load cell, may measure the firing force applied to an I-beam during a firing stroke of a surgical instrument or tool. The I-beam is configured to engage a wedge thread, which is configured to cam a staple driver upward to eject a staple into deformation contact with the anvil. The I-beam may also include a sharp cutting edge that can be used to cut tissue when the I-beam is advanced distally by a firing bar. Alternatively, a current sensor 20231 may be used to measure the current consumed by the motor 20230. The force required to advance the firing member can correspond to, for example, the current drawn by the motor 20230. The measured force can be converted into a digital signal and provided to the processor 20222.

[0126] For example, a strain gauge sensor 20226 may be used to measure the force applied to tissue by an end effector. To measure the force exerted by the end effector on the tissue being treated, the strain gauge may be coupled to the end effector. A system for measuring the force applied to tissue grasped by an end effector may include a strain gauge sensor 20226, such as a micro strain gauge, configured to measure one or more parameters of the end effector. In one aspect, the strain gauge sensor 20226 can measure the amplitude or magnitude of the strain exerted on the jaw members of the end effector during a clamping operation, which can indicate tissue compression. The measured strain can be converted to a digital signal and supplied to a processor 20222 of a microcontroller 20221. A load sensor 20227 may measure, for example, the force used to operate a knife element to cut tissue captured between an anvil and a staple cartridge. A magnetic field sensor can be used to measure the thickness of the captured tissue. The measurements of the magnetic field sensor may also be converted to a digital signal and provided to the processor 20222.

[0127] The measurements of tissue compression, tissue thickness, and / or the force required to close the end effector on the tissue, respectively measured by sensors 20226, 20227, may be used by the microcontroller 20221 to characterize corresponding values of a selected position of the firing member and / or the velocity of the firing member. In one embodiment, the memory 20223 may store techniques, equations, and / or look-up tables that may be used by the microcontroller 20221 during evaluation.

[0128] The control system 20220 of the surgical instrument or tool may also include a wired or wireless communication circuit for communicating with a surgical hub 20065 as shown in FIG. 4.

[0129] FIG. 6 illustrates an exemplary surgical system 20280 according to the present disclosure, which may include a surgical instrument 20282 that can communicate with a console 20294 or a portable device 20296 through a local area network 20292 and / or a cloud network 20293 via a wired and / or wireless connection. The console 20294 and the portable device 20296 may be any suitable computing device. The surgical instrument 20282 may include a handle 20297, an adapter 20285, and a loading unit 20287. The adapter 20285 is releasably coupled to the handle 20297, and the loading unit 20287 is releasably coupled to the adapter 20285 such that the adapter 20285 transmits force from the drive shaft to the loading unit 20287. The adapter 20285 or the loading unit 20287 may include a force gauge (not explicitly shown) disposed therein for measuring the force applied to the loading unit 20287. The loading unit 20287 may include an end effector 20289 having a first jaw 20291 and a second jaw 20290. The loading unit 20287 may be an in vivo loading unit, that is, a multi-firing loading unit (MFLU), that allows a clinician to fire a plurality of fasteners multiple times without the need to remove the loading unit 20287 from the surgical site in order to reload the loading unit 20287.

[0130] The first jaw 20291 and the second jaw 20290 may be configured to clamp tissue therebetween, fire a fastener through the clamped tissue, and cut the clamped tissue. The first jaw 20291 may be configured to fire at least one fastener multiple times, or may be configured to include a replaceable multi-firing fastener cartridge that can fire a plurality of fasteners (e.g., staples, clips, etc.) two or more times before being replaced. The second jaw 20290 may include an anvil that deforms or otherwise secures the fastener as the fastener is ejected from the multi-firing fastener cartridge.

[0131] The handle 20297 may include a motor coupled to the drive shaft so as to act on the rotation of the drive shaft. The handle 20297 may include a control interface for selectively activating the motor. The control interface may include buttons, switches, levers, sliders, touchscreens, and any other suitable input mechanism or user interface, which may be engaged by a clinician to activate the motor.

[0132] The control interface of the handle 20297 may communicate with the controller 20298 of the handle 20297 to selectively activate the motor and act on the rotation of the drive shaft. The controller 20298 may be disposed within the handle 20297 and may be configured to receive inputs from the control interface and adapter data from the adapter 20285 or loading unit data from the loading unit 20287. The controller 20298 may analyze the inputs from the control interface and the data received from the adapter 20285 and / or the loading unit 20287 to selectively activate the motor. The handle 20297 may also include a display visible to the clinician during use of the handle 20297. The display may be configured to display portions of the adapter or loading unit data before, during, or after firing of the instrument 20282.

[0133] The adapter 20285 may include an adapter identification device 20284 disposed therein, and the loading unit 20287 may include a loading unit identification device 20288 disposed therein. The adapter identification device 20284 may communicate with the controller 20298, and the loading unit identification device 20288 may communicate with the controller 20298. It will be appreciated that the loading unit identification device 20288 may communicate with the adapter identification device 20284 that relays or passes the communication from the loading unit identification device 20288 to the controller 20298.

[0134] Adapter 20285 may also include a plurality of sensors 20286 (one is shown) disposed therearound to detect various states of the adapter 20285 or the environment (e.g., whether the adapter 20285 is connected to the loading unit, whether the adapter 20285 is connected to the handle, whether the drive shaft is rotating, the torque of the drive shaft, the strain of the drive shaft, the temperature within the adapter 20285, the number of firings of the adapter 20285, the peak force of the adapter 20285 during firing, the total amount of force applied to the adapter 20285, the peak recoil force of the adapter 20285, the number of rest periods of the adapter 20285 during firing, etc.). The plurality of sensors 20286 may provide an input to the adapter identification device 20284 in the form of a data signal. The data signals of the plurality of sensors 20286 may be stored within the adapter identification device 20284 or may be used to update the adapter data stored within the adapter identification device 20284. The data signals of the plurality of sensors 20286 may be analog or digital. The plurality of sensors 20286 may include a force gauge for measuring the force exerted on the loading unit 20287 during firing.

[0135] The handle 20297 and the adapter 20285 may be configured to interconnect the adapter identification device 20284 and the loading unit identification device 20288 with the controller 20298 via an electrical interface. The electrical interface may be a direct electrical interface (i.e., including electrical contacts that engage with each other to transmit energy and signals therebetween). Additionally, or alternatively, the electrical interface may be a non-contact electrical interface for wirelessly transmitting (e.g., inductively transmitting) energy and signals therebetween. It is also contemplated that the adapter identification device 20284 and the controller 20298 may wirelessly communicate with each other via a wireless connection separate from the electrical interface.

[0136] The handle 20297 may include a transceiver 20283 configured to transmit instrument data from the controller 20298 to other components of the system 20280 (e.g., the LAN 20292, the cloud 20293, the console 20294, or the portable device 20296). The controller 20298 may also transmit instrument data and / or measurement data associated with one or more sensors 20286 to the surgical hub. The transceiver 20283 may receive data (e.g., cartridge data, loading unit data, adapter data, or other notifications) from the surgical hub 20270. The transceiver 20283 may receive data (e.g., cartridge data, loading unit data, or adapter data) from other components of the system 20280. For example, the controller 20298 may transmit instrument data including the serial number of a mounting adapter (e.g., adapter 20285) attached to the handle 20297, the serial number of a loading unit (e.g., loading unit 20287) attached to the adapter 20285, and the serial numbers of a plurality of firing fastener cartridges loaded in the loading unit to the console 20294. The console 20294 may then reply to the controller 20298 with data (e.g., cartridge data, loading unit data, or adapter data) associated with the attached cartridge, loading unit, and adapter, respectively. The controller 20298 may display a message on the local instrument display or, alternatively, transmit a message via the transceiver 20283 to the console 20294 or the portable device 20296 to display the message on the display 20295 or the portable device screen, respectively.

[0137] FIG. 7 illustrates a diagram of a situation awareness surgical system 5100 according to at least one aspect of the present disclosure. The data source 5126 can include, for example, a modular device 5102 (which can include sensors configured to detect parameters associated with a patient, an HCP, and the environment, and / or the modular device itself), a database 5122 (e.g., an EMR database including patient records), a patient monitoring device 5124 (e.g., a blood pressure (BP) monitor and an electrocardiography (EKG) monitor), an HCP monitoring device 35510, and / or an environmental monitoring device 35512. The surgical hub 5104 can be configured to derive context information regarding a surgical procedure from the data, for example, based on a particular combination of the received data or the particular order in which data is received from the data source 5126. The context information inferred from the received data can include, for example, the type of surgical procedure being performed, a particular step of the surgical procedure that the surgeon is performing, the type of tissue being operated on, or the body cavity that is the subject of the procedure. This function, according to some aspects of the surgical hub 5104 for deriving or inferring information regarding a surgical procedure from the received data, can be referred to as “situation awareness”. For example, the surgical hub 5104 can incorporate a situation awareness system, which is the hardware and / or programming associated with the surgical hub 5104 that derives context information regarding a surgical procedure from received data and / or surgical planning information received from an edge computing system 35514 or a corporate cloud server 35516.

[0138] The situation recognition system of the surgical hub 5104 can be configured to derive context information from data received from various different data sources 5126. For example, the situation recognition system can include a pattern recognition system, or a machine learning system (such as an artificial neural network) trained with training data to correlate various inputs (such as data from the database 5122, the patient monitoring device 5124, the modular device 5102, the HCP monitoring device 35510, and / or the environmental monitoring device 35512) with corresponding context information regarding the surgical procedure. The machine learning system can be trained to accurately derive context information regarding the surgical procedure from the provided inputs. In an example, the situation recognition system can include a look-up table that stores pre-characterized context information regarding the surgical procedure in association with one or more inputs (or a range of inputs) corresponding to that context information. In response to a query with one or more inputs, the look-up table can return the corresponding context information of the situation recognition system to control the modular device 5102. In an example, the context information received by the situation recognition system of the surgical hub 5104 can be associated with specific control adjustments, or a series of control adjustments, of one or more modular devices 5102. In an example, the situation recognition system can include a further machine learning system, a look-up table, or other such system that generates or reads one or more control adjustments of one or more modular devices 5102 when context information is provided as an input.

[0139] The surgical hub 5104 incorporating the situation awareness system can provide many advantages to the surgical system 5100. One advantage may include providing improved interpretation of the sensed and collected data, which can improve the processing accuracy during the surgical procedure and / or the use of the data. Returning to the previous example, the situation awareness surgical hub 5104 can determine which type of tissue is being operated on, and thus, if an unexpectedly high force is detected to close the end effector of the surgical instrument, the situation awareness surgical hub 5104 can correctly accelerate or decelerate the motor of the surgical instrument according to the tissue type.

[0140] The type of tissue being operated on can affect the adjustments made to the compression speed and load threshold of a surgical stapling and cutting instrument for specific tissue gap measurements. The situation awareness surgical hub 5104 can infer whether the surgical procedure being performed is a thoracic procedure or an abdominal procedure, whereby the surgical hub 5104 can determine whether the tissue clamped by the end effector of the surgical stapling and cutting instrument is lung tissue (in the case of a thoracic procedure) or stomach tissue (in the case of an abdominal procedure). The surgical hub 5104 can then appropriately adjust the compression speed and load threshold of the surgical stapling and cutting instrument according to the tissue type.

[0141] The type of body cavity being operated on during a insufflation procedure can affect the function of the smoke evacuation device. The situation awareness surgical hub 5104 can determine whether the surgical site is under pressure (by determining that the surgical procedure is using insufflation) and can determine the type of procedure. Generally, since a certain type of procedure can be performed within a specific body cavity, the surgical hub 5104 can appropriately control the motor speed of the smoke evacuation device according to the body cavity being operated on. Thus, the situation awareness surgical hub 5104 can provide a consistent amount of smoke evacuation for both thoracic and abdominal procedures.

[0142] The type of procedure being performed can affect the energy level optimal for the operation of an ultrasonic surgical instrument or a radio frequency (RF) electrosurgical instrument. For example, in arthroscopic procedures, the end effector of an ultrasonic surgical instrument or an RF electrosurgical instrument is immersed in fluid and may therefore require a higher energy level. The Situational Awareness Surgical Hub 5104 can determine whether the surgical procedure is an arthroscopic procedure. The Surgical Hub 5104 can then adjust the RF power level of the generator or the ultrasonic amplitude (e.g., "energy level") to compensate for the fluid-filled environment. In relation, the type of tissue being operated on can affect the energy level optimal for the operation of an ultrasonic surgical instrument or an RF electrosurgical instrument. The Situational Awareness Surgical Hub 5104 can determine which type of surgical procedure is being performed and then customize the energy level of the ultrasonic surgical instrument or the RF electrosurgical instrument, respectively, according to the tissue profile expected for the surgical procedure. Further, the Situational Awareness Surgical Hub 5104 can be configured to adjust the energy level of the ultrasonic surgical instrument or the RF electrosurgical instrument not only for each procedure but also over the course of the surgical procedure. The Situational Awareness Surgical Hub 5104 can determine which step of the surgical procedure is being performed or will be performed next and then update the control algorithm of the generator and / or the ultrasonic surgical instrument or the RF electrosurgical instrument to set the energy level to an appropriate value for the type of tissue expected according to the steps of the surgical procedure.

[0143] In an example, the surgical hub 5104 can derive data from additional data sources 5126 in order to improve the conclusions drawn from one data source 5126. The situation awareness surgical hub 5104 can enhance the data received from the modular device 5102 with context information constructed regarding the surgical procedure from other data sources 5126. For example, the situation awareness surgical hub 5104 can be configured to determine whether hemostasis has occurred (e.g., whether bleeding has stopped at the surgical site) according to video or image data received from a medical imaging device. The surgical hub 5104 can be further configured to make a determination regarding the integrity of the staple line or tissue welding by comparing physiological measurements (e.g., blood pressure sensed by a BP monitor communicatively connected to the surgical hub 5104) with visual or image data of hemostasis (e.g., from a medical imaging device communicatively coupled to the surgical hub 5104). The situation awareness system of the surgical hub 5104 can provide additional context when analyzing the visualization data in consideration of the physiological measurement data. The additional context can be useful when the visualization data may not be conclusive or may be incomplete by itself.

[0144] For example, if it is determined that the use of an instrument is required in a subsequent step of the procedure, the situation awareness surgical hub 5104 can actively activate the generator to which the RF electrosurgical instrument is connected. Actively activating the energy source can enable the instrument to be ready for use as soon as the preceding step of the procedure is completed.

[0145] The situation awareness surgical hub 5104 can determine whether the current or subsequent steps of a surgical procedure require different views or magnifications on a display according to the features of the surgical site that a surgeon is expected to view. The surgical hub 5104 can actively change the displayed view (e.g., supplied from a medical imaging device for a visualization system) as appropriate, such that the display automatically adjusts throughout the surgical procedure.

[0146] The situation awareness surgical hub 5104 can determine which step of a surgical procedure is being performed or will be performed next, and whether specific data or a comparison between data is required for that step of the surgical procedure. The surgical hub 5104 can be configured to automatically call up a data screen based on the step of the surgical procedure being performed, without waiting for a surgeon to request specific information.

[0147] During the setup of a surgical procedure or during the surgical procedure itself, errors can be checked. For example, the Situation Awareness Surgical Hub 5104 can determine whether the operating room is properly or optimally set up for the surgical procedure to be performed. The Surgical Hub 5104 can determine the type of surgical procedure being performed, read the corresponding checklist, product locations, or setup requirements (e.g., from memory), and then be configured to compare the current operating room layout to the standard layout for the type of surgical procedure that the Surgical Hub 5104 has determined is being performed. In some examples, the Surgical Hub 5104 can compare a list of items for the procedure and / or a list of devices paired with the Surgical Hub 5104 to the recommended or expected manifest of items and / or devices for a given surgical procedure. If there is any discrepancy between the lists, the Surgical Hub 5104 can provide an alert indicating that a particular modular device 5102, patient monitoring device 5124, HCP monitoring device 35510, environmental monitoring device 35512, and / or other surgical supplies are missing. In some examples, the Surgical Hub 5104 can determine, for example, the relative distance or relative position of the modular device 5102 and the patient monitoring device 5124 via proximity sensors. The Surgical Hub 5104 can compare the relative position of the devices to the layout recommended or expected for a particular surgical procedure. If there is any discrepancy between the layouts, the Surgical Hub 5104 can be configured to provide an alert indicating that the current layout of the surgical procedure deviates from the recommended layout.

[0148] The situation awareness surgical hub 5104 can determine whether a surgeon (or other HCP) is making a mistake or deviating from a series of actions expected during a surgical procedure. For example, the surgical hub 5104 can determine the type of surgical procedure being performed, read a corresponding list of steps or order of instrument use (e.g., from memory), and then compare the steps being taken or instruments being used during the surgical procedure to the steps or instruments expected for the type of surgical procedure that the surgical hub 5104 has determined is being performed. The surgical hub 5104 can provide an alert indicating that an unexpected action is being taken at a particular step in the surgical procedure or that an unexpected device is being utilized.

[0149] Surgical instruments (and other modular devices 5102) can be adjusted to suit the specific context of each surgical procedure (such as adjustment to different tissue types) and can verify actions during the surgical procedure. The next steps, data, and display adjustments can be provided to the surgical instruments (and other modular devices 5102) in the operating room according to the specific context of the procedure.

[0150] FIG. 8 shows an exemplary autonomous operation of a surgical instrument (49600). The surgical instrument may be a smart surgical stapler. Further details regarding the operation of the smart stapler are disclosed in U.S. Patent Application No. 16 / 209,423, "Method of compressing tissue within a stapling device and simultaneously displaying the location of the tissue within the jaws," filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0151] The operation of a smart surgical stapler may include clamp control and firing control. Clamp control may include control related to one or more of the following steps: closing / starting to close the clamp jaws, initial contact with tissue, clamping down (e.g., to a predetermined pressure), waiting (e.g., for a predetermined period during which tissue creep occurs), maintaining pressure (e.g., during firing), releasing pressure (e.g., after firing is complete), and starting to open / releasing the clamp jaws. Firing control may include control associated with one or more of the following steps: starting to fire (e.g., after waiting), advancing a cutting member (e.g., a knife), firing staples, or retracting the cutting member.

[0152] The operation of a smart surgical stapler may include closing control. Closing control may include control related to closing the clamp jaws (e.g., until initial contact with tissue). The operation of a smart surgical instrument may include opening control. Opening control may include control associated with opening the clamp jaws, for example, after the cutting member has been retracted to the starting position.

[0153] The operation of a smart surgical stapler can be autonomously controlled. The autonomous operation can be at different levels. One level of autonomous operation includes autonomously performing clamping down of the clamp jaws to a predetermined level (e.g., a predetermined pressure or a predetermined closing speed (e.g., 50% of full closure)), waiting (e.g., for a predetermined period), starting firing (e.g., after a predetermined waiting period has elapsed), and retracting the cutting member (e.g., after the cutting member has advanced and the staples have been fired). One level of autonomous operation can include autonomously performing pre-defined clamp control between initial contact with the tissue and when it is ready to start firing (e.g., when the waiting period is complete). One level of autonomous operation may include autonomously performing starting firing and firing the cutting member to the distal end stroke and waiting until manually indicated (e.g., by a surgeon) to retract the cutting member. One level of autonomous operation may include autonomously performing starting firing, firing the cutting member to the distal end stroke, and retracting the cutting member to the starting position. One level of autonomous operation can include autonomously performing clamping down of the clamp jaws to a predetermined level, waiting, starting firing, retracting the cutting member, and releasing the pressure on the tissue (e.g., without including releasing the tissue / releasing the clamp jaws).

[0154] The level of autonomous operation of the smart surgical stapler can be indicated by a medical professional (e.g., a surgeon). The medical professional can activate the clamp control trigger and hold (e.g., continue to press) the clamp control trigger at the end of the autonomous clamp control. In such a case, the clamp control operation may be autonomous, and the medical professional may regain control and perform manual firing control. The medical professional may momentarily activate the clamp control trigger and then release it. In such a case, the clamp control operation and the firing control operation may be autonomous. The medical professional may activate the firing control trigger and then hold the trigger. In such a case, the firing control operation may be manual. The medical professional may activate the firing control trigger and then release it (e.g., as described herein, after first activating the clamp control trigger and then holding the clamp control trigger at the end of the autonomous clamp trigger). In such a case, the firing control operation may be autonomous (e.g., regardless of whether the clamp control operation is autonomous or manual). The medical professional may activate and hold the clamp control trigger and then activate and hold the firing control trigger. In such a case, both the clamp control operation and the firing control operation may be manual. The medical professional can release the hold of the activation control trigger (e.g., the clamp control trigger or the firing control trigger). In such a case, the clamp control operation or the firing control operation can shift from the manual mode to the autonomous mode. The medical professional can restart the hold on the activation control trigger (e.g., the clamp control trigger or the firing control trigger) when the clamp control operation or the firing control operation is in its autonomous operation. In such a case, the clamp control operation or the firing control operation can shift from autonomous to manual. Further, in such a case, the medical professional may then release the control trigger. In this way, the clamp control operation or the firing control operation can shift from manual to autonomous. The medical professional can activate the release control and release the control. In such a case, the release control operation may be autonomous. The medical professional can activate the release control and hold the control.In this case, the opening control operation may be manual.

[0155] As shown in FIG. 8, the smart surgical stapler 49604 may include a processor configured with one or more control algorithms for its autonomous operation. The smart surgical stapler 49604 may include, for example, a control system as described in FIG. 5. The smart surgical stapler 49604 may be configured using a control algorithm (the “autonomous clamp control algorithm”) associated with the autonomous clamp control operation 49606. The smart surgical stapler 49604 may be configured using a control algorithm (the “autonomous firing control algorithm”) associated with the autonomous firing control operation 49608. The smart surgical stapler 49604 may be configured using control algorithms associated with the autonomous clamp control operation and the autonomous firing control operation. The autonomous clamp control algorithm 49606 may include one or more of starting the closing (49610) of the clamp jaws, the initial contact with the tissue (49612), the clamping down (49614) (e.g., to a predetermined pressure), the standby (49616) (e.g., before starting the firing (49624)), maintaining the pressure (49618) (e.g., during firing), releasing the pressure (49620) (e.g., after the firing is complete), or starting to open the clamp jaws (49622). The autonomous firing control algorithm 49608 may include one or more of starting the firing 49624, advancing the cutting member 49626 (and associated staple firing), or retracting the cutting member 49628.

[0156] The smart surgical stapler 49604 may receive a first discrete signal associated with the clamp control operation (e.g., via a control circuit). The first discrete signal may be initiated by a medical professional actuating a clamp control trigger (e.g., via a control circuit). In response to the first discrete signal, a first continuous signal may be generated (e.g., via a control circuit) to cause the continuous application of force (e.g., to the clamp jaws) based on the autonomous clamp control algorithm 49606.

[0157] Based on the self-regulating clamp control algorithm 49606, the continuous application of force can cause the clamp jaw to clamp down so as to reach a pre-defined tissue compression pressure and / or (e.g., when fully closed) reach within a pre-defined range of the tissue compression pressure (49614). The clamp jaw can be clamped down in a controlled manner (49614). In one example, a first pre-defined closing speed may be used between the initial contact with the tissue 49612 and 50% closure of the clamp jaw. A second pre-defined closing speed may be used between 50% closure of the clamp jaw and when reaching the pre-defined tissue compression pressure. In one example, a first pre-defined rate of increase in tissue compression pressure may be used between the initial contact with the tissue 49612 and 50% closure of the clamp jaw. A second pre-defined rate of increase in tissue compression pressure may be used between 50% closure of the clamp jaw and when reaching the pre-defined tissue compression pressure.

[0158] Based on the self-regulating clamp control algorithm 49606, the first continuous signal can be self-regulated. For example, the continuous signal can be adjusted based on one or more measured values. The continuous application of force can be adjusted, for example, to cause the closing speed of the clamp jaw to be adjusted during clamp down 49614 based on a predefined tissue compression pressure limit (e.g., tissue load limit). The tissue compression pressure limit (e.g., tissue load limit) can be based on safety characteristics such as the risk of tissue damage or other concerns such as excessive tissue movement. If the tissue compression pressure (e.g., sensed by the clamp jaw) is a measured value that is about to exceed (e.g., has exceeded) the predefined tissue compression pressure limit, the closing speed may be reduced to a lower speed (e.g., a predefined lower speed), or the closing may be paused (e.g., completely paused). The closing may be paused for a predefined period such as 1 second or 2 seconds. Such a reduction in the closing speed or pausing of the closing can allow the tissue to relax viscoelastically. When the tissue compression pressure measurement is below an acceptable threshold (e.g., a predefined threshold), the closing speed may be increased accordingly to return to the previous closing speed, or resumed at the previous closing speed.

[0159] The continuous application of force can also be adjusted, for example, to limit clamp down on the clamp jaw based on a tissue characteristic measurement (e.g., tissue impedance measurement) (49614), which can indicate the presence of a rigid object if the measurement is higher than an expected measurement associated with the tissue. Visual detection of the rigid object can be used to supplement the tissue characteristic measurement for detecting the presence of the rigid object(s). In response to the detection of the rigid object(s), the continuous application of force can be paused and the clamp jaw can be stopped during clamp down 49614. In such a case, the medical professional 49602 can be provided with an opportunity to address the detection (e.g., release the clamp jaw and manually remove the detected rigid object).

[0160] After reaching a pre-defined tissue compression pressure for a fully closed clamp jaw (e.g., after clamp down 49614 is completed), continuous application of force may hold the tissue in the clamp jaw for a pre-defined period (waiting time 49616 / also known as tissue creep) before firing is initiated (49624) (e.g., based on the self-regulating firing control algorithm 49608 as described herein).

[0161] Continuous application of force may maintain pressure / grip 49618 on the tissue in the clamp jaw during the firing sequence (e.g., as described herein, when firing is initiated (49624), during the period when the cutting member is advancing (49626), and during the period when the cutting member is retracting (49628), based on the self-regulating firing control algorithm 49608 for example). In one example, when the cutting member is advancing (49626) and thus pushing the tissue and increasing the load on the tissue, additional clamp force may be applied to the clamp jaw. The additional clamp force may be proportional to the increased load on the tissue. The additional clamp force can be utilized to maintain pressure / grip 49618 (e.g., to restrain the tissue) and minimize tissue movement.

[0162] Continuous application of force may maintain pressure / grip 49618 on the tissue in the clamp jaw, which may include applying additional clamp force to the tissue during the firing sequence (e.g., based on the self-regulating clamp control algorithm 49606 as described herein). For example, the firing sequence may include multiple firing stages, and the cutting member may pause (e.g., briefly) at the end of each firing stage. In such a case, additional clamp force is applied to drain fluid from the cut tissue during the pause (e.g., over a pre-defined period), and the clamp jaw resumes advancement of the clamp jaw after the pause.

[0163] Continuous application of force can cause the clamp jaws to maintain tissue pressure / grip 49618, which may include further gradually closing during the firing sequence (e.g., based on an autonomous clamp control algorithm 49606 as described herein). For example, when the cutting member advances and thus presses the tissue and an increasing firing load is required to cut the tissue, the clamp jaws may be further gradually closed. The gradual closing of the clamp jaws may be proportional to the increasing firing load. The gradual closing helps to stabilize the tissue and thus can reduce the firing load required to cut the tissue.

[0164] The smart surgical stapler 49604 may receive a second discrete signal associated with a firing control operation (e.g., via a control circuit as described herein). In one example, the second discrete signal may be initiated (e.g., via a control circuit) by a medical professional actuating a firing control trigger. In one example, the second discrete signal may be autonomously actuated by an autonomous clamp control algorithm 49606. For example, the autonomous actuation may be in response to completion of step wait 49616. In response to the second discrete signal, a second continuous signal is generated (e.g., via a control circuit) and may cause a deployment operation (e.g., advancing the cutting member (49626), retracting the cutting member to the starting position (49628)) based on an autonomous firing control algorithm 49608.

[0165] Based on the self-regulating firing control algorithm 49608, the second continuous signal can advance the cutting member in a controlled manner. For example, the advancement may accelerate to a predefined speed, maintain that speed until it is sensed that there is no longer tissue in front of the cutting line, and then decelerate to a stop. The firing control algorithm / control program is described in detail in U.S. Patent Application No. 16 / 209,416, "Method of hub communication, processing, display, and cloud analytics," filed on December 4, 2018, and U.S. Patent Application No. 16 / 209,423, "Method of compressing tissue within a stapling device and simultaneously displaying the location of the tissue within the jaws," filed on December 4, 2018, the disclosures of which are hereby incorporated by reference in their entirety.

[0166] Based on the self-regulating firing control algorithm, the advancement of the cutting member can be adjusted autonomously. For example, the advancement of the cutting member can be adjusted autonomously based on measured values. In one example, the control algorithm can pause the advancement of the cutting member if the sensed firing load exceeds a predefined threshold. In some cases, the pause may continue for a predefined amount of time before the advancement is resumed. In some cases, the pause may continue until the measured tissue load (e.g., viscoelastic load characteristics by the tissue) is below an acceptable threshold before the advancement is resumed. In some cases, a maximum number of advancement attempts may be made to resume after the pause, and if the maximum number of attempts is reached, the advancement may be stopped completely and manual intervention by a medical professional 49602 may be required.

[0167] In one example, the control algorithm may adjust the forward speed of the cutting member based on the force(s) (or load) sensed on the clamp jaw. If the sensed firing load increases at a rate that exceeds a pre-defined threshold, the forward speed may be reduced over a pre-defined amount of time. After the pre-defined amount of time has elapsed, the forward speed may be increased to the previous forward speed.

[0168] In one example, the control algorithm may completely stop the forward movement if the maximum firing load (e.g., maximum forward force) or the maximum firing load is sensed more than a pre-defined maximum number of times. In one example, the control algorithm may completely stop the retraction of the cutting member if the maximum firing load (e.g., maximum forward force) or the maximum firing load is sensed more than a pre-defined maximum number of times.

[0169] Using pre - defined settings, automated or discrete movements can be controlled. The setup configuration of the surgical instrument can have settings that command the surgical instrument to execute completely autonomously and finish, execute partially up to pre - defined steps, or remain in a discrete mode with restricted autonomous operation or no autonomous actions. A hierarchical system - based autonomous architecture can be implemented. The surgical instrument can be obtained from a factory set up for full - manual mode. The surgical instrument can enter the operating room (OR), and the surgical hub can establish a communication path to the surgical instrument. The path may be queried about speed and accuracy. If the communication path is appropriate, the surgical hub can command the surgical instrument regarding the level of autonomy that can be used in the surgical procedure. Specific breakpoints may be established in the autonomous operation, and the surgical instrument may pause and hold until a medical professional (e.g., a surgeon) or OR staff confirms the interruption and commands the surgical instrument to continue. The breakpoints may be established based on individual medical professional preferences and / or AI aggregate data may be used for common breakpoints. Previous use by a specific medical professional may be used to learn the medical professional's preferences and may enable the surgical hub to command the surgical instrument to set the autonomy level to pre - defined settings. Detected problems with the device, previous surgical steps, patient biomarkers, or medical professional biomarkers, or surgical hub communication with the device can set the level of automated actions that the surgical instrument can use.

[0170] Automated operations can be locked out based on the detection of inaccurate situations. For example, a conforming closure typically enables the automatic action of the closure tube while the firing system is operating. If the closure system is approaching its limit or detects fairly early contact with tissue, in - flight adjustments can be disabled. The same triggers that control the prevention of automation may adjust the automation of another system.

[0171] FIG. 9 shows an exemplary autonomous operation of a surgical instrument (49650). The surgical instrument may be a smart energy device. Further details regarding the operation of smart energy devices are disclosed in U.S. Patent Application No. 16 / 209,453, "METHOD FOR CONTROLLING SMART ENERGY DEVICES," filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0172] The smart energy device may be a harmonic device (e.g., an ultrasonic scalpel). The ultrasonic scalpel includes an upper blade (which may include or be a tissue pad), the upper blade being a non-active blade, while the lower active jaw vibrates to denature proteins in the tissue and form an adhesive coagulum, helping to grip the tissue while preventing further dissipation of the vibrational energy. The mechanical vibration may be generated by a piezoelectric transducer embedded within the device (e.g., within the upper blade and / or the lower active jaw) that converts the applied (e.g., generated) electrical energy into mechanical vibration and then transmits it to the active blade for cutting or coagulation. The ultrasonic scalpel operates at a frequency of 55.5 kHz and has five power levels. Increasing the power level may increase the cutting speed and decrease coagulation. Less power may decrease the cutting speed and increase coagulation.

[0173] The operation of a smart energy device (e.g., an ultrasonic energy device such as an ultrasonic scalpel) may include clamp control and energy control. Clamp control may include control related to one or more of the following steps: closing of the clamp arm / initiation of closing of the clamp arm, initial contact with tissue, clamping down (e.g., to a predetermined pressure), waiting (e.g., for a predetermined period during which tissue creep occurs), maintaining pressure (e.g., during energy generation), or initiation of opening of the clamp arm / opening of the clamp arm. Energy control may include control associated with one or more of the following steps: initiating energy generation (e.g., by an energy blade), initiating tissue separation (e.g., by an energy blade), or initiating tissue sealing (e.g., by an energy blade).

[0174] A smart energy device 49652 (e.g., an ultrasonic energy device) may include a processor configured with one or more control algorithms for its autonomous operation, as described, for example, in FIG. 5 of this specification. The smart energy device 49652 may be configured using a control algorithm associated with an autonomous clamp control operation 49654. The smart energy device 49652 may be configured with a control algorithm (e.g., an autonomous energy control algorithm 49656) associated with an autonomous energy control operation. The smart energy device 49652 may be configured with control algorithms associated with both an autonomous clamp control operation and an autonomous energy control operation. The autonomous clamp control algorithm 49654 may be associated with one or more steps such as the start / closure 49658 of the clamp arm, the initial contact 49660 with tissue, the clamp down 49662 (e.g., to a predetermined pressure), the standby 49664 (e.g., before activating energy 49624), the pressure maintenance 49666 (e.g., during energy generation), the start of release / open of the clamp arm 49668 (e.g., after the tissue seal 49674 is complete), etc. The autonomous energy control algorithm 49656 may be associated with one or more steps such as energy activation 49670, energy generation 49672 for tissue separation, energy generation 49674 for tissue sealing, etc.

[0175] The smart energy device 49652 (e.g., an ultrasonic energy device) may receive a first discrete signal related to a clamp control operation (e.g., via a control circuit). The first discrete signal may be initiated (e.g., via a control circuit) by a medical professional activating a clamp control trigger. In response to the first discrete signal, a first continuous signal may be generated (e.g., via a control circuit) to cause a continuous application of force (e.g., on the clamp arm) based on the autonomous clamp control algorithm 49654.

[0176] Based on the self-regulating clamp control algorithm 49654, the continuous application of force causes the clamp arm to initiate closure / closing 49658, initial contact with the tissue 49660, clamping down 49662 (e.g., to a predetermined pressure), waiting 49664 (e.g., before activating energy 49624), maintaining pressure 49666 (e.g., during energy generation), and may initiate the opening of the clamp arm / release the clamp arm 49668 (e.g., after tissue sealing 49674 is complete).

[0177] The smart energy device 49652 may receive a second discrete signal associated with the energy control operation (e.g., via a control circuit). The second discrete signal may be initiated (e.g., via a control circuit) by the activation of an energy control trigger by a medical professional (or, e.g., autonomous activation by the self-regulating clamp control algorithm 49654 in response to the completion 49665 of step waiting 49664). In response to the second discrete signal, a second continuous signal is generated (e.g., via a control circuit) and may cause a deployment operation (e.g., energy generation for tissue separation 49672 and energy generation for tissue sealing 49674) based on the self-regulating energy control algorithm 49656.

[0178] Based on the self-regulating energy control algorithm 49656, the second continuous signal may cause the energy blade to generate energy for separating the tissue. For example, the energy blade may generate energy at a first predefined power level (e.g., a higher level) sufficient to separate / cut the tissue.

[0179] During tissue separation, the tissue contents can be monitored / measured to determine whether to adjust the energy generation for tissue sealing. For example, the ratio of collagen to elastin in the tissue can be measured (e.g., continuously during tissue separation). If it is measured that the collagen is denatured below a predefined threshold, the energy blade may generate energy at a second predefined power level (e.g., a lower level) sufficient to seal the tissue.

[0180] During tissue occlusion, for example, to avoid damage to the upper clamp arm, tissue contents can be monitored / measured to determine whether energy generation should be stopped (e.g., after tissue occlusion is complete). For example, the amount of tissue may be measured (e.g., continuously during tissue occlusion). If it is detected that there is no tissue between the clamp arm and the energy blade, energy generation is stopped.

[0181] The clamping pressure between the active blade (e.g., energy blade) and the non-active blade (e.g., clamp arm) can affect tissue separation / tissue occlusion (e.g., tissue cutting / vessel occlusion), and / or can change the frequency and / or impedance, and / or can cause blade fatigue. The clamping pressure may be controlled (e.g., manually) by a medical professional and may (e.g., significantly) alter the intended result. As described herein, enabling a smart energy device to autonomously control the clamping pressure and the amount / level of energy applied based on the tissue / vessel or the intended action can improve the consistency of the therapeutic treatment and / or minimize trauma to unintended areas.

[0182] Based on the self-regulating clamp control algorithm 49654, continuous application of force can cause the clamp arm to autonomously control grasping and tissue manipulation. In the case of tissue manipulation, the pressure between the jaws can be controlled so as not to damage the tissue. The clamp pressure can be increased or decreased as the medical professional grasps and moves the tissue. When the medical professional moves the tissue, additional load may be applied to the tissue as it moves / stretches and / or the load on the tissue may decrease and the tissue may fall out of the jaws. In such cases, damage to the tissue may be caused and confusion and / or delay in the procedure may be caused. (For example, as described herein, via the sensors of the smart energy device and / or visual detection data of the tissue from an image system / scope) the type of tissue that the smart energy device is attempting to grasp may be identified. When the scope / smart energy device detects the movement, direction, and load applied to the tissue, the clamp pressure may be increased / decreased based on the self-regulating clamp control algorithm 49654.

[0183] In conjunction with a control algorithm for a smart energy device for monitoring impedance (e.g., self-regulating energy control algorithm 49656), the use of visual feedback through the scope may be used to alter the clamp pressure and energy level delivered to the smart energy device. The scope and smart energy device 49652 can control the therapeutic procedure (e.g., in contrast to control by a medical professional) (e.g., via the responses of both systems). In one example, a bipolar energy device may depend on clamp pressure and heat for a therapeutic procedure, and the approach described herein for the use of visual feedback and control algorithms for monitoring impedance may be utilized, for example, to maintain the clamp pressure based on tissue type and / or vessel size in order to optimize sealing.

[0184] Autonomous operation of the system based on detection of the maximum resultant force applied to the tissue may be implemented to minimize inadvertent tissue trauma during interaction. The clamping force may be limited by the tangential tensile force, and the clamps may be combined to limit tearing. The load on the tissue may be autonomously calculated based on the cumulative load of multiple devices, and this information may be used to affect the activation of additional devices. In one example, two grippers may be used to hold the tissue in place for energy cutting (e.g., tissue cutting by an energy device). The tissue load may be calculated based on the load between these grippers. The tissue load during energy activation (or energy generation) can affect the quality of the seal. The energy activation level (or energy generation level) can be adjusted based on the calculated tissue load. Minor autonomous adjustments to the gripper position may be made to change the tissue load, for example, to reduce the load for improved seal quality.

[0185] When the autonomous operation deviates from the planned surgical steps and positions, tactile feedback may be provided to a medical professional (e.g., a surgeon). For example, orthostatic defense around tumor resection in a solid organ can be performed. Creation of the liver resection surface can be performed with respect to the lobe / zone of the tumor being removed.

[0186] Figure 10 is a flowchart of an exemplary autonomous operation 49680 of a surgical instrument (e.g., a smart surgical device). At 49682, a first discrete signal associated with clamp control is received. For example, the smart surgical device can be a smart surgical cutting device or a smart surgical energy device. The first discrete signal can be associated with the start of the closure of the clamp jaw. The first discrete signal may be triggered by a medical professional.

[0187] In 49684, in response to a first discrete signal, a first continuous signal is generated to cause a continuous application of force based on a first autonomous control algorithm. For example, the continuous application of force can be autonomously adjusted based on at least a first measurement value.

[0188] For example, the smart surgical device may be a smart surgical cutting device. The continuous application of force may be applied during one or more of the following steps: an initial contact, a clamp-down, a standby, a pressure maintenance, or a pressure release step. The first measurement value may be one of the load on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement value indicating the presence of a rigid object.

[0189] For example, the smart surgical device can be a smart surgical energy device. The continuous application of force to the tissue may be applied during one or more of the following clamp controls: an initial contact, a clamp-down, a standby, or a pressure maintenance. The first measurement value may be the position of the tissue between the clamp arm and the energy blade.

[0190] In 49686, a second discrete signal associated with a deployment operation is received. For example, the smart surgical device may be a smart surgical cutting device. The deployment operation may be the advancement and the retraction of the cutting member.

[0191] For example, the smart surgical device can be a smart surgical energy device. The second discrete signal may be associated with the start of a firing sequence. The second discrete signal can be triggered by a medical professional or autonomously. The deployment operation may be the generation of energy.

[0192] In 49688, in response to the second discrete signal, a second continuous signal is generated to cause a deployment operation based on a second autonomous control algorithm. For example, the deployment operation may be autonomously adjusted based at least on a second measurement value. The second measurement value may be the ratio of collagen to elastin in the tissue.

[0193] The scope function may be autonomously controlled, for example, based on tissue parameters and / or medical expert-defined parameters. In the case of the focusing / zooming function, the focus may be autonomously adapted, for example, based on monitoring the current main action of the end effector and / or the current interaction position. In the case of repositioning to control the field of view, the repositioning may be controlled to follow the actions of the medical expert's instrument. The repositioning may be controlled to be based on the next step associated with the surgical plan or an instruction for the next step by the medical expert. The repositioning may be controlled to balance between two separate imaging sources to maximize the field of view. In the case of adjusting the imaging configuration based on situation awareness, it may be done to change from visible light to multi-spectral wavelengths, or (for example, based on the job at hand, results, constraints (JOC)) to change back. When monitoring the correlation of objects detected to identify imaging and potential interactions outside the displayed field of view, the displayed field of view may be digitally limited to a level smaller than what the CMOS array can detect. The detector mesh may be used to search for the object(s) and determine their position(s) and the likelihood of collision / interaction. When signaling an undesirable result that is not currently visible on the main screen, a pop-up (for example, in the corner of the main monitor) may be used to indicate the currently undetected leak (for example, a leak that is not currently visible due to being off-screen or not currently visible in the current visible spectrum). For example, pancreatic leaks can be distinct and very difficult to perceive, and alternative visualization technique(s) can be used to detect these and pancreatic leaks, and the medical expert can then issue a warning accordingly.

[0194] Surgical device movement control (e.g., joint movement) can be autonomously corrected based on the orientation of the surgical device on the medical expert monitor. Video analysis can be performed to determine the position of the end effector relative to the medical expert monitor. The control by the medical expert may be adjusted based on the screen orientation of the end effector. For example, when the medical expert thinks in terms of left to right, she / he can think in terms of left and right with respect to the monitor screen. In such a case, the method of movement control correction described herein can reduce any confusion that may exist when the handling of the end effector is in a difficult-to-handle position. The method of movement control correction as described herein may be with respect to the monitor being viewed by the medical expert (e.g., the left button on the monitor may move the end effector to the left). The method of movement control correction can be independent of the orientation of the medical expert interface of the surgical device. In one example, the medical expert can place the end cutter inside the patient's body and handle the device at an unknown position relative to the shaft of the device. Real-time video analysis may identify the end effector with the anvil facing up or down. The analysis may determine the position of the end effector relative to the medical expert monitor (e.g., left or right of the monitor). The surgical hub (e.g., control tower) can communicate with the end cutter (e.g., for a consistent medical expert experience) to adjust the control of the end cutter. In this way, the medical expert can operate the joint movement section with minimal confusion (e.g., left joint movement button corresponding to moving left and right joint movement button corresponding to moving right with respect to the medical expert's field of view).

[0195] The detection mechanism can be implemented to monitor locations where surgical devices can be placed. Options regarding notifying information about the surgical device may be provided. The notified information may indicate how to handle the surgical device. For example, the information may include instructions regarding how to cut the surgical device and / or how to place the surgical device. Disposing of highly concerning substances may be regulated by one or more of a regional authority, a national authority, or a local municipality. For example, the Batteries Directive of the European Union regulates the manufacture and disposal of batteries and accumulators in the European Union to protect human health and the environment from harmful substances such as mercury and cadmium. Similarly, Waste from Electrical and Electronic Equipment (WEEE) is another directive focused on waste electrical and electronic equipment or e-waste. The focus of the WEEE directive is on preventing the generation of WEEE, contributing to the efficient use of resources and the recovery of secondary raw materials through reuse, recycling, and other forms of collection, and improving the environmental performance of all persons involved in the life cycle of EEE.

[0196] Choices about how to notify medical professionals about the handling of surgical devices that may contain highly concerning substances, and how to cut and dispose of such surgical devices, can help ensure the sustainability of instructions for manufacturers (e.g., the manufacturer of the surgical device). The surgical system may be additionally adaptable based on, for example, geographical location or country. The surgical system may be updated and / or the instructions may be provided as they become available.

[0197] The surgical hub may provide instructions for disposing of batteries, electronic devices, and / or Substances of Very High Concern (SVHC) based on location-specific regulations. In one example, the surgical hub can be adapted to a given disposal system. The disposal system can be country-specific and / or region-specific where surgical devices or medical facilities (e.g., hospitals) may be located. In one example, the surgical hub can determine its geographical location. Based on the geographical location, the surgical hub can determine country-specific, provincial or state-specific, and / or region-specific regulations for compliance.

[0198] In one example, the batteries may be separated and placed into the battery waste stream. For example, the surgical hub may instruct the operating room personnel about the relevant streams, depending on, for example, the device and battery chemistry.

[0199] In one example, the surgical hub can access the Bill of Materials (BOM) of the surgical device. Based on the BOM of the surgical device, the surgical hub can perform a check (e.g., a periodic check) for any updates to the safety data sheets (e.g., Material Safety Data Sheet (MSDS) or Pathogen Chemical Safety Data Sheet (PSDS)) for any updated disposal instructions. The surgical hub may perform the check on the Internet. The surgical hub can scan the manufacturer's database to determine the latest instructions regarding the materials associated with the surgical device. The surgical hub can provide instructions to the healthcare provider (e.g., OR associate) on how to recapture components for reuse.

[0200] A surgical hub can determine a geographical location and use the determined geographical location to determine appropriate disposal instructions associated with a surgical device. The surgical hub can use the geographical location to determine region-specific and / or country-specific cleaning and sterilization protocols. In one example, the surgical hub can use the device code of a surgical device to determine its location. In another example, the surgical hub may determine the location of a surgical device using a Global Positioning System (GPS), a network, a hospital identifier, a manufacturer's cloud-based system. In another example, the surgical hub may determine the location of a surgical device using an Internet Protocol (IP) address and / or a software license. In one example, the surgical hub can use an airport code to determine the location of a device. The mechanism by which the location can be determined can be selectable by a medical professional. In one example, a surgical device (e.g., a surgical EEPROM) can be coded with region-specific and / or country-specific information. In one example, a surgical device can determine its location as part of an initialization check by checking settings (e.g., field service settings).

[0201] In one example, a disposal bin with a smart scan of a surgical device or a component used in a surgical device can be provided for proper disposal. Specific disposal bins may be provided for disposing of various classifications of disposable surgical devices and / or components within a surgical device. The surgical device and / or component can be matched with a disposal bin for disposing the surgical device or component into an appropriate disposal stream. If a mismatch is found between the surgical device or component and the disposal bin, a healthcare provider (e.g., and OR colleague) can be notified of the mismatch. In one example, the disposal bin can have a near field communication (NFC) or radio frequency identifier (RFID) reader to track the various types of surgical devices and / or components that have been dropped into those disposal bins. In one example, a surgical device having an NFC type chip or an RFID type chip can be checked, for example, when the device is dropped into a disposal bin, to ensure proper placement in the disposal stream.

[0202] Systems and / or devices for smart disposal may be provided for collaborative interaction between one or more healthcare professionals, one or more disposal bins, and / or one or more surgical hubs. The disposal bin may communicate with one or more surgical hubs, either wired or wirelessly. In one example, the disposal bin can communicate with a healthcare professional regarding the type of disposal that can occur or is expected to occur between the disposal bin and the surgical device or component being disposed of. The communication can be direct or via a surgical hub or application. In one example, a mini display on the disposal bin can be provided to communicate disposal instructions to a healthcare professional.

[0203] In one example, the information and / or signals associated with the smart disposal bin may match the information and / or signals associated with the surgical hub medical professional interface. In one example, a display (e.g., in the form of LED light or other communication means) may be provided to indicate a match or mismatch between the signal on the smart disposal bin and the signal on the hub UI. In one example, bins can be provided for disposal, reuse, and / or re-use. Each of the bins may have a color code of green, blue, and / or orange. The surgical hub display can communicate the type of disposal of each surgical device or component that a medical professional may be handling. The surgical hub display may indicate green light for the device to be placed, and the light on the green bin may blink or start to light up. In one example, the disposal bin can wait for a surgical device to be disposed of. When the surgical device is dropped into the disposal bin and the RFID scanner detects the correct device, the light may stop blinking, and the surgical hub may indicate a confirmation message that the disposal of the surgical device was successful. If a medical professional drops a surgical device into the wrong disposal bin, the disposal bin may blink red or a warning light may blink, and the surgical hub display may indicate an error indicating that the disposal of the surgical device was not successful.

[0204] In one example, the surgical hub medical professional interface may be aware of (e.g., dynamically aware of) the status of the disposal bin. The surgical hub may cross-check (e.g., dynamically cross-check) the instructions with the components within the disposal bin. In one example, the surgical hub may provide instructions (e.g., real-time instructions) for the disassembly of a surgical device. For example, the surgical hub may provide instructions that can detect that a surgical device and / or component is placed within the disposal bin. In one example, the surgical hub can track a checklist of what is to be disposed of in the disposal bin for what is placed in the disposal bin.

[0205] Surgical hubs can cause devices to be lost or missing during the disposal of surgical devices and / or components. In one example, a surgical hub may recognize surgical devices and / or components that can be used during a surgical procedure, and a smart disposal bin may cross-check that each of the surgical devices used during a surgical procedure is accounted for during disposal to identify lost or missing devices.

[0206] The disposal bin and / or one or more surgical hubs can communicate disposal information with other hospital systems. In one example, the disposal bin and / or one or more surgical hubs can communicate with a healthcare facility inventory management system. In one example, a cross-check can be performed between the expected surgical devices that can be used in a surgical procedure and the actual surgical devices that were used and disposed of. Such comparison information can communicate with a healthcare facility inventory management system.

[0207] In one example, the disposal bin and / or one or more surgical hubs can communicate disposal information with the cleaning staff of a healthcare facility. The disposal information can include information about the cleaning and / or disposal unit within the healthcare facility so that one can know what is expected when a disposed device is received. In one example, the disposal information can communicate to the cleaning staff of a healthcare facility that one or more disposal bins are full.

[0208] For example, a smart disposal bin with a device ID mechanism can collect surgical device data when a surgical device and / or component is dropped into the disposal bin. For example, the collected data or a portion of the collected data can be stored on the surgical device without communicating with a surgical hub. In one example, the smart disposal bin can scan the device when the device is dropped into the bin and extract device data. The smart disposal bin may communicate the collected data with a surgical hub or the manufacturer's cloud system. In one example, the smart disposal bin can connect to a surgical device via RFID, NFC, etc. In one example, the smart disposal bin and / or the surgical device can interact with the manufacturer's cloud system via a gateway device.

[0209] In one example, an application on a mobile device (e.g., a phone application or a tablet application) may be used with the device ID mechanism and may be used to gain access to a surgical hub or the manufacturer's cloud-based data system.

[0210] In one example, cleaning and / or sterilization staff can utilize a mobile device (e.g., a phone or a tablet) application that has the ability to scan and ID devices. The application may be integrated with the surgical hub network for full interconnectivity or may connect to the manufacturer's cloud site to gain access to device cleaning and sterilization protocols when a surgical hub is not available. The protocols can be communicated to medical professionals through the mobile device. The device ID can be used to automatically extract device data via NFC, RFID, BLE, etc. and upload it to the manufacturer's cloud system.

[0211] In one example, an application (e.g., a portable application) can be provided on a mobile device having download and upload capabilities to access cleaning and / or sterilization protocols. The mobile device can utilize one or more of the following device identification mechanisms, namely, QR code (registered trademark), BLE connection, NFC, or RFID.

[0212] In one example, the application may be directly connected to the manufacturer's cloud system. Such a configuration can be utilized in the case of a medical facility that may not have access to a surgical hub. Access to surgical device cleaning and / or sterilization protocols may be provided. Step-by-step instructions regarding cleaning and / or sterilization may be provided to medical professionals.

[0213] In one example, the application can utilize location information to provide country- or region-specific cleaning and / or sterilization protocols and methods. The application can provide location services. The location can be automatically detected by using mechanisms as described herein or can be specified during setup of an account associated with the use of the application. The application can automatically connect to the manufacturer's customer service or call center for assistance.

[0214] In one example, the application may (e.g., alternatively or additionally) be connected to a surgical hub system. The application may have special permission (e.g., restricted permission) for surgical hub connectivity. For example, cleaning and / or sterilization staff may have restricted or no access to a part of the surgical hub system. Access may be limited to cleaning and / or sterilization related information. The application may communicate with an interconnected hospital system, for example, for an OR system and / or inventory tracking.

[0215] The application can communicate with one or more sterilization groups to notify about the following tasks that may need to be performed. For example, cleaning staff within the OR may scan surgical devices while disassembling or disposing of them. In one example, when a device intended to be sterilized is scanned, the sterilization group (e.g., at the location of the same medical facility or at the location of a different medical facility) can be notified of the incoming device.

[0216] The application can be used to identify or confirm lost and / or missing surgical devices. The surgical hub can track surgical devices used during a surgical procedure. The application may scan each of the surgical devices during a cleaning procedure and may confirm that each of the surgical devices has been accounted for. The application can confirm disposal (e.g., proper disposal) when communicating with a smart disposal system. The application can notify maintenance when the device is ready for service.

[0217] Data associated with a surgical device can be autonomously uploaded to a cloud system. The data upload may be initiated during device cleaning and / or sterilization. One or more surgical devices can store surgical data associated with the surgical device throughout a surgical procedure. In one example, the surgical data can be stored on the device, for example, when connectivity to the surgical hub is not available. The surgical data may include device motor data, malfunctions, error codes, etc.

[0218] In one example, a surgical device may have limited or no connectivity to a surgical hub. In one example, a surgical device may not have a surgical hub for collecting data associated with the surgical device.

[0219] Surgical device data can be extracted from a surgical device when the surgical device is scanned in for cleaning. Data extraction may be performed using BLE, RFID, NFC, or other communication protocols.

[0220] Surgical device data processing can be performed autonomously. Such processing may not be accessible to cleaning and / or sterilization staff. Surgical device data processing can occur autonomously when the surgical device is connected.

[0221] Surgical device data may be sent directly from an application (e.g., an application on a mobile device) to a surgical hub system or to the manufacturer's cloud system. In the case of the manufacturer's cloud, a gateway device may be used between the surgical device and the manufacturer's cloud system.

[0222] Intraoperative autonomous device evaluation, adjustment, or repair can be provided. Surgical devices used during a surgical procedure over a period of time may experience a performance degradation or be damaged so that they do not function optimally but can still be used. Such surgical devices can be repaired autonomously within the surgical procedure.

[0223] In one example, a harmonic Teflon pad can be made available and replaceable during surgery. The Teflon pad replacement cartridge / tool can be provided for longer than a normal surgical procedure (e.g., a very long surgical procedure) or in situations where the Teflon pad is most susceptible to damage. The harmonic device may be autonomously inserted from the patient into this tool where the damaged pad is removed from the device and the replacement pad is positioned in place.

[0224] In one example, if the Teflon pads are damaged and cannot be replaced, mechanical clamp arm adjustment can be performed to optimize the gap setting by moving the pivot of the clamp arm up and down.

[0225] The resulting action output of a surgical device may be mapped to some degradation of the surgical device. The output can be used as an input to the system to show medical professionals the current remaining or predicted performance and its degradation relative to the original performance. Automated monitoring and comparison can be used to trigger updates to the control program and / or replacement or swap-out of parts or aspects of the device in order to return the performance of the device to its original level. In one example, the conductivity or contamination of an RF bipolar surgical device electrode can be used to trigger or indicate when the jaws of the surgical device should be cleaned. Cleaning of the jaws may be performed autonomously and / or during surgery. If cleaning of the jaws does not result in the desired improvement, or the functional degradation does not return to the desired level, selectively replaceable parts can be replaced or exchanged autonomously and / or during surgery. Such replacement can be performed, for example, when the system measures an inappropriate resistance during an intentional short startup when the device is inserted into or removed from a trocar. Data collected over a period of time from continuous and autonomous monitoring and / or checking of the surgical device and automated comparison can be utilized to provide a better understanding of the surgical device.

[0226] In laparoscopic surgery, a trocar can be used to seal the skin opening while allowing the insertion and removal of surgical instruments necessary for the surgery. FIGS. 11A and 11B show exemplary trocar arrangements during surgery. As shown in FIG. 11A, shape 49701 represents the front abdomen of a patient. Shape 49701 is divided into an upper right (UR) quadrant 49702, an upper left (UL) quadrant 49708, a lower right (LR) quadrant 49704, and a lower left (LL) quadrant 49706 centered on the umbilicus 49716. The midline consisting of the upper midline 49710 and the lower midline 49712 divides shape 49701 into equal left and right halves. The oval 49718 that overlaps the umbilicus 49716 represents the location of the incision for the laparoscopic trocar port. The oval shape 49720 within the LR region represents the location of the incision for the trocar port of the harmonic energy device. The circle 49714 on the upper midline 49710 represents the location of the incision for the trocar port of the grasper. The star 49722 represents the location of the target anatomical structure (e.g., the sigmoid colon in laparoscopic sigmoid colectomy). 49718, 49720, 49714 represent surgical selections of incision positions for trocar port incisions. The solid lines between 49720 and 49722, between 49718 and 49722, and between 49714 and 49722 represent the spatial relationship between the laparoscope, the harmonic device, and the grasper when all three of the laparoscope, the harmonic device, and the grasper are directed towards the target anatomical structure 49722 during surgery. Such a spatial relationship represents the spatial arrangement of the laparoscope and two surgical instruments and provides sufficient visibility of the surgical instruments when three of them are working on the target anatomical structure. Such an arrangement is sometimes referred to as triangulation.

[0227] Figure 11B shows a perspective view of the spatial relationship between the laparoscope, the harmonic device, and the grasper when all three of the laparoscope, the harmonic device, and the grasper are directed towards the target anatomical structure 49722 during a surgical procedure. Thus, the field of view of the harmonic device and the field of view of the grasper are maximized when the three of the laparoscope, the harmonic device, and the grasper are directed towards and operating on the target anatomical structure 49722 during a surgical procedure.

[0228] An instrument (e.g., a surgical instrument) may perform autonomous action(s) during reloading, repositioning, and / or cleaning for the completion of an action (e.g., a surgical action). For example, autonomous repositioning of an energy device may be performed during cleaning. Keeping the jaws and / or blades clean and / or debris-free throughout a surgical procedure can prevent the accumulation of tissue and / or debris, which can lead to unintended generator error(s) that may require troubleshooting (e.g., additional troubleshooting). A system (e.g., having a surgical hub within or coupled to an energy device) may autonomously monitor when the jaws need cleaning and may retract the energy device from the surgical site to be cleaned. After the jaws are cleaned, the energy device may autonomously return to the position (e.g., the exact position) where the device was before cleaning.

[0229] A system (e.g., a system within a surgical instrument or within a surgical hub coupled to a surgical instrument) may use context information (e.g., context information collected from additional surgical hub inputs) to determine the time (e.g., the most appropriate time) to remove (e.g., from the surgical site) and clean during operation. For example, a surgical instrument may be automatically removed from and cleaned at the surgical site during monotonous mesentery or omentum separation. When a medical professional (e.g., a surgeon) is incising an important structure (e.g., carefully), the risk (e.g., the risk of tissue / debris accumulation) may be alerted to the medical professional, and the medical professional may be enabled to continue using the surgical instrument.

[0230] Some remaining steps (e.g., surgical steps) and / or remaining incision distances can be used to determine when to clean the surgical instrument. The determination may be based on the performance of the surgical instrument and / or the impact (e.g., the expected impact) of interrupting the progression (e.g., the momentum of the surgical progression).

[0231] The stapling device can be automatically repositioned (e.g., after reloading the cartridge). The stapling device may require reloading during the surgical procedure (e.g., based on the length of the area to be stapled). After the stapling device is fired, it can be detected that another reload is required, and the stapling device can automatically retract from the surgical site to a position that was, for example, easily accessible for reloading. After reloading, the stapling device can return to a position (e.g., the exact position where the stapling device was before reloading). The stapling device can identify the reload (e.g., the required reload) and can confirm the reload (e.g., confirm that the required reload is correct).

[0232] The end cutter reload tray may be arranged such that the system (e.g., a system within an end cutter or a surgical hub) can autonomously remove the end cutter from the patient and can autonomously reload the device. Appropriate positioning of the reload ensures that the arm (e.g., a robotic arm holding the end cutter) can be moved without interfering with other arms (e.g., robotic arms) or obstacles. The optimal reload tray position may be determined based on, for example, the surgical type, the installed device, the preferred arm position of the medical professional (e.g., the surgeon), etc. The system (e.g., a system associated with the surgical hub) may position the reload tray and / or instruct the assistant(s) for proper positioning. The reload tray may be attached to an unused robotic arm, and the tray may be moved to a certain position (e.g., the most appropriate position) as needed. The in-room (e.g., operating room) monitoring of the surgical hub may be used to determine the exact position of the holding structure relative to the robotic arm trocar holding position. In this way, the robotic arm may be positioned at a position (e.g., the best position) for automatic reloading. The robotic arm(s) can only retract the instrument up to the point where the instrument is still within the trocar. In such a case, the medical professional (e.g., the surgeon) may be required to remove the instrument from the instrument driver (e.g., the tool driver), and may manually remove one cartridge and load another cartridge. When the newly loaded instrument (e.g., the tool) is reattached to the instrument driver (e.g., the tool driver), the robotic arm may then terminate the automation of the movement for repositioning. In such an example, the automated movement may be retraction and / or repositioning, and the loading of the new cartridge may be automated or performed manually.

[0233] Alignment to the marker(s) / instrument(s) may be for tracking / repositioning. Positioning of the instrument / tool may require tracking of an anatomical structure (e.g., an underlying anatomical structure) and / or may need to maintain alignment to an underlying anatomical structure, and thus may be difficult to perform. Maintaining alignment to the underlying anatomical structure may enable the instrument / tool to align its position within a space (e.g., a body cavity space) before retraction. By identifying the marker / anatomical structure, the instrument / tool can relocate itself to its previous position. For example, the instrument / tool can account for any patient movement / movement of the anatomical structure and relocate itself to its previous point (e.g., adjust to compensate for the movement). The instrument / tool may use another instrument / device / tool as a marker for alignment within the space (e.g., a body cavity space) during repositioning when the instrument / tool is being cleaned or reloaded.

[0234] A virtual access boundary can be implemented for automatic repositioning of large movements. The surgical instrument can be controlled to autonomously reposition obstacles during navigation to the treatment site. Surgery may require manipulation and incision of tissue, organs (if any), and / or repositioning of obstacles (if any) to gain access to the treatment site. For example, a vision system, an imaging system, and AI control may be used to identify the target area and obstacles, and for example, the gripper / trocar may be autonomously controlled to move the obstacles to obtain the most access. In such cases, the required access may be visible, or the medical professional (eg, surgeon) may determine based on the volume of space required for the instrument to pass through and / or gain access, and / or the size of the resection that the medical professional had planned to remove. In some examples, preoperative imaging (eg, imaging associated with previous surgeries such as the gold standard procedure), patient biometrics, and / or a scope may be used to determine how much / far an organ / tissue / obstacle can be moved to minimize trauma, etc. to determine optimal control of organ / tissue / obstacle repositioning.

[0235] The movable range of the surgical device can be autonomously controlled. A virtual boundary may be created for the movable range of the surgical device (eg, to constrain the range of motion / articulation). For example, a virtual boundary / structure can be created autonomously for each surgical instrument / device. Such a virtual boundary / structure can constrain the movement (if any) of the medical professional to a specific volume (eg, can identify whether the medical professional needs to reposition, can protect the patient from unintentional contact by the instrument / device that may damage the area, can function as a guide to the medical professional). Such a virtual boundary / structure may be shown on a monitor (eg, the medical professional's monitor) and / or may limit the movement of the instrument / device in areas outside the intended treatment area.

[0236] A fire-and-forget system operation (e.g., of a surgical instrument) may be implemented. The fire-and-forget system operation may be sequential. The fire-and-forget system operation may be based on inputs selected by a medical professional and / or autonomous instrument control. For example, an autonomous fire-and-forget system operation of a monopolar device may be implemented. The monopolar device may use an electrosurgical generator that can have two main functions such as cutting and coagulation settings. The cutting function can use an unmodulated continuous waveform. The unmodulated continuous waveform may result in a flow of low-energy electrons and may produce minimal smoke generation during tissue cutting. The coagulation function can use a modulated intermittent waveform. The modulated intermittent waveform may be associated with a high-energy electron flow and may produce more smoke generation, which is at a high temperature but with better hemostasis. The monopolar device may have the ability to use continuous current and / or mixed / blended current to incise tissue and achieve hemostasis. The autonomous selection of cutting, coagulation, and / or blend energy may be applied, for example, based on detection by an imaging system and / or identification of tissue type. The amount and direction of energy may be autonomously controlled, for example, based on tissue and / or surrounding structures.

[0237] For example, an autonomous fire-and-forget system operation of a bipolar and / or ultrasonic device may be implemented. The bipolar / ultrasonic device may be used for incision and / or tissue sealing and hemostasis. The automation of firing may be performed to control the speed at which the jaws are closed, to control the required clamp pressure (e.g., based on tissue / vessel type), to control the waiting time for compression, and / or to control the energy level applied. Monitoring through a scope device may be used to verify / confirm and / or to modulate or adjust.

[0238] For example, autonomous fire-and-forget system operation of a combination energy device may be implemented. The combination energy device can, for example, autonomously control the type of energy applied based on the tissue type and / or proximity to the surrounding structure in order to minimize unintended tissue damage. Visual detection through a scope device and identification of the surrounding structure may be used to control the energy with which the device is configured (e.g., capable of) to activate.

[0239] For example, autonomous fire-and-forget system operation of a stapling device. Automation of firing may be performed to control the speed to close the jaws, to control the required clamp pressure (e.g., based on the tissue / vessel type), to control the waiting time for compression, and / or to control the speed of the motor driving the firing mechanism.

[0240] Autonomous fire-and-forget tissue tension monitoring may be implemented. Sealing and / or cutting of blood vessels / tissue / organs when using an energy device and / or a stapling device can be varied, for example, based on the tension under the targeted area / zone being fired. Automatically monitoring the tension under the targeted area / zone may be used before activating the device firing. In the case of a harmonic device, bench top tests (e.g., bench top tests used to submit for regulatory authority approval) may show that vessel sealing is performed under an axial tension of 50 g on the vessel. To optimize the system (autonomous fire-and-forget system), the jaws may apply an axial load of 50 g to the tissue for sealing.

[0241] Tissue tension can be manipulated as a variable in sealing. Using slight device movement(s), tissue tension can be increased / decreased, for example, based on prioritization of sealing / cutting / speed. For example, tissue can be detected as mesentery. The energy device / instrument may prioritize a faster speed and increase tension during energy activation. When increasing the speed of cutting, the energy device / instrument may apply movement (e.g., slight movement) to lift the end effector (e.g., perpendicular to a jaw clamp). Such lifting may be performed throughout the energy activation or during a portion of the energy activation, for example, based on the speed required. Such movement can be slight (e.g., imperceptible to a medical professional).

[0242] In one example, tissue can be detected as a critical blood vessel. The energy device / instrument may prioritize sealing quality and decrease tension during energy activation. When prioritizing sealing quality, tissue tension can be reduced (e.g., minimized). Reduction of tissue tension may be performed by monitoring end effector joint loading and / or tissue characteristics through visualization and moving the end effector (e.g., slightly) from a higher load direction. Tissue tension may vary throughout the sealing cycle and slight movement(s) may be performed throughout sealing, for example, to ensure minimal tension.

[0243] One or more triggers may be used to evaluate tissue tension. In an example, a visual analysis of the tissue may be performed, and a change in tissue coloring adjacent to Joe may be a trigger. A visual analysis of the width of the tissue may be performed, and high tension may be indicated by narrow tissue, which may be a trigger. A perfusion analysis may be performed, and a decrease in perfusion may be a trigger / indicator that excessive tension is being applied. The device load may be monitored, and shaft load and / or Joe load may be detected as triggers. Tissue impedance, etc., may be monitored, and a change in tissue impedance (e.g., with respect to clamp load) combined with Joe gap and / or tissue position may indicate a change in tissue tension. The tissue position within Joe may be monitored and used to detect a trigger for evaluating tissue tension. In the case of tissue without tension, under a given clamp load, it may have a specific area absorbed within Joe. When tissue tension increases, the tissue may narrow within Joe, which may indicate tension. A secondary device, such as an ultrasonic probe or other means, may be used to monitor tissue characteristics and detect a trigger for evaluating tissue tension.

[0244] Semi-autonomous robot arm repositioning may be implemented. To minimize movement and / or interaction with objects, autonomous arm / stand repositioning can be performed. For example, manual arm positioning in admittance mode can be based on, for example, geofencing (e.g., by anatomical scan) for optimized control and reach (e.g., after a surgical instrument / device is docked to the arm / stand). Arm pre-positioning can be performed for the introduction of a new instrument (e.g., an end cutter) in a surgical procedure. To minimize the interaction between a new arm and existing arm(s) (if any) for access to the surgical site, existing in-use arm(s) and / or positions (e.g., potential positions) outside the patient's body can be evaluated (e.g., considered). A virtual instrument / tool for simulation of the position by an end effector may be used (e.g., in the evaluations described herein).

[0245] Figure 12 shows an exemplary trocar placement in laparoscopic surgery. The first trocar port 49804 (e.g., 49718 as described in FIGS. 11A - 11B) can be a port near the umbilicus (e.g., 49716 as described in FIG. 11A). A scope device (e.g., a laparoscope) 49802 can be inserted into the trocar port 49804 to generate a field of view (e.g., 49826 as explained from FIGS. 13A - 13B). The field of view can be presented to a display device (e.g., via a live stream) for (e.g., a medical professional to view). The second trocar port 49806 and the third trocar port 49808 can be two other trocar ports. The port 49806 can be a port for a grasper, and the port 49808 can be a port for an energy device or a linear stapler, and vice versa. The three trocar ports 49804, 49806, and 49808 form triangulation as described herein (e.g., in FIGS. 11A - 11B).

[0246] Figure 13A shows an exemplary surgical step autonomously controlled by a computing device. The computing device can be a robotic computing device that controls one or more surgical devices / instruments. As illustrated in FIG. 13A, the field of view 49826 can be associated with a surgical procedure such as a laparoscopic sigmoid colectomy. The laparoscopic sigmoid colectomy procedure can include the following surgical steps: initiate, access, mobilize the colon, excise the sigmoid colon, perform an anastomosis, and terminate. The surgical steps can include surgical tasks. For example, the surgical step "access" can include the following surgical tasks: incise adhesions, incise the mesentery, and identify the ureter.

[0247] As shown, the field of view 49826 shows a computing device that performs autonomous operations associated with the surgical task of incising the mesentery. The field of view 49826 shows the anatomical structures of the surgical site including the sigmoid mesocolon 49810, sigmoid colon 49812, rectal mesentery 49814, rectum 49818, and uterus 49816. The field of view 49826 shows surgical instruments for the mesentery incision surgical task, such as the grasper 49822 and energy device 49824.

[0248] The computing device can include a processor. The computing device (e.g., the processor included in the computing device) can be configured to control the surgical device to operate autonomously within a predefined boundary. Based on conditions being met, the computing device can be configured to determine a safety adjustment for the operation. The computing device can be configured to control the surgical device to operate based on the safety adjustment.

[0249] In one example, the computing device can be configured to control the grasper 49822 to operate autonomously within a predefined boundary (e.g., as shown in FIG. 13A) to perform the surgical task of incising the mesentery. The grasper (e.g., the grasper 49822) can be used to mobilize, hold, and / or place under tension tissue (e.g., the sigmoid colon 49812). The predefined boundary can be a virtual movement boundary associated with the surgical task of incising the mesentery.

[0250] The virtual movement boundary may be an area defined by a medical professional (e.g., a surgeon) or adjustable geofencing, for example, for protecting against autonomous actions outside a pre-defined area. Such a pre-defined area may use the operation parameters defined by the medical professional for autonomous operation. The surgical instrument may be enabled to operate autonomously within the operation parameters. In one example, surgical instruments / devices outside the field of view of the current scope may have autonomous operation prohibited. For example, the medical professional may draw a virtual line indicating the position of a line or path to be tracked by the surgical instrument. In this way, the medical professional can set where to cut / staple and can then monitor the surgical instrument as it completes the surgical task. Variables and feedback may be processed by a computing device (e.g., in real-time or near real-time), which may enable the surgical instrument to make adjustments to its operation based on detected characteristics and / or behavior. In an example, the thickness of the tissue may be determined during closure / while closing. The waiting time may be pre-programmed based on the tissue type and the thickness within the clamp jaw. The cutting speed(s) (e.g., the forward speed(s) of a cutting member such as a knife) may be set, for example, based on pre-configured and / or previously used parameters. In this way, the control of individual aspects of the surgical instrument operation by the medical professional can be limited, and the medical professional may be able to place additional focus on the surgical step(s) (e.g., more than each individual firing).

[0251] As shown in FIG. 13A, staple line 49820 may be a virtual movement boundary defined by a medical professional to perform the surgical steps of sigmoid resection as described herein. Staple line 49820 may be marked and superimposed on the anatomical structures within the field of view 49826 (e.g., using a 3D model / augmented reality). Staple line 49820 may mark the resection line for the surgical steps of sigmoid resection. Staple line 49820 may extend to incision line 49821 within the mesorectum 49814 for the surgical task of incising the mesentery. In the case of energy device 49824, the surgical task of incising the mesentery may be controlled by a computing device to be autonomously performed, for example, by following incision line 49821.

[0252] FIG. 13B shows an exemplary autonomous operation of a surgical instrument. As shown, the field of view 49826 may be associated with a laparoscopic sigmoid resection procedure. The field of view 49826 shows the anatomical structures of the surgical site (e.g., as shown in FIG. 13A). As shown, the operations of the sigmoid resection surgical steps may be autonomously controlled. As shown, linear stapler 49850 is controlled by a computing device to autonomously resect the sigmoid colon 49812 by following staple line 49820.

[0253] Figure 13C shows an exemplary operation of a surgical instrument. As shown, the field of view 49826 may be associated with a laparoscopic sigmoid colectomy procedure. The field of view 49826 shows the anatomical structure of the surgical site (e.g., as shown in FIGS. 13A to 13B). As shown, the autonomous movement of the sigmoid colectomy surgical steps is limited. After the sigmoid colectomy surgical steps are completed, the grasper 49822 and the linear stapler 49850 are retracted from the surgical site. In such a case, the autonomous movement of the jaws within the grasper 49822 and / or the linear stapler 49850 is locked (e.g., prohibited). The autonomous movement of the jaws within the grasper 49822 and / or the linear stapler 49850 is locked, for example, to avoid unintentional damage to the tissue during retraction.

[0254] Referring to FIG. 13A, the grasper 49822 may be autonomously controlled to mobilize, hold, and / or place under tension a tissue (e.g., the sigmoid colon 49812). The computing device may perform tissue tension measurement(s) to ensure the safety associated with the tissue. Strain measurement (e.g., measurement of the strain applied to the tissue) may be performed. In one example, an imaging system can be used and marks (e.g., dots) can be placed on the tissue to calculate relative forces on the tissue (e.g., using an anatomical structure and an augmented reality 3D model). In one example, a stretchable flex circuit may be used as a temporary implantable device, for example, to provide measurements of strain gauges and provide information regarding the state of the tissue. In one example, measurement(s) of the strain applied to the grasper 49822 may be performed. The measurement of the strain applied to the grasper 49822 may be performed using a strain gauge within the grasper 49822. Acceleration measurement (e.g., the acceleration of the movement of the grasper 49822) may be performed.

[0255] Rapid changes in measured values can be monitored. For example, a rapid change in acceleration may be detected. If the acceleration begins to decelerate when a certain force is applied, this may indicate that the tissue is resistant (e.g., more resistant than average). A sudden increase in force (e.g., strain) may be detected. A sudden decrease in force (e.g., strain) may be detected. It is possible to detect a transition from a steady state to a state where the acceleration begins to increase. In such a case, it may be an indicator that the gripper 49822 may have lost hold of the tissue and / or that the gripper 49822 may have started to tear the tissue. In response, the computing device may send a control signal to the gripper 49822 to cause a reduction in gripping force / strain.

[0256] The measurement of strain / acceleration may be based on absolute values. The strain / acceleration may also be measured higher than an acceptable threshold value (e.g., maximum strain / force / tension). The acceptable threshold value may be based on the direction of the force applied to the tissue / gripper 49822. The acceptable threshold value may be based on the force that the gripper 49822 can tolerate (e.g., safely). The acceptable threshold value may be a speed threshold at which the gripper 49822 can move and may limit the acceleration of the gripper 49822. The acceptable threshold value may be a movement limit based on the body cavity of the patient being operated on. In an example, the movement limit may be based on (e.g., dynamically) the available body cavity size. The movement limit may be more restrictive as the gripper 49822 approaches different parts within the body cavity.

[0257] Referring to FIG. 13B, the linear stapler 49850 may be autonomously controlled to cut / staple tissue (e.g., sigmoid colon 49812). In one example, the linear stapler 49850 may have its cutting cycle stopped before the linear stapler 49850 completes its cycle, and in such a case, the linear stapler 49850 may warn other devices / systems that the cutting has been prematurely stopped.

[0258] In one example, a fault condition (e.g., the internal state of the linear stapler 49850) is detected very early in the firing sequence. The inrush current for the firing subsystem of the linear stapler 49850 can be monitored during the very first instant of the firing sequence. For example, after 100 milliseconds of the firing sequence, it can be detected that the current is too low, and the computing device can stop the linear stapler 49850 before any significant amount of firing is performed.

[0259] In one example, a failure or concern can be detected in the tissue sealing quality (e.g., indicating that the tissue seal is compromised), and the cutting cycle can be stopped. For example, it can be detected that the staples are deformed, or there may be a possibility that the energy device was unable to produce a high-quality seal. In such cases, subsequent cutting actions may not need to be performed.

[0260] In one example, the instrument state may be detected, and the firing type may be changed accordingly. For example, such a state may be a low battery state or a motor overheat state. Under such conditions, the linear stapler 49850 can automatically change its firing mode (e.g., to a pulse, etc.) so that, for example, it is more energy / heat efficient and ensures that the firing cycle can be completed successfully.

[0261] In one example, the firing type may not be changed, which may be the nominal or default state of the linear stapler 49850. For example, in such a case, it is assumed that everything is operating correctly and there is no reason for the system to not operate in this way.

[0262] The linear stapler 49850 may be autonomously controlled to cut / staple tissue (e.g., the sigmoid colon 49812), for example, based on the selection of a medical professional. The selection of the medical professional may include a selection of precision or cutting cycle completion rate. In one example, the medical professional may be able to select a 60 mm load for a 50 mm cut, and such a selection can be programmed via a computing device. In such a case, the cutting cycle may stop 10 mm earlier and may retract at the end of the 50 mm cut. In one example, the medical professional may be able to select the total available length of the cartridge for the cutting cycle. In such a case, the cutting cycle may stop at the end of the default cutting cycle (e.g., a 50 mm cut with a 50 mm load).

[0263] Referring to FIG. 12, as illustrated, trocar positions 49804, 49806, 49808 may be determined. Such trocar positions may be determined manually (e.g., by a human such as a surgeon). In one example, a medical professional may be able to input the position of the trocar on a computer screen (e.g., in the OR) with the aid of, for example, laser imaging and positioning and / or an external sensor (e.g., a camera used for patient positioning). In this way, the absolute positioning of the trocar in space, such as the position of the trocar on the patient and / or the angle at which the trocar is inserted into the patient's body, can be determined.

[0264] Such trocar positions can be determined automatically (e.g., using automated trocar insertion and optimization of the final placement). The computing device can propose an optimal port placement, for example, by projecting laser points onto the patient's body to indicate the optimal (e.g., best optimal) placement location. The medical professional may accept or reject such a proposal. The computing device can be made to have full control over the pressure applied to insert the trocar. After the tip of the trocar pierces the outer tissue, the trocar may determine that it has detected the interior, and in such a case, the knife associated with the trocar may be retracted to avoid cutting any internal structures. During insertion, the trocar may detect an obstacle and may stop the insertion. Different such trocars can be distinguished (e.g., by the computing device). Surgical instruments / devices / tools can be inserted into such trocars. Such insertion may be automatic or may be performed manually by a medical professional (e.g., a surgeon). Limits can be associated with the speed and / or force that the trocar / device can sense. The computing device (e.g., using cameras and sensors) can monitor the manual process and may warn the medical professional if an error is detected.

[0265] The position and / or orientation of a surgical instrument / device / tool can be determined, for example, based on the associated trocar position. In one example, when a surgical instrument / device is inserted into a trocar, the surgical instrument / device can be automatically synchronized with the trocar. In this way, the position / orientation of the surgical instrument / device (e.g., within a body cavity) can be determined based on the position / angle of trocar insertion. The triangulation of a surgical instrument / device within a body cavity (e.g., as shown in FIGS. 11A - 11B) can be controlled based on the position / angle of trocar insertion. Based on the port arrangement, a computing device can adjust the movement of the surgical instrument / device / tool to minimize movement problems. For example, the computing device can adjust the movement of one surgical instrument / device / tool to reflect the trocar port arrangement based on, for example, a linear distance(s) and / or an angular distance(s).

[0266] The body position / orientation of a patient can be determined. In one example, a smart hospital bed can use pressure sensors to detect where the body is and can transmit that information to a computing device. In one example, smart bands around important limbs (e.g., wrists, ankles, etc.) can be used to detect the patient's movement, and the movement information can be used to construct a model of the body. Such a model can guide the positioning of surgical instrument(s) / device(s) within a body cavity. In one example, a camera can capture the patient's position and can use the captured information to construct a machine learning / AI model. Such a model can guide the positioning of surgical instrument(s) / device(s) within a body cavity. The models described herein can help determine the absolute positioning of surgical instrument(s) / device(s) within a space (e.g., a body cavity).

[0267] The computing device may determine the movement of the surgical instrument / device and associated safety limits based on the relationship between the trocar, the patient's body, and the surgical instrument / device. For example, as shown in FIGS. 12 and 13A, the position / orientation of the energy device 49824 and the grasper 49822 may be determined based on the position / angle of the trocars 49806 and 49808, respectively, after, for example, the energy device 49824 and the trocar 49806 are synchronized (e.g., after device insertion) and the grasper 49822 and the trocar 49808 are synchronized (e.g., after device insertion). The position / orientation of the patient's body 49803 may be determined (e.g., based on a smart bed (not shown) on which the patient is lying). In one example, the computing device may determine that the distance between the energy device 49824 and the grasper 49822 is less than a safety threshold, and for example, determine to retract the energy device 49824 by a predefined distance (e.g., 1 mm) to avoid a potential collision between the energy device 49824 and the grasper 49822.

[0268] Verification, authorization, and / or initiation of the intended activation of automated steps may be directed by a medical professional (e.g., a surgeon). Automation may be monitored, for example, via one or more authorizations of the steps (e.g., a series of authorizations by the surgeon). In this way, better control over the surgical procedure can be maintained, and the uncertainty and / or patient-related variability of the surgical procedure can be reduced.

[0269] A robotic system (e.g., a computing device described herein) may be trained to order through steps and may also learn from the steps. A medical expert (e.g., a surgeon) may manually move the surgical instrument / device to the required position and may confirm that the position is the correct position (e.g., a corrected position having x, y, and z coordinates). The computing device can recognize steps in addition to the steps previously performed by the medical expert, and can seek permission from the medical expert to add additional steps to the surgical map (e.g., a surgical plan including all steps for performing the procedure). Conditional robot breakpoints can be used based on sensors or environmental conditions associated with the robotic system.

[0270] A computing device (e.g., a robotic system) may automatically define breakpoints based on apparent differences in steps (e.g., performed by a medical expert), and may enable a medical expert to review the breakpoints (e.g., to provide more permissions for the breakpoints). Breakpoints may be defined based on, for example, the complexity of the operation, tools, training of the medical expert, risk, etc. In one example, a computing device may submit (e.g., place) a breakpoint for further evaluation (e.g., by a surgeon) before proceeding with a surgical procedure. Such apparent differences may be detected based on monitored parameters such as, for example, video stream data, tissue impedance data, force data, etc. In the case of a sleeve gastrectomy step, the stapling operation may encounter a staple during a cycle that causes the staple to exceed a high force threshold, which may allow for further creep and cause a pause to reduce force / trauma to the tissue. In such a case, the stapling operation may initiate a backup and may continue automatically. At the end of the stroke, a distinct breakpoint may occur (e.g., because the medical expert can clearly see that the stapling operation is at the end of its movement), and the stapling operation may wait for the medical expert to enable a reverse activation after the breakpoint.

[0271] The apparent breakpoints described herein may be a clear delineation of a continuous set of automated steps, and a medical expert (e.g., a surgeon) may verify the operation to ensure that the automation is performing the steps. In some examples, such similar operations may be part of a closed-loop control.

[0272] In some instances, repeated steps may be desirable and the medical professional may have means to instruct to repeat a previous set of automated steps. The medical professional may be provided the ability to modify and / or add additional steps if they feel a need to reproduce the step(s). If the surgical device indicates that the tissue is incorrectly positioned, the medical professional may request that the surgical device be released and repositioned. The medical professional may determine, for example, that more tissue needs to be removed to secure a good margin. The surgical device may provide the ability to adapt a series of steps, for example, based on the native tissue state. In an example, the native tissue may be automatically recognized by the surgical device. The recognition may be based on the physician's knowledge or prior knowledge (e.g., the patient is prone to bleeding, has a low blood pressure value, etc.).

[0273] During unexpected events, emergencies, the medical professional can take full control of the automated steps (regardless of whether deviating from defined automated surgical steps). In one example, the robotic arm can be safely returned to a safe position. The system (e.g., robotic system) may pause and wait for direct command(s) from the medical professional.

[0274] Verification of automated step operations or initiation by a medical professional of automated steps may be performed by a medical professional (e.g., a surgeon). Verification of out-of-order steps may be performed to avoid triggering any accidental requirements of the automated steps. For example, a medical professional may partially clamp tissue and may accidentally initiate joint movement or firing. The system may verify (e.g., using a medical professional) that the requested action is intended before the automated set of steps is initiated. In one example, a display may not be available or may not be part of the system. In such a case, the system may first provide tactile feedback to the medical professional to confirm that the medical professional intends to perform the detected function, and subsequent restarts may be permitted to initiate automation without feedback. A medical professional (e.g., a surgeon) may enter pre-defined breakpoints in the automated steps, for example, to ensure verification and completion of the automated steps.

[0275] Figure 14 is an exemplary flowchart of autonomous operations associated with a surgical device 49840. At 49842, the surgical device may be controlled to operate autonomously within pre-defined boundaries. For example, the surgical device may be a smart grasper, a smart surgical stapler, or a smart energy device. The pre-defined boundaries may be virtual movement boundaries associated with a surgical task. The pre-defined boundaries may be a field of view defined by a scope device.

[0276] At 49844, based on the conditions being met, a safety adjustment for the operation may be determined. If the surgical device is a smart grasper, the condition may be that the tissue tension measurement associated with the smart grasper is above a maximum tissue tension, and the safety adjustment may be a reduction in the grasping force.

[0277] When the surgical device is a smart surgical stapler, the condition may be that the measured inrush current value is less than the minimum threshold value, and the safety adjustment may be to stop the firing sequence.

[0278] When the surgical device is a smart energy device, the first placement data associated with the first trocar and the second placement data associated with the second trocar may be received. The first trocar may be associated with a smart grasper, and the second trocar is associated with the smart energy device. The first position data associated with the smart grasper may be determined based on the first placement data. The second position data associated with the smart energy device may be determined based on the second placement data. The third position data associated with the patient's body and the first orientation data associated with the patient's body can be received. The condition may be that the distance between the smart energy device and the smart grasper is less than a threshold value, and the safety adjustment may be an adjustment of the movement of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data.

[0279] In 49846, the surgical device can be controlled to operate based on the safety adjustment. When the surgical device is a smart grasper, the condition may be that the measured tissue tension value associated with the smart grasper is greater than or equal to the maximum tissue tension, and the safety adjustment may be a reduction in the grasping force. Controlling the surgical device to operate based on the safety adjustment may include transmitting a control signal to the surgical device to cause a reduction in the grasping force.

[0280] When the surgical device is a smart surgical stapler, the condition may be that the measured inrush current value is less than the minimum threshold value, and the safety adjustment may be to stop the firing sequence. Controlling the surgical device to operate based on the safety adjustment may include stopping the transmission of a control signal to the surgical device to stop the firing sequence.

[0281] The following is a numbered list of embodiments that may or may not be claimed. 1. A smart surgical device, wherein the smart surgical device comprises a processor, and the processor generates a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm in response to receiving a first discrete signal associated with clamp control, wherein the continuous application of force is autonomously adjusted based on at least a first measurement value, and generates a second continuous signal for causing a deployment operation based on a second autonomous control algorithm in response to receiving a second discrete signal associated with the deployment operation, wherein the deployment operation is autonomously adjusted based on at least a second measurement value,

[0282] Requiring two discrete signals to cause autonomous clamping and deployment operations provides the advantage of allowing these steps to be separated for control. Thus, a medical professional can indicate whether the clamping or deployment operation is to be performed manually or autonomously, providing greater flexibility with respect to autonomous control. For example, a medical professional can manually perform the clamping and then indicate that the deployment operation should be performed autonomously.

[0283] 2. The smart surgical device according to embodiment 1, wherein the processor is configured to generate a second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal is received.

[0284] Advantageously, the clamping and deployment operations can be independent such that the deployment operation can be performed autonomously regardless of whether the clamping operation is performed manually or autonomously.

[0285] 3. The first discrete signal is the smart surgical device according to Embodiment 1 or Embodiment 2, triggered by a medical professional.

[0286] Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by a medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or have the device perform it autonomously.

[0287] 4. The first discrete signal is the smart surgical device according to Embodiment 3, triggered by a medical professional activating and then releasing an actuation control trigger.

[0288] 5. The second discrete signal is the smart surgical device according to any one of Embodiments 1 to 4, triggered by a medical professional or autonomously.

[0289] Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by a medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or have the device perform it autonomously.

[0290] 6. The second discrete signal is the smart surgical device according to Embodiment 5, triggered by a medical professional activating and then releasing an actuation control trigger or by the completion of a first autonomous control algorithm.

[0291] Advantageously, the autonomous control of the deployment operation can be indicated in several ways. For example, a medical professional can activate and then release an actuation trigger associated with the deployment operation to provide an autonomous deployment operation after a manual clamping operation or an autonomous clamping operation.

[0292] Alternatively, when the clamping operation is performed autonomously, the trigger to initiate the autonomous control of the deployment operation may be the completion of the clamping operation. For example, a medical professional may momentarily activate the clamping control trigger and then release it. In this example, the clamping control operation and the firing control operation may be autonomous. When the clamping control operation is completed, a second signal is autonomously triggered and the firing control operation is initiated.

[0293] Furthermore, a medical professional may switch from manual control to autonomous control by releasing the hold of an activation control trigger (e.g., a firing control trigger). In such a case, a second discrete signal is triggered and the deployment operation may transition from manual mode to autonomous mode.

[0294] 7. A smart surgical device, wherein the smart surgical device a first activation trigger associated with clamping control, a second activation trigger associated with a deployment operation, comprises a processor, and the processor in response to receiving a first discrete signal associated with the first activation trigger, generates a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm, wherein the continuous application of force is autonomously adjusted based on at least a first measurement value, in response to receiving a second discrete signal associated with the second activation trigger, generates a second continuous signal for causing a deployment operation based on a second autonomous control algorithm, wherein the deployment operation is autonomously adjusted based on at least a second measurement value, in response to a first user-initiated continuous signal associated with the first activation trigger, the device enables the clamping control to be performed manually, in response to a second user-initiated continuous signal associated with the second activation trigger, the device enables the deployment operation to be performed manually. A smart surgical device.

[0295] Requiring two discrete signals to cause the self-regulating clamping operation and the self-regulating deployment operation provides the advantage of allowing these steps to be separated for control. Thus, a medical professional can, by the actuation of those first and second control triggers, indicate whether the clamping operation or the deployment operation is to be performed manually or self-regulatingly, providing greater flexibility with respect to self-regulating control. For example, a medical professional can manually perform the clamping and then indicate that the deployment operation should be performed self-regulatingly.

[0296] 8. The smart surgical device according to embodiment 7, wherein the processor is configured to generate a second continuous signal in response to the reception of the second discrete signal regardless of whether the first discrete signal has been received.

[0297] Advantageously, the clamping operation and the deployment operation can be independent such that the deployment operation can be performed self-regulatingly regardless of whether the clamping operation has been performed manually or self-regulatingly.

[0298] 9. The smart surgical device according to embodiment 7 or embodiment 8, wherein the first discrete signal is triggered by a medical professional actuating and then releasing the first actuation trigger.

[0299] Advantageously, the level of self-regulating operation of the smart surgical stapler may be indicated by a medical professional, enabling the medical professional to indicate whether the medical professional desires to perform the task manually or desires to have the device perform it self-regulatingly.

[0300] 10. The second discrete signal is triggered by a medical professional actuating and then releasing the second actuation trigger or by the completion of the first self-regulating control algorithm, for the smart surgical device according to any one of embodiments 7 to 9.

[0301] Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by a medical professional, enabling the medical professional to indicate whether they desire to perform the task manually or desire the device to perform it autonomously.

[0302] Advantageously, the autonomous control of the deployment operation can be indicated in several ways. For example, a medical professional can actuate and then release an actuation trigger associated with the deployment operation to provide an autonomous deployment operation after a manual clamping operation or an autonomous clamping operation.

[0303] Alternatively, when the clamping operation is performed autonomously, the trigger to initiate the autonomous control of the deployment operation may be the completion of the clamping operation. For example, a medical professional may momentarily actuate a clamping control trigger and then release it. In this example, the clamping control operation and the firing control operation can be autonomous. When the clamping control operation is completed, a second signal is autonomously triggered and the firing control operation is initiated.

[0304] Furthermore, a medical professional may switch from manual control to autonomous control by releasing the hold of an actuation control trigger (e.g., a firing control trigger). In such a case, a second discrete signal is triggered and the deployment operation may transition from manual mode to autonomous mode.

[0305] 11. The smart surgical device is a smart surgical cutting device or a smart surgical energy device, and the first discrete signal is associated with initiating the closure of the clamp jaws, the smart surgical device according to any one of embodiments 1 to 10.

[0306] 12. The smart surgical device is a smart surgical cutting device, and the continuous application of force is applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release, the smart surgical device according to any one of embodiments 1 to 11.

[0307] 13. The smart surgical device is a smart surgical cutting device, and the first measurement value is one of the load on the clamp jaw in the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement value indicating the presence of a rigid object, for the smart surgical device according to any one of Embodiments 1 to 12.

[0308] 14. The smart surgical device is a smart surgical cutting device, and the deployment operation is the forward movement and the backward movement of the cutting member, for the smart surgical device according to any one of Embodiments 1 to 13.

[0309] 15. The smart surgical device is a smart surgical energy device, and the second discrete signal is associated with starting the emission sequence, for the smart surgical device according to any one of Embodiments 1 to 10.

[0310] 16. The smart surgical device is a smart surgical energy device, and the deployment operation is the generation of energy, for the smart surgical device according to any one of Embodiments 1 to 10 and Embodiment 15.

[0311] 17. The smart surgical device is a smart surgical energy device, and the continuous application of force to the tissue is applied during one or more of the following clamp controls, namely, initial contact, clamping down, standby, or pressure maintenance, for the smart surgical device according to any one of Embodiments 1 to 10, Embodiment 15, and Embodiment 16.

[0312] 18. The smart surgical device is a smart surgical energy device, the first measurement value is the position of the tissue between the clamp arm and the energy blade, and the second measurement value is the ratio of collagen to elastin in the tissue, for the smart surgical device according to any one of Embodiments 1 to 10, Embodiment 15, Embodiment 16, and Embodiment 17.

[0313] 19. A computer-implemented method, the method comprising: receiving a first discrete signal associated with clamp control; generating, in response to the first discrete signal, a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm, wherein the continuous application of force is autonomously adjusted based at least on a first measurement; receiving a second discrete signal associated with a deployment operation; generating, in response to the second discrete signal, a second continuous signal for causing a deployment operation based on a second autonomous control algorithm, wherein the deployment operation is autonomously adjusted based at least on a second measurement.

[0314] A method that requires two discrete signals to cause autonomous clamping and autonomous deployment operations provides the advantage that these steps can be separated for control.

[0315] Thus, a medical professional can indicate whether the clamping or deployment operation is to be performed manually or autonomously, providing greater flexibility with respect to autonomous control.

[0316] 20. The method of embodiment 19, wherein the method can be performed on a smart surgical device for performing a surgical task or on a hub in a state where the hub provides commands to the smart surgical device.

[0317] 21. The method of embodiment 19 or embodiment 20, comprising generating a second continuous signal in response to receiving the second discrete signal, regardless of whether the first discrete signal has been received.

[0318] Advantageously, the clamping and deployment operations can be independent such that the deployment operation can be performed autonomously regardless of whether the clamping operation is performed manually or autonomously.

[0319] 22. The first discrete signal is the method according to embodiment 19, embodiment 20, or embodiment 21, which is triggered by a medical professional activating and then releasing an actuation control trigger.

[0320] Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by the medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or wish to have the device perform it autonomously.

[0321] 23. The second discrete signal is the method according to any one of embodiments 19 to 22, which is triggered by a medical professional or autonomously.

[0322] Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by the medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or wish to have the device perform it autonomously.

[0323] 24. The second discrete signal is the method according to embodiment 23, which is triggered by a medical professional activating and then releasing an actuation control trigger.

[0324] Advantageously, the level of autonomous operation of the smart surgical stapler may be indicated by the medical professional, enabling the medical professional to indicate whether they wish to perform the task manually or wish to have the device perform it autonomously.

[0325] Advantageously, the autonomous control of the deployment operation can be indicated in several ways. For example, the medical professional can activate and then release an actuation trigger associated with the deployment operation to provide an autonomous deployment operation after a manual clamping operation or an autonomous clamping operation.

[0326] Alternatively, when the clamping operation is performed autonomously, the trigger to initiate the autonomous control of the deployment operation may be the completion of the clamping operation. For example, a medical professional may instantaneously activate and then release the clamping control trigger. In this example, the clamping control operation and the firing control operation may be autonomous. When the clamping control operation is completed, a second signal is autonomously triggered and the firing control operation is initiated.

[0327] Furthermore, a medical professional may switch from manual control to autonomous control by releasing the hold of the activation control trigger (e.g., the firing control trigger). In such a case, a second discrete signal is triggered and the deployment operation may transition from manual mode to autonomous mode.

[0328] 25. The smart surgical device is a smart surgical cutting device or a smart surgical energy device, and the first discrete signal is associated with initiating the closure of the clamp jaw, according to any one of embodiments 19 to 24.

[0329] 26. The smart surgical device is a smart surgical cutting device, and the continuous application of force is applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release, according to any one of embodiments 19 to 25.

[0330] 27. The smart surgical device is a smart surgical cutting device, and the first measurement is one of the load on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement indicating the presence of a rigid object, according to any one of embodiments 19 to 26.

[0331] 28. The smart surgical device is a smart surgical cutting device, and the deployment operation is the advancement and retraction of the cutting member, according to any one of embodiments 19 to 27.

[0332] 29. The smart surgical device is a smart surgical energy device, and the second discrete signal is the method according to any one of Embodiments 19 to 25, which is associated with starting an emission sequence.

[0333] 30. The smart surgical device is a smart surgical energy device, and the deployment operation is the generation of energy, which is the method according to any one of Embodiments 19 to 25 and Embodiment 29.

[0334] 31. The smart surgical device is a smart surgical energy device, and the continuous application of force to the tissue is applied during one or more of the following clamp controls, namely, initial contact, clamp down, standby, or pressure maintenance, which is the method according to any one of Embodiments 19 to 25, Embodiment 29, and Embodiment 30.

[0335] 32. The smart surgical device is a smart surgical energy device, the first measurement value is the position of the tissue between the clamp arm and the energy blade, and the second measurement value is the ratio of collagen to elastin in the tissue, which is the method according to any one of Embodiments 19 to 25, Embodiment 29, Embodiment 30, and Embodiment 31.

[0336] 33. A computer program including instructions that, when executed by a computer, cause the computer to execute the method according to any one of Embodiments 19 to 32.

[0337] 34. A computer-readable medium including instructions that, when executed by a computer, cause the computer to execute the method according to any one of Embodiments 19 to 32.

[0338] The following is a list of numbered embodiments that may or may not be claimed. 1. A computing device, wherein the computing device comprises a processor, and the processor controls a surgical device to operate autonomously within a predefined boundary, determines a safety adjustment for the operation based on a condition being satisfied, and is configured to control the surgical device to operate based on the safety adjustment.

[0339] The computing device controls the surgical device to enhance its safety during operation, thereby providing the advantage of reducing the risk of any harmful outcomes or complications associated with the operation of the surgical device during a surgical task.

[0340] 2. The computing device according to embodiment 1, wherein the predefined boundary is a virtual movement boundary associated with a surgical task.

[0341] By controlling the surgical device within the virtual movement boundary, for example, by restricting the movement and / or articulation of the surgical device to prevent any unintended contact with tissue or other areas of the surgical device, it is possible to increase the safety of the surgical task.

[0342] 3. The computing device according to embodiment 1 or embodiment 2, wherein the condition is satisfied when a measurement associated with the surgical device or the surgical task is above / below a preset maximum / minimum threshold.

[0343] 4. The computing device according to embodiment 2 or embodiment 3, wherein the surgical device is a smart grasper, the condition is that a tissue tension measurement associated with the smart grasper is above a maximum tissue tension, the safety adjustment is a reduction in the grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to cause a reduction in the grasping force.

[0344] 5. The surgical device is a smart surgical stapler, the condition is that the measured inrush current value is less than the minimum threshold value, the safety adjustment is to stop the firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping the transmission of the control signal to the surgical device to stop the firing sequence, the computing device according to Embodiment 2 or Embodiment 3.

[0345] 6. The surgical device is a smart energy device, and the processor receives first placement data associated with a first trocar and second placement data associated with a second trocar, wherein the first trocar is associated with a smart grasper and the second trocar is associated with a smart energy device, and further configured to determine first position data associated with the smart grasper based on the first placement data and determine second position data associated with the smart energy device based on the second placement data, the computing device according to Embodiment 2 or Embodiment 3.

[0346] 7. The processor is further configured to receive third position data associated with the patient's body and first orientation data associated with the patient's body, the condition is that the distance between the smart energy device and the smart grasper is less than a threshold value, and the safety adjustment is the movement adjustment of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, the computing device according to Embodiment 6.

[0347] 8. The predefined boundary is the field of view defined by the scope device, the computing device according to any one of Embodiments 1 to 7.

[0348] Advantageously, surgical devices outside the current scope of view can be prohibited from autonomous operation, reducing the risk that unseen portions of the surgical device will contact unintended tissue or another device.

[0349] 9. The computing device is a robotic system, the computing device according to any one of embodiments 1 to 8.

[0350] 10. A computer-implemented method, controlling a surgical device to operate autonomously within a predefined boundary, determining a safety adjustment for the operation based on the condition being satisfied, and controlling the surgical device to operate based on the safety adjustment, the computer-implemented method.

[0351] The method provides the advantage of controlling the surgical device to increase its safety during operation, thereby reducing the risk of any harmful effects or complications associated with the operation of the surgical device during a surgical task.

[0352] 11. The predefined boundary is a virtual movement boundary associated with a surgical task, and the method further includes restricting the movement of the surgical device according to the virtual movement boundary, the method according to embodiment 10.

[0353] By controlling the surgical device within the virtual movement boundary, it is possible to increase the safety of the surgical task by restricting the movement and / or articulation movement of the surgical device to prevent any contact with unintended areas of tissue or other surgical devices.

[0354] 12. The condition is satisfied when a measured value associated with the surgical device or the surgical task is above / below a preset maximum / minimum threshold, the method according to embodiment 10 or embodiment 11.

[0355] 13. The surgical device is a smart grasper, the condition is that the tissue tension measurement value associated with the smart grasper is greater than or equal to the maximum tissue tension, the safety adjustment is a reduction in the grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to cause a reduction in the grasping force, the method according to embodiment 11 or embodiment 12.

[0356] 14. The surgical device is a smart surgical stapler, the condition is that the measured value of the inrush current is less than the minimum threshold value, the safety adjustment is to stop the firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping transmitting a control signal to the surgical device to stop the firing sequence, the method according to embodiment 11 or embodiment 12.

[0357] 15. The surgical device is a smart energy device, The predefined boundary is a virtual movement boundary associated with the surgical task, and the method includes receiving first placement data associated with a first trocar and second placement data associated with a second trocar, wherein the first trocar is associated with a smart grasper and the second trocar is associated with a smart energy device, determining first position data associated with the smart grasper based on the first placement data, and determining second position data associated with the smart energy device based on the second placement data, the method according to embodiment 11 or embodiment 12.

[0358] 16. The method further includes receiving third position data associated with the patient's body and first orientation data associated with the patient's body, the condition is that the distance between the smart energy device and the smart grasper is less than a threshold value, and the safety adjustment is a movement adjustment of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, the method according to embodiment 15.

[0359] 17. The method according to any one of embodiments 10 to 16, wherein the pre-defined boundary is the field of view defined by the scope device.

[0360] Advantageously, a surgical device outside the current scope of view may be prohibited from autonomous operation, reducing the risk that unseen parts of the surgical device will contact unintended tissue or another device.

[0361] 18. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of embodiments 10 to 17.

[0362] 19. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of embodiments 10 to 17.

[0363] The following is a numbered list of aspects, which may or may not be claimed. 1. A smart surgical device, wherein the smart surgical device comprises a processor, and the processor receives a first discrete signal associated with clamp control, and in response to the first discrete signal, generates a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm, wherein the continuous application of force is autonomously adjusted based at least on a first measurement value, receives a second discrete signal associated with a deployment operation, and in response to the second discrete signal, generates a second continuous signal for causing a deployment operation based on a second autonomous control algorithm, wherein the deployment operation is autonomously adjusted based at least on a second measurement value, and is configured to perform. 2. The smart surgical device is a smart surgical cutting device or a smart surgical energy device, wherein the first discrete signal is associated with initiating the closure of the clamp jaw and the first discrete signal is triggered by a medical professional, the smart surgical device according to aspect 1. 3. The smart surgical device is a smart surgical cutting device, and a continuous application of force is applied during one or more of the following steps: initial contact, clamping down, waiting, maintaining pressure, or releasing pressure, the smart surgical device according to aspect 1. 4. The smart surgical device is a smart surgical cutting device, and the first measurement is one of the load on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement indicating the presence of a rigid object, the smart surgical device according to aspect 1. 5. The smart surgical device is a smart surgical cutting device, and the deployment operation is the advancement and retraction of the cutting member, the smart surgical device according to aspect 1. 6. The smart surgical device is a smart surgical energy device, and the second discrete signal is associated with initiating the firing sequence and the second discrete signal is triggered by a medical professional or autonomously, the smart surgical device according to aspect 1. 7. The smart surgical device is a smart surgical energy device, and the deployment operation is the generation of energy, the smart surgical device according to aspect 1. 8. The smart surgical device is a smart surgical energy device, and a continuous application of force to the tissue is applied during one or more of the following clamp controls: initial contact, clamping down, waiting, or maintaining pressure, the smart surgical device according to aspect 1. 9. The smart surgical device is a smart surgical energy device, the first measurement is the position of the tissue between the clamp arm and the energy blade, and the second measurement is the ratio of collagen to elastin in the tissue, the smart surgical device according to aspect 1. 10. A method, the method comprising: Receiving a first discrete signal associated with clamp control; Generating, in response to the first discrete signal, a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm, wherein the continuous application of force is autonomously adjusted based at least on a first measurement; Receiving a second discrete signal associated with a deployment operation; Generating, in response to the second discrete signal, a second continuous signal for causing a deployment operation based on a second autonomous control algorithm, wherein the deployment operation is autonomously adjusted based at least on a second measurement. 11. The method of aspect 10, wherein the smart surgical device is a smart surgical cutting device or a smart surgical energy device, the first discrete signal is associated with initiating closure of a clamp jaw, and the first discrete signal is triggered by a medical professional. 12. The method of aspect 10, wherein the smart surgical device is a smart surgical cutting device, and the continuous application of force is applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release. 13. The method of aspect 10, wherein the smart surgical device is a smart surgical cutting device, and the first measurement is one of a load on the clamp jaw at a first contact with tissue, a load on the tissue when clamping down, and a tissue measurement indicating the presence of a rigid object. 14. The method of aspect 10, wherein the smart surgical device is a smart surgical cutting device, and the deployment operation is forward movement and backward movement of a cutting member. 15. The method of aspect 10, wherein the smart surgical device is a smart surgical energy device, the second discrete signal is associated with initiating a firing sequence, and the second discrete signal is triggered by a medical professional or autonomously. 16. The method according to aspect 10, wherein the smart surgical device is a smart surgical energy device and the deployment operation is energy generation. 17. The method according to aspect 10, wherein the smart surgical device is a smart surgical energy device and the continuous application of force to the tissue is applied during one or more of the following clamp controls: initial contact, clamp down, standby, or pressure maintenance. 18. The method according to aspect 10, wherein the smart surgical device is a smart surgical energy device, the first measurement is the position of the tissue between the clamp arm and the energy blade, and the second measurement is the ratio of collagen to elastin in the tissue.

[0364] The following is a list of numbered aspects that may or may not be claimed. 1. A computing device, wherein the computing device comprises a processor, and the processor controls the surgical device to operate autonomously within a predefined boundary, determines a safety adjustment for the operation based on the condition being met, and is configured to control the surgical device to operate based on the safety adjustment. 2. The computing device according to aspect 1, wherein the computing device is a robotic system. 3. The surgical device is a smart grasper, the predefined boundary is a virtual movement boundary associated with the surgical task, the condition is that the tissue tension measurement associated with the smart grasper is greater than or equal to the maximum tissue tension, the safety adjustment is a reduction in the grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to cause a reduction in the grasping force. Aspect. The surgical device is a smart surgical stapler, the predefined boundary is a virtual movement boundary associated with the surgical task, the condition is that the measured inrush current is less than the minimum threshold value, the safety adjustment is to stop the firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping the transmission of control signals to the surgical device to stop the firing sequence, the computing device according to Aspect 1. 5. The surgical device is a smart energy device, the predefined boundary is a virtual movement boundary associated with the surgical task, and the processor receives first placement data associated with the first trocar and second placement data associated with the second trocar, where the first trocar is associated with a smart grasper and the second trocar is associated with a smart energy device, and is further configured to determine first position data associated with the smart grasper based on the first placement data and determine second position data associated with the smart energy device based on the second placement data, the computing device according to Aspect 1. 6. The processor is further configured to receive third position data associated with the patient's body and first orientation data associated with the patient's body, the condition is that the distance between the smart energy device and the smart grasper is less than the threshold value, and the safety adjustment is the movement adjustment of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, the computing device according to Aspect 5. 7. The predefined boundary is the field of view defined by the scope device, the computing device according to Aspect 1. 8. A method comprising controlling a surgical device to operate autonomously within a predefined boundary, determining a safety adjustment for the operation based on the condition being met, A method comprising controlling a surgical device to operate based on a safety adjustment. 9. The surgical device is a smart grasper, the predefined boundary is a virtual movement boundary associated with a surgical task, the condition is that a tissue tension measurement associated with the smart grasper is greater than or equal to a maximum tissue tension, the safety adjustment is a reduction in grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to reduce the grasping force, according to the method of aspect 8. 10. The surgical device is a smart surgical stapler, the predefined boundary is a virtual movement boundary associated with a surgical task, the condition is that an inrush current measurement is less than a minimum threshold value, the safety adjustment is to stop the firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping the transmission of a control signal to the surgical device to stop the firing sequence, according to the method of aspect 8. 11. The surgical device is a smart energy device, the predefined boundary is a virtual movement boundary associated with a surgical task, and the method includes receiving first placement data associated with a first trocar and second placement data associated with a second trocar, wherein the first trocar is associated with the smart grasper and the second trocar is associated with the smart energy device, determining first position data associated with the smart grasper based on the first placement data, and determining second position data associated with the smart energy device based on the second placement data, according to the method of aspect 8. 12. The method further includes receiving third position data associated with the patient's body and first orientation data associated with the patient's body, the condition is that the distance between the smart energy device and the smart grasper is less than a threshold value, and the safety adjustment is an adjustment of the movement of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, according to the method of aspect 11. 13. The method according to aspect 8, wherein the predefined boundary is the field of view defined by the scope device.

[0365] 〔Embodiment〕 (1) A smart surgical device, wherein the smart surgical device comprises a processor, and the processor generates a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm in response to receiving a first discrete signal associated with clamp control, wherein the continuous application of force is autonomously adjusted based on at least a first measurement value, and generates a second continuous signal for causing the deployment operation based on a second autonomous control algorithm in response to receiving a second discrete signal associated with the deployment operation, wherein the deployment operation is autonomously adjusted based on at least a second measurement value, (2) The smart surgical device according to embodiment 1, wherein the processor is configured to generate a second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal is received. (3) The smart surgical device according to embodiment 1 or embodiment 2, wherein the first discrete signal is triggered by a medical professional. (4) The smart surgical device according to embodiment 3, wherein the first discrete signal is triggered by the medical professional activating and then releasing an actuation control trigger. (5) The smart surgical device according to any one of embodiments 1 to 4, wherein the second discrete signal is triggered by a medical professional or autonomously.

[0366] (6) The smart surgical device according to embodiment 5, wherein the second discrete signal is triggered by the medical professional activating and then releasing an actuation control trigger or by the completion of the first autonomous control algorithm. (7) A smart surgical device, wherein the smart surgical device a first actuation trigger associated with clamp control, a second actuation trigger associated with a deployment operation, a processor, and wherein the processor generates a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm in response to receiving a first discrete signal associated with the first actuation trigger, wherein the continuous application of force is autonomously adjusted based on at least a first measurement value, generates a second continuous signal for causing the deployment operation based on a second autonomous control algorithm in response to receiving a second discrete signal associated with the second actuation trigger, wherein the deployment operation is autonomously adjusted based on at least a second measurement value, in response to a first user-initiated continuous signal associated with the first actuation trigger, the device enables the clamp control to be manually executed, in response to a second user-initiated continuous signal associated with the second actuation trigger, the device enables the deployment operation to be manually executed, a smart surgical device. (8) The smart surgical device according to embodiment 7, wherein the processor is configured to generate a second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal is received. (9) The smart surgical device according to embodiment 7 or embodiment 8, wherein the first discrete signal is triggered by the medical professional actuating and then releasing the first actuation trigger. (10) The smart surgical device according to any one of embodiments 7 to 9, wherein the second discrete signal is triggered by the medical professional actuating and then releasing the second actuation trigger or by completion of the first autonomous control algorithm.

[0367] (11) The smart surgical device is a smart surgical cutting device or a smart surgical energy device, and the first discrete signal is associated with initiating the closure of the clamp jaw, the smart surgical device according to any one of Embodiments 1 to 10. (12) The smart surgical device is a smart surgical cutting device, and the continuous application of the force is applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release, the smart surgical device according to any one of Embodiments 1 to 11. (13) The smart surgical device is a smart surgical cutting device, and the first measurement value is one of the load on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement value indicating the presence of a rigid object, the smart surgical device according to any one of Embodiments 1 to 12. (14) The smart surgical device is a smart surgical cutting device, and the deployment operation is the advancement and the retraction of the cutting member, the smart surgical device according to any one of Embodiments 1 to 13. (15) The smart surgical device is a smart surgical energy device, and the second discrete signal is associated with initiating the firing sequence, the smart surgical device according to any one of Embodiments 1 to 10.

[0368] (16) The smart surgical device is a smart surgical energy device, and the deployment operation is the generation of energy, the smart surgical device according to any one of Embodiments 1 to 10 and Embodiment 15. (17) The smart surgical device is a smart surgical energy device, and the continuous application of the force to the tissue is applied during one or more of the following clamp controls: initial contact, clamp down, standby, or pressure maintenance, the smart surgical device according to any one of Embodiments 1 to 10, Embodiment 15, and Embodiment 16. (18) The smart surgical device is a smart surgical energy device, the first measurement value is the position of tissue between the clamp arm and the energy blade, and the second measurement value is the ratio of collagen to elastin in the tissue. The smart surgical device according to any one of Embodiments 1 to 10, Embodiment 15, Embodiment 16, and Embodiment 17. (19) A computer-implemented method, the method comprising: Receiving a first discrete signal associated with clamp control; Generating, in response to the first discrete signal, a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm, wherein the continuous application of force is autonomously adjusted based at least on a first measurement value; Receiving a second discrete signal associated with a deployment operation; Generating, in response to the second discrete signal, a second continuous signal for causing the deployment operation based on a second autonomous control algorithm, wherein the deployment operation is autonomously adjusted based at least on a second measurement value. A computer-implemented method. (20) The method may be executed on a smart surgical device for performing a surgical task or on the hub in a state where the hub provides commands to the smart surgical device. The method according to Embodiment 19.

[0369] (21) The method according to Embodiment 19 or 20, comprising generating a second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal is received. (22) The first discrete signal is triggered by the medical professional activating and then releasing an actuation control trigger. The method according to Embodiment 19, Embodiment 20, or Embodiment 21. (23) The second discrete signal is triggered by a medical professional or autonomously. The method according to any one of Embodiments 19 to 22. (24) The second discrete signal is the method according to embodiment 23, which is triggered by a medical professional activating and then releasing an activation control trigger. (25) The smart surgical device is a smart surgical cutting device or a smart surgical energy device, and the first discrete signal is the method according to any one of embodiments 19 to 24, which is associated with initiating the closing of a clamp jaw.

[0370] (26) The smart surgical device is a smart surgical cutting device, and the continuous application of the force is applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release, according to the method described in any one of embodiments 19 to 25. (27) The smart surgical device is a smart surgical cutting device, and the first measurement value is one of the load on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement value indicating the presence of a rigid object, according to the method described in any one of embodiments 19 to 26. (28) The smart surgical device is a smart surgical cutting device, and the deployment operation is the forward movement and the backward movement of the cutting member, according to the method described in any one of embodiments 19 to 27. (29) The smart surgical device is a smart surgical energy device, and the second discrete signal is the method according to any one of embodiments 19 to 25, which is associated with initiating a firing sequence. (30) The smart surgical device is a smart surgical energy device, and the deployment operation is the generation of energy, according to the method described in any one of embodiments 19 to 25 and embodiment 29.

[0371] (31) The smart surgical device is a smart surgical energy device, and the continuous application of the force to the tissue is applied during one or more of the following clamp controls, namely, initial contact, clamp down, standby, or pressure maintenance, according to the method described in any of Embodiment 19 to Embodiment 25, Embodiment 29, and Embodiment 30. (32) The smart surgical device is a smart surgical energy device, the first measurement value is the position of the tissue between the clamp arm and the energy blade, and the second measurement value is the ratio of collagen to elastin in the tissue, according to the method described in any of Embodiment 19 to Embodiment 25, Embodiment 29, Embodiment 30 and Embodiment 31. (33) A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method described in any of Embodiment 19 to Embodiment 32. (34) A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method described in any of Embodiment 19 to Embodiment 32. (35) A computing device, wherein the computing device comprises a processor, and the processor controls a surgical device to operate autonomously within a predefined boundary, determines a safety adjustment for the operation based on the condition being satisfied, and is configured to control the surgical device to operate based on the safety adjustment.

[0372] (36) The predefined boundary is a virtual movement boundary associated with a surgical task, for the computing device described in Embodiment 35. (37) The computing device according to embodiment 35 or embodiment 36, wherein the condition is satisfied when a measured value associated with the surgical device or the surgical task is above or below a preset maximum / minimum threshold value. (38) The surgical device is a smart grasper, the condition is that a tissue tension measurement value associated with the smart grasper is greater than or equal to a maximum tissue tension, the safety adjustment is a reduction in the grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to cause the reduction in the grasping force, the computing device according to embodiment 36 or embodiment 37. (39) The surgical device is a smart surgical stapler, the condition is that an inrush current measurement value is less than a minimum threshold value, the safety adjustment is to stop the firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping the transmission of a control signal to the surgical device to stop the firing sequence, the computing device according to embodiment 36 or embodiment 37. (40) The surgical device is a smart energy device, and the processor receives first placement data associated with a first trocar and second placement data associated with a second trocar, wherein the first trocar is associated with the smart grasper and the second trocar is associated with the smart energy device, and is further configured to determine first position data associated with the smart grasper based on the first placement data and determine second position data associated with the smart energy device based on the second placement data, the computing device according to embodiment 36 or embodiment 37.

[0373] (41) The processor is further configured to receive third position data associated with the patient's body and first orientation data associated with the patient's body, the condition being that the distance between the smart energy device and the smart grasper is less than a threshold value, and the safety adjustment being a movement adjustment of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, the computing device according to embodiment 40. (42) The pre-defined boundary is a field of view defined by a scope device, the computing device according to any one of embodiments 35 to 41. (43) The computing device is a robotic system, the computing device according to any one of embodiments 35 to 42. (44) A computer-implemented method comprising: controlling a surgical device to operate autonomously within a pre-defined boundary; determining a safety adjustment for the operation based on the condition being satisfied; controlling the surgical device to operate based on the safety adjustment, the computer-implemented method. (45) The pre-defined boundary is a virtual movement boundary associated with a surgical task, and the method further comprises restricting movement of the surgical device in accordance with the virtual movement boundary, the method according to embodiment 44.

[0374] (46) The condition is satisfied when a measurement associated with the surgical device or the surgical task is above / below a pre-set maximum / minimum threshold value, the method according to embodiment 44 or embodiment 45. (47) The surgical device is a smart grasper, the condition is that a tissue tension measurement value associated with the smart grasper is greater than or equal to a maximum tissue tension, the safety adjustment is a reduction in grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to cause the reduction in grasping force, the method according to embodiment 45 or embodiment 46. (48) The surgical device is a smart surgical stapler, the condition is that an inrush current measurement value is less than a minimum threshold value, the safety adjustment is to stop a firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping transmitting a control signal to the surgical device to stop the firing sequence, the method according to embodiment 45 or embodiment 46. (49) The surgical device is a smart energy device, the predefined boundary is a virtual movement boundary associated with a surgical task, and the method includes receiving first placement data associated with a first trocar and second placement data associated with a second trocar, where the first trocar is associated with the smart grasper and the second trocar is associated with the smart energy device, determining first position data associated with the smart grasper based on the first placement data, and determining second position data associated with the smart energy device based on the second placement data, the method according to embodiment 45 or embodiment 46. (50) The method further includes receiving third position data associated with the patient's body and first orientation data associated with the patient's body, the condition being that the distance between the smart energy device and the smart gripper is less than a threshold value, and the safety adjustment being a movement adjustment of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, the method according to embodiment 49.

[0375] (51) The predefined boundary is a field of view defined by a scope device, the method according to any one of embodiments 44 to 50. (52) A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of embodiments 44 to 51. (53) A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of embodiments 44 to 51.

Claims

1. A smart surgical device, wherein the smart surgical device comprises a processor, and the processor generates a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm in response to receiving a first discrete signal associated with clamp control, wherein the continuous application of force is autonomously adjusted based on at least a first measurement value; and generates a second continuous signal for causing the deployment operation based on a second autonomous control algorithm in response to receiving a second discrete signal associated with the deployment operation, wherein the deployment operation is autonomously adjusted based on at least a second measurement value, and is configured to perform the above. A smart surgical device.

2. The smart surgical device according to claim 1, wherein the processor is configured to generate a second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal is received.

3. The smart surgical device according to claim 1 or 2, wherein the first discrete signal is triggered by a medical professional.

4. The smart surgical device according to claim 3, wherein the first discrete signal is triggered by the medical professional activating and then releasing an actuation control trigger.

5. The smart surgical device according to claim 1, wherein the second discrete signal is triggered by a medical professional or autonomously.

6. The smart surgical device according to claim 5, wherein the second discrete signal is triggered by the medical professional activating and then releasing an actuation control trigger or by completion of the first autonomous control algorithm.

7. A smart surgical device, wherein the smart surgical device a first actuation trigger associated with clamp control, a second actuation trigger associated with a deployment operation, a processor, and the processor generating a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm in response to receiving a first discrete signal associated with the first actuation trigger, wherein the continuous application of force is autonomously adjusted based on at least a first measurement value; generating a second continuous signal for causing the deployment operation based on a second autonomous control algorithm in response to receiving a second discrete signal associated with the second actuation trigger, wherein the deployment operation is autonomously adjusted based on at least a second measurement value; and in response to a first user-initiated continuous signal associated with the first actuation trigger, the device enables the clamp control to be manually executed, in response to a second user-initiated continuous signal associated with the second actuation trigger, the device enables the deployment operation to be manually executed. A smart surgical device.

8. The smart surgical device according to claim 7, wherein the processor is configured to generate a second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal is received.

9. The smart surgical device according to claim 7 or claim 8, wherein the first discrete signal is triggered by a medical professional actuating and then releasing the first actuation trigger.

10. The smart surgical device according to claim 7, wherein the second discrete signal is triggered by a medical professional actuating and then releasing the second actuation trigger or by completion of the first autonomous control algorithm.

11. The smart surgical device is a smart surgical cutting device or a smart surgical energy device, and the first discrete signal is associated with initiating the closure of the clamp jaw, the smart surgical device according to claim 1.

12. The smart surgical device is a smart surgical cutting device, and the continuous application of the force is applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release, the smart surgical device according to claim 1.

13. The smart surgical device is a smart surgical cutting device, and the first measurement value is one of the load on the clamp jaw at the first contact with the tissue, the load on the tissue when clamping down, and the tissue measurement value indicating the presence of a rigid object, the smart surgical device according to claim 1.

14. The smart surgical device is a smart surgical cutting device, and the deployment operation is the advancement and the retraction of the cutting member, the smart surgical device according to claim 1.

15. The smart surgical device is a smart surgical energy device, and the second discrete signal is associated with initiating the firing sequence, the smart surgical device according to claim 1.

16. The smart surgical device is a smart surgical energy device, and the deployment operation is the generation of energy, the smart surgical device according to claim 1.

17. The smart surgical device is a smart surgical energy device, and the continuous application of the force to the tissue is applied during one or more of the following clamp controls: initial contact, clamp down, standby, or pressure maintenance, the smart surgical device according to claim 1.

18. The smart surgical device is a smart surgical energy device, the first measurement value is the position of tissue between the clamp arm and the energy blade, and the second measurement value is the ratio of collagen to elastin in the tissue. The smart surgical device according to claim 1.

19. A computer-implemented method, the method comprising: Receiving a first discrete signal associated with clamp control; In response to the first discrete signal, generating a first continuous signal for causing a continuous application of force based on a first autonomous control algorithm, wherein the continuous application of force is autonomously adjusted based at least on a first measurement value; Receiving a second discrete signal associated with a deployment operation; In response to the second discrete signal, generating a second continuous signal for causing the deployment operation based on a second autonomous control algorithm, wherein the deployment operation is autonomously adjusted based at least on a second measurement value. A computer-implemented method comprising:

20. The method may be performed on a smart surgical device for performing a surgical task or on the hub in a state where the hub provides commands to the smart surgical device. The method according to claim 19.

21. The method includes generating a second continuous signal in response to receiving the second discrete signal regardless of whether the first discrete signal is received. The method according to claim 19 or 20.

22. The first discrete signal is triggered by the medical professional activating and then releasing an actuation control trigger. The method according to claim 19.

23. The second discrete signal is triggered by a medical professional or autonomously. The method according to claim 19.

24. The method according to claim 23, wherein the second discrete signal is triggered by a medical professional actuating and then releasing an actuation control trigger. **Claim 25** The method according to claim 19, wherein the smart surgical device is a smart surgical cutting device or a smart surgical energy device, and the first discrete signal is associated with initiating closure of a clamp jaw. **Claim 26** The method according to claim 19, wherein the smart surgical device is a smart surgical cutting device, and the continuous application of the force is applied during one or more of the following steps: initial contact, clamp down, standby, pressure maintenance, or pressure release. **Claim 27** The method according to claim 19, wherein the smart surgical device is a smart surgical cutting device, and the first measurement value is one of a load on the clamp jaw at a first contact with tissue, a load on the tissue when clamping down, and a tissue measurement value indicating the presence of a rigid object. **Claim 28** The method according to claim 19, wherein the smart surgical device is a smart surgical cutting device, and the deployment operation is forward movement and backward movement of a cutting member. **Claim 29** The method according to claim 19, wherein the smart surgical device is a smart surgical energy device, and the second discrete signal is associated with initiating a firing sequence. **Claim 30** The method according to claim 19, wherein the smart surgical device is a smart surgical energy device, and the deployment operation is energy generation. **Claim 31** The method according to claim 19, wherein the smart surgical device is a smart surgical energy device, and the continuous application of the force to the tissue is applied during one or more of the following clamp controls: initial contact, clamp down, standby, or pressure maintenance.

32. The smart surgical device is a smart surgical energy device, the first measurement value is the position of tissue between the clamp arm and the energy blade, and the second measurement value is the ratio of collagen to elastin in the tissue. The method according to claim 19.

33. A computer program including instructions that, when the program is executed by a computer, cause the computer to execute the method according to claim 19.

34. A computer-readable medium including instructions that, when executed by a computer, cause the computer to execute the method according to claim 19.

35. A computing device, the computing device includes a processor, and the processor controls a surgical device to operate autonomously within a predefined boundary, determines a safety adjustment for the operation based on a condition being met, and is configured to control the surgical device to operate based on the safety adjustment.

36. The computing device according to claim 35, wherein the predefined boundary is a virtual movement boundary associated with a surgical task.

37. The computing device according to claim 35 or claim 36, wherein the condition is met when a measurement value associated with the surgical device or a surgical task exceeds or is less than a preset maximum / minimum threshold.

38. The surgical device is a smart grasper, the condition is that a tissue tension measurement value associated with the smart grasper is greater than or equal to a maximum tissue tension, the safety adjustment is a reduction in grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to cause the reduction in grasping force, the computing device according to claim 36.

39. The surgical device is a smart surgical stapler, the condition is that an inrush current measurement value is less than a minimum threshold value, the safety adjustment is to stop a firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping transmitting a control signal to the surgical device to stop the firing sequence, the computing device according to claim 36.

40. The surgical device is a smart energy device, and the processor receives first placement data associated with a first trocar and second placement data associated with a second trocar, where the first trocar is associated with the smart grasper and the second trocar is associated with the smart energy device, is further configured to determine first position data associated with the smart grasper based on the first placement data and determine second position data associated with the smart energy device based on the second placement data, the computing device according to claim 36.

41. The processor is further configured to receive third position data associated with the patient's body and first orientation data associated with the patient's body, the condition being that the distance between the smart energy device and the smart grasper is less than a threshold value, and the safety adjustment is an adjustment of the movement of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, the computing device according to claim 40.

42. The predefined boundary is a field of view defined by a scope device, the computing device according to claim 35.

43. The computing device is a robotic system, the computing device according to claim 35.

44. A computer-implemented method, comprising: controlling a surgical device to operate autonomously within a predefined boundary; determining a safety adjustment for the operation based on the condition being satisfied; controlling the surgical device to operate based on the safety adjustment, a computer-implemented method.

45. The predefined boundary is a virtual movement boundary associated with a surgical task, and the method further includes restricting movement of the surgical device according to the virtual movement boundary, the method according to claim 44.

46. The condition is satisfied when a measurement associated with the surgical device or a surgical task exceeds or is less than a preset maximum / minimum threshold value, the method according to claim 44 or claim 45.

47. The surgical device is a smart grasper, the condition is that a tissue tension measurement value associated with the smart grasper is equal to or greater than a maximum tissue tension, the safety adjustment is a reduction in grasping force, and controlling the surgical device to operate based on the safety adjustment includes transmitting a control signal to the surgical device to cause the reduction in grasping force, the method according to claim 45.

48. The surgical device is a smart surgical stapler, the condition is that an inrush current measurement value is less than a minimum threshold value, the safety adjustment is to stop a firing sequence, and controlling the surgical device to operate based on the safety adjustment includes stopping transmission of a control signal to the surgical device to stop the firing sequence, the method according to claim 45.

49. The surgical device is a smart energy device, The predefined boundary is a virtual movement boundary associated with a surgical task, and the method includes receiving first placement data associated with a first trocar and second placement data associated with a second trocar, wherein the first trocar is associated with the smart grasper and the second trocar is associated with the smart energy device, determining first position data associated with the smart grasper based on the first placement data and determining second position data associated with the smart energy device based on the second placement data, further comprising the method according to claim 45.

50. The method further includes receiving third position data associated with the patient's body and first orientation data associated with the patient's body, the condition being that the distance between the smart energy device and the smart gripper is less than a threshold value, and the safety adjustment being a movement adjustment of the smart energy device based on the first position data, the second position data, the third position data, and the first orientation data, the method of claim 49.

51. The predefined boundary is a field of view defined by a scope device, the method of claim 44.

52. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of claim 44.

53. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method of claim 44.

Images