Adapted Autonomous Functions and System Interconnection

Abstract

Systems, methods, and means for suitable autonomous functions and system interconnection are described herein. The surgical hub can detect a first surgical device within the surgical environment. The first surgical device may include a first plurality of functions. The surgical hub may include a plurality of resources. The surgical hub can detect a second surgical device within the surgical environment. The second surgical device may include a second plurality of functions. The surgical hub can determine, based on the plurality of resources, one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application is related to the following applications filed simultaneously, the contents of each of which are incorporated herein by reference. · U.S. Patent Application entitled "METHOD OF CONTROLLING AUTONOMOUS OPERATIONS IN A SURGICAL SYSTEM" filed together with this specification and having Attorney Docket No. END9430USNP1. · U.S. Patent Application entitled "AUTONOMOUS ADAPTATION OF SURGICAL DEVICE ALGORITHM" filed together with this specification and having Attorney Docket No. END9430USNP5.

Background Art

[0002] Surgical operations are typically performed in an operating room or room within a medical facility, such as a hospital. A variety of surgical devices and systems are utilized in the performance of surgical operations. In the digital information age, medical systems and facilities often implement systems or procedures using newer and improved technologies more slowly due to the general desire to maintain patient safety and traditional practices.

Summary of the Invention

Means for Solving the Problems

[0003] Systems, methods, and means for adapted autonomous functions and system interconnection are described herein. A surgical hub can detect a first surgical device within a surgical environment. The first surgical device may include a first plurality of functions. The surgical hub may include a plurality of resources. The surgical hub can detect a second surgical device within the surgical environment. The second surgical device may include a second plurality of functions. The surgical hub can determine, based on the plurality of resources, one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device.

[0004] A surgical hub for autonomous determination of surgical device functions to be enabled for a surgical procedure is described. The surgical hub includes a processor. The processor is configured to detect a first surgical device within the surgical environment. The first surgical device includes a first plurality of functions. The surgical hub includes a plurality of resources. The processor is further configured to detect a second surgical device within the surgical environment, the second surgical device including a second plurality of functions. The processor is further configured to receive or derive an indication of the surgical procedure to be performed, the surgical procedure including a plurality of surgical tasks. The processor is further configured to determine one or more functions from the first plurality of functions to enable the first surgical device to perform one of the plurality of surgical tasks and to determine one or more functions from the second plurality of functions to enable the second surgical device to perform one of the plurality of surgical tasks. The determination of which functions to enable is based on the plurality of resources. The plurality of resources of the hub are finite, and the enabled instrument functions require the use of the hub's resources to perform one or more surgical tasks. The hub advantageously autonomously adjusts the allocation of the hub's finite resources to the instrument functions required to perform one or more tasks of the surgical procedure to enable those instrument functions. The plurality of resources may comprise computer resources that may include power and bandwidth. The functions may be associated with a resource range (e.g., a power and / or bandwidth range) within which the surgical instrument can autonomously perform the surgical task. In other words, the resources may be the computer processing requirements for the functions for autonomously performing the surgical task.

[0005] The processor is further configured to send instructions for the determined one or more functions to the user of the first surgical device and the user of the second surgical device, or to send signals to the first device and / or the second device to activate the determined one or more functions. Advantageously, in addition to determining the functions to be activated, the hub can send signals to the first device and the second device to activate the determined functions, thereby preparing the devices and their functions for the surgical procedure or a task of the surgical procedure (since the activation of the functions can be dynamic based on the stage of the procedure). Alternatively, the hub can send a message to the user of the device indicating which functions should be activated.

[0006] The activated function may be a function for autonomously performing a surgical task. The surgical hub may be configured to determine which tasks should be autonomously performed by the instrument functions and which should be manually performed by the surgeon, and the resource allocation is a factor for determining which tasks should be performed autonomously or manually, and thus which functions should be activated or deactivated.

[0007] The function can be a function that the device is capable of performing.

[0008] The first device and / or the second device may be an end cutter, and the first and / or second plurality of functions may include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil jaw force, reload alignment slot management.

[0009] The processor may be further configured to receive or determine instructions for a first plurality of functions and a second plurality of functions. A user may manually input the functions of the device. Alternatively or additionally, the hub may be able to determine the functions by knowing the device. For example, the device may be connected to the hub and share an identifier that identifies the device, through which the hub may be able to determine the functions of the device (e.g., via a look-up table stored in memory or via an external memory / server).

[0010] The processor may be configured to detect the first device and the second device when the first device and the second device send requests to connect to the hub, or when the first device and the second device are connected to the hub.

[0011] The surgical hub may include a plurality of ports. The first surgical device may be connected to the surgical hub via a first port from the plurality of ports. The second surgical device may be connected to the surgical hub via a second port from the plurality of ports.

[0012] Determining which functions should be enabled may include using optimizations associated with one or more rules.

[0013] Optimizations associated with one or more rules can be based on at least one of a surgical context, a first plurality of types of functions, a second plurality of types of functions, past data, or a plurality of resources. Some functions may be given a higher priority, for example, based on the complexity of the tasks performed by those functions or based on the surgical outcomes of surgical tasks when performed manually versus automatically (which can be based on past data from past surgeries including that task performed by that function). For example, past data may indicate that surgical outcomes of surgical tasks are improved when the tasks are performed autonomously by a function, and thus the optimization may prioritize enabling that function to perform the surgical task autonomously. By optimizing in this way, the results of the surgical procedure can generally be improved. As an example, the results of tissue resection can correspond to the integrity of the seal line, and a seal with less leakage is equivalent to a more positive result than a seal with more leakage. The results of the procedure can also correspond to the number of complications that occur during the surgery, which can include, for example, device malfunction, leakage of the seal line, misfiring of the staple line, etc.

[0014] The processor can be further configured to detect a third surgical device within the surgical environment. The third surgical device may comprise a third plurality of functions. The processor can be further configured to adjust one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device based on the plurality of resources and the third plurality of functions. Advantageously, the surgical hub can dynamically determine which functions of the devices should be enabled or disabled as the surgical procedure progresses and / or as devices are brought into the surgical environment and / or connected to the hub.

[0015] The adjustment may include disabling a previously enabled function. By dynamically enabling and disabling functions, it is possible to reallocate resources when a new device is detected or as the surgical procedure progresses.

[0016] The processor may be further configured to detect a third surgical device within the surgical environment. The third surgical device may comprise a third plurality of functions. The processor may be further configured to determine, based on a plurality of resources, one or more functions from the third plurality of functions for execution by the third surgical device.

[0017] The processor may be further configured to determine, based on a plurality of resources, one or more optional functions from the first plurality of functions for execution by the first surgical device and one or more optional functions from the second plurality of functions for execution by the second surgical device.

[0018] The optional functions may be selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0019] A method for autonomous determination of surgical device functions to be enabled for a surgical procedure is described. The method includes detecting a first surgical device within the surgical environment. The first surgical device comprises a first plurality of functions. The surgical hub comprises a plurality of resources. The method further includes detecting a second surgical device within the surgical environment. The second surgical device comprises a second plurality of functions. The method further includes receiving or deriving an indication of the surgical procedure to be performed, the surgical procedure including a plurality of surgical tasks. The method includes determining, based on the plurality of resources, one or more functions from the first plurality of functions to enable the first surgical device to perform one of the plurality of surgical tasks and one or more functions from the second plurality of functions to enable the second surgical device to perform one of the plurality of surgical tasks, wherein the determination is based on the plurality of resources. The plurality of resources of the hub may be finite, and the enabled instrument functions require the use of the hub's resources to perform one or more surgical tasks. The method advantageously autonomously adjusts the allocation of the finite hub resources to the instrument functions required to perform one or more tasks of the surgical procedure to enable those instrument functions. The plurality of resources may comprise computer resources including power and bandwidth. The functions may be associated with a resource range (e.g., a power and / or bandwidth range) within which the surgical instrument can autonomously perform the surgical task. In other words, the resources may be the computer processing requirements for the functions for autonomously performing the surgical task.

[0020] The method may further include transmitting an indication of one or more determined functions to a user of a first surgical device and a user of a second surgical device, or transmitting a signal to the first device and / or the second device to activate one or more determined functions. Advantageously, in addition to determining the functions to be activated, the method includes transmitting a signal to the first device and the second device to activate the determined functions, thereby preparing the devices and their functions for a surgical procedure or a task of that surgical procedure (since the activation of the functions may be dynamic based on the stage of the procedure). Alternatively, the method includes transmitting a message to the user of the device indicating which functions are to be activated.

[0021] The activated function may be a function for autonomously performing a surgical task. The method is configured to determine which tasks are to be autonomously performed by the instrument functions and which are to be manually performed by the surgeon, and the allocation of resources is a factor for determining which tasks are to be performed autonomously or manually, and thus which functions are to be activated or deactivated.

[0022] The function may be a function that the device is capable of performing.

[0023] The first device and / or the second device may be an end cutter, and the plurality of first and / or second functions may include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil jaw force, reload alignment slot management.

[0024] The method may further include receiving or determining an indication of a first plurality of functions and a second plurality of functions.

[0025] Detecting the first device and the second device can include detecting when the first device and the second device send a request to connect to the hub, or detecting when the first device and the second device are coupled to the hub.

[0026] The surgical hub may comprise a plurality of ports. The first surgical device may be connected to the surgical hub via a first port from the plurality of ports. The second surgical device may be connected to the surgical hub via a second port from the plurality of ports.

[0027] Determining which functions should be enabled can include using optimizations associated with one or more rules.

[0028] Optimizations associated with one or more rules can be based on at least one of a surgical context, a first plurality of function types, a second plurality of function types, past data, or a plurality of resources. Some functions may be given a higher priority, for example, based on the complexity of the tasks being performed by those functions, or based on the surgical outcome of the surgical task when performed manually versus automatically (which can be based on past data from past surgeries including that task being performed by that function). For example, the past data may indicate that the surgical outcome of the surgical task is improved when the task is autonomously performed by the function, and thus the optimization may prioritize enabling that function to perform the surgical task autonomously. By optimizing in this way, the results of the surgical procedure can generally be improved. As an example, the result of tissue resection can correspond to the integrity of the seal line, and a seal with less leakage is equivalent to a more positive result than a seal with more leakage. The result of the procedure can also correspond to, for example, the number of complications that occur during the surgery, which can include instrument malfunction, leakage of the seal line, misfiring of the staple line, etc.

[0029] The method may further include detecting a third surgical device in the surgical environment. The third surgical device may comprise a third plurality of functions. The method may further include coordinating one or more functions from the first plurality of functions for the first surgical device to perform and one or more functions from the second plurality of functions for the second surgical device to perform based on the plurality of resources and the third plurality of functions. Advantageously, the method includes dynamically determining which functions of the device should be enabled or disabled as the surgical procedure progresses and / or as the device is brought into the surgical environment and / or connected to the hub.

[0030] Adjustments may include disabling previously enabled features. By dynamically enabling and disabling features, resources can be reallocated as new devices are discovered or as a surgical procedure progresses.

[0031] The method may further include detecting a third surgical device within the surgical environment. The third surgical device may comprise a third plurality of functions. The method may further include determining, based on the plurality of resources, one or more functions from the third plurality of functions for the third surgical device to perform.

[0032] The method may further include determining, based on the plurality of resources, one or more optional functions from the first plurality of functions for the first surgical device to perform and one or more optional functions from the second plurality of functions for the second surgical device to perform.

[0033] The optional features may be selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0034] A computer program comprising instructions which, when executed by a computer, cause the computer to perform any of the above-mentioned methods is described.

[0035] A computer-readable medium including instructions that, when executed by a computer, cause the computer to perform any of the aforementioned methods is described.

Brief Description of the Drawings

[0036]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Modes for Carrying Out the Invention

[0037] FIG. 1 is a block diagram of a computer-implemented surgical system 20000. An exemplary surgical system such as surgical system 20000 can include one or more surgical systems (e.g., surgical subsystems) 20002, 20003, and 20004. For example, each surgical system 20002 can include a computer-implemented bidirectional surgical system. For example, surgical system 20002 may include a surgical hub 20006 and / or a computing device 20016 that communicate with a cloud computing system 20008, as described, for example, in FIG. 2. The cloud computing system 20008 can include at least one remote cloud server 20009 and at least one remote cloud storage unit 20010. Exemplary surgical systems 20002, 20003, or 20004 may include a wearable sensing system 20011, an environmental sensing system 20015, a robotic system 20013, one or more intelligent instruments 20014, a human interface system 20012, etc. The human interface system is also referred to herein as a human interface device. The wearable sensing system 20011 may include one or more HCP sensing systems and / or one or more patient sensing systems. The environmental sensing system 20015 may include, for example, one or more devices used to measure one or more environmental attributes, as further described, for example, in FIG. 2. The robotic system 20013 may include, for example, a plurality of devices used to perform a surgical procedure, as further described, for example, in FIG. 2.

[0038] The surgical system 20002 may communicate with a remote server 20009 that may be part of a cloud computing system 20008. In one example, the surgical system 20002 may communicate with the remote server 20009 via a cable / FIOS networking node of an Internet service provider. In one example, the patient sensing system may communicate directly with the remote server 20009. The surgical system 20002 and / or its components may use one or more of the cellular protocols of GSM / GPRS / EDGE (2G), UMTS / HSPA (3G), long term evolution (LTE) or 4G, LTE-Advanced (LTE-A), new radio (NR) or 5G to communicate with the remote server 20009 via a cellular transmission / reception point (TRP) or base station.

[0039] The surgical hub 20006 may have a collaborative interaction with one of more means for displaying images from the laparoscope, as well as information from one or more other smart devices and one or more sensing systems 20011. The surgical hub 20006 may interact with one or more sensing systems 20011, one or more smart devices, and multiple displays. The surgical hub 20006 may be configured to collect measurement data from one or more sensing systems 20011 and transmit notification or control messages to one or more sensing systems 20011. The surgical hub 20006 may transmit and / or receive information, including notification information, to and from a human interface system 20012. The human interface system 20012 may include one or more human interface devices (HIDs). The surgical hub 20006 may transmit and / or receive acoustic devices, notification or control information to the display, and / or control information to various devices communicating with the surgical hub.

[0040] For example, as discussed in FIG. 1, the sensing system 20001 may include a wearable sensing system 20011 (which may include one or more HCP sensing systems and one or more patient sensing systems) and an environmental sensing system 20015. One or more sensing systems 20001 may measure data regarding various biomarkers. One or more sensing systems 20001 may use one or more sensors, such as optical sensors (e.g., photodiodes, photoreceptors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, thermoelectric sensors, infrared sensors, etc., to measure the biomarkers. One or more sensors may use one of more of the sensing techniques such as photoplethysmography, electrocardiogram examination, electroencephalogram examination, colorimetric analysis, impedancemetry, potential difference measurement, current measurement, etc., to measure the biomarkers as described herein.

[0041] The biomarkers measured by one or more sensing systems 20001 may include, but are not limited to, sleep, core body temperature, maximum oxygen consumption, physical activity, alcohol intake, respiratory rate, oxygen saturation, blood pressure, blood glucose, heart rate variability, blood potential of hydrogen, hydration status, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, cough and sneeze, gastrointestinal motility, gastrointestinal imaging, airway bacteria, edema, mental state, sweat, circulating tumor cells, autonomic nervous tension, circadian rhythm, and / or menstrual cycle.

[0042] Biomarkers can be related to physiological systems, including but not limited to the behavioral and psychological, cardiovascular, renal, dermal, nervous, gastrointestinal, respiratory, endocrine, immune, tumor, musculoskeletal, and / or reproductive systems. Information from biomarkers may be determined and / or used, for example, by the computer-implemented patient and surgical system 20000. Information from biomarkers may be determined and / or used by the computer-implemented patient and surgical system 20000 to, for example, improve the above systems and / or improve patient outcomes. One or more sensing systems 20001, biomarkers 20005, and physiological systems are described in detail in U.S. Patent Application No. 17 / 156,287, filed on January 22, 2021, entitled "METHOD OF ADJUSTING A SURGICAL PARAMETER BASED ON BIOMARKER MEASUREMENTS" (Attorney Docket No. END9290USNP1), the disclosure of which is hereby incorporated by reference in its entirety.

[0043] FIG. 2 shows an example of a surgical system 20002 in an operating room. As illustrated in FIG. 2, the patient is operated on by one or more healthcare professionals (HCPs). The HCPs are monitored by one or more HCP sensing systems 20020 worn by the HCPs. The HCPs and the environment surrounding the HCPs may also be monitored by one or more environmental sensing systems, including, for example, a set of cameras 20021, a set of microphones 20022, and other sensors deployed in the operating room. The HCP sensing system 20020 and the environmental sensing system communicate with the surgical hub 20006 and may further communicate with one or more cloud servers 20009 of the cloud computing system 20008, as shown in FIG. 1. The environmental sensing system may be used to measure one or more environmental attributes, such as the position of the HCPs in the operating room, the movement of the HCPs, the ambient noise in the operating room, the temperature / humidity in the operating room, and the like.

[0044] As illustrated in FIG. 2, the main display 20023 and one or more audio output devices (e.g., speaker 20019) are placed in the sterile field so as to be visible to the operator on the operating table 20024. Additionally, the visualization / notification tower 20026 is placed outside the sterile field. The visualization / notification tower 20026 may include a first non-sterile human interface device (HID) 20027 and a second non-sterile HID 20029 facing opposite each other. The HID may be a display, or may be a display having a touch screen that enables a human to directly interface with the HID. The human interface system guided by the surgical hub 20006 may be configured to utilize the HIDs 20027, 20029, and 20023 to regulate the information flow to the operators inside and outside the sterile field. In one example, the surgical hub 20006 may cause the HID (e.g., the main HID 20023) to display notifications and / or information regarding the patient and / or surgical steps. In one example, the surgical hub 20006 may prompt and / or receive input from a person within the sterile field or non-sterile area. In one example, the surgical hub 20006 may cause the HID to display a snapshot of the surgical site recorded by the imaging device 20030 on the non-sterile HID 20027 or 20029 while maintaining a live video of the surgical site on the main HID 20023. The snapshot on the non-sterile display 20027 or 20029 may, for example, permit a non-sterile operator to perform diagnostic steps related to the surgery.

[0045] In one aspect, the surgical hub 20006 may be configured to send diagnostic input or feedback entered by a non-sterile operator at the visualization tower 20026 to the main display 20023 within the sterile field, such that it can be viewed by the sterile operator on the operating table. In one example, the input may be in the form of a modification to a snapshot displayed on the non-sterile display 20027 or 20029, which may be sent by the surgical hub 20006 to the main display 20023.

[0046] Referring to FIG. 2, the surgical instrument 20031 is used as part of a surgical system 20002 in a surgical operation. The hub 20006 can also be configured to regulate the information flow to the display of the surgical instrument 20031. For example, in U.S. Patent Application Publication No. 2019 / 0200844 (A1) (U.S. Patent Application No. 16 / 209,385) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY", filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety. Diagnostic inputs or feedback entered by a non-sterile operator at the visualization tower 20026 are sent by the hub 20006 to the surgical instrument display within the sterile field, where the operator of the surgical instrument 20031 can view it. Exemplary surgical instruments suitable for use with the surgical system 20002 are described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019 / 0200844 (A1) (U.S. Patent Application No. 16 / 209,385), filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0047] FIG. 2 illustrates an example of a surgical system 20002 used to perform surgery on a patient lying on an operating table 20024 within an operating room 20035. A robotic system 20034 can be used as part of the surgical system 20002 in a surgical operation. The robotic system 20034 can include a surgeon's console 20036, a patient-side cart 20032 (surgical robot), and a surgical robot hub 20033. While the surgeon views the surgical site through the surgeon's console 20036, the patient-side cart 20032 can operate at least one removably coupled surgical tool 20037 through a minimally invasive incision in the patient's body. An image of the surgical site can be obtained by a medical imaging device 20030 that can be operated by the patient-side cart 20032 to change the orientation of the imaging device 20030. The surgical robot hub 20033 can be used to process the image of the surgical site and then display it to the surgeon through the surgeon's console 20036.

[0048] Other types of robotic systems can be readily adapted to be used with the surgical system 20002. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019 / 0201137 (A1) (U.S. Patent Application No. 16 / 209,407) entitled "METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL", filed on Dec. 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0049] Various examples of cloud-based analysis methods implemented by a cloud computing system 20008 and suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0206569 (A1) (U.S. Patent Application No. 16 / 209,403) entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB", filed on Dec. 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0050] In various embodiments, the imaging device 20030 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, Charge-Coupled Device (CCD) sensors and Complementary Metal-Oxide Semiconductor (CMOS) sensors.

[0051] The optical components of the imaging device 20030 may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate a portion of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0052] The one or more illumination sources may be configured to irradiate electromagnetic energy within the visible spectrum as well as the invisible spectrum. The visible spectrum is sometimes also referred to as the optical spectrum or emission spectrum and is a portion of the electromagnetic spectrum that is visible to the human eye (i.e., detectable by the human eye) and may be referred to as visible light or simply light. Typically, the human eye responds to wavelengths of approximately 380 nm to approximately 750 nm in air.

[0053] The invisible spectrum (e.g., non-emission spectrum) is a portion of the electromagnetic spectrum that is located below and above the visible spectrum (i.e., wavelengths less than approximately 380 nm and greater than approximately 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than approximately 750 nm are longer than the red visible spectrum and become invisible infrared (IR), microwaves, and radio electromagnetic radiation. Wavelengths less than approximately 380 nm are shorter than the violet spectrum and become invisible ultraviolet light, x-rays, and gamma ray electromagnetic radiation.

[0054] In various aspects, the imaging device 20030 is configured for use in minimally invasive procedures. Examples of imaging devices suitable for use with the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, choledochoscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophagogastroduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngo-neproscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0055] The imaging device may employ multispectral monitoring to distinguish topography from the underlying structure. A multispectral image captures image data within a specific wavelength range from across the electromagnetic spectrum. The wavelengths can be separated by filters or by using instruments having sensitivity to specific wavelengths including frequencies beyond the visible light range, e.g., IR, and light from ultraviolet. Spectral imaging enables extraction of additional information that cannot be captured by the red, green, and blue receptors of the human eye. The use of multispectral imaging is detailed under the heading “Advanced Imaging Acquisition Module” in U.S. Patent Application Publication No. 2019 / 0200844(A1) (U.S. Patent Application No. 16 / 209,385), filed Dec. 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. Multispectral monitoring can be a useful tool for repositioning the surgical field after a surgical task for performing one or more of the above-described tests on the treated tissue has been completed. It is understood that strict sterilization of the operating room and surgical instruments is required during any surgical procedure. The strict hygiene and sterilization conditions required in the “operating room,” i.e., the operating or treatment room, require the highest possible sterility of all medical devices and instruments. Part of the sterilization process is the need to sterilize anything that comes into contact with the patient or enters the sterile field, including the imaging device 20030 and its accessories and components. It will be understood that the sterile field can be considered a specific area considered to be free of microorganisms, such as within a tray or on a sterile towel, or the sterile field can be considered the area immediately surrounding a patient prepared for surgery. The sterile field can include properly attired and scrubbed team members, as well as all equipment and fixtures within that area.

[0056] The wearable sensing system 20011 shown in FIG. 1 may include one or more sensing systems, such as the HCP sensing system 20020 as shown in FIG. 2. The HCP sensing system 20020 may include a sensing system for monitoring and detecting a set of physical and / or physiological states of a healthcare provider (HCP). The HCP may generally be one or more healthcare providers who assist a surgeon or other healthcare service provider. In one example, the sensing system 20020 may measure a set of biomarkers to monitor the heart rate of the HCP. In one example, the sensing system 20020 (e.g., a watch or a wristband) worn on the wrist of a surgeon may use an accelerometer to detect hand movement and / or shaking, and determine the magnitude and frequency of tremors. The sensing system 20020 may transmit measurement data associated with a set of biomarkers and data associated with the physical state of the surgeon to the surgical hub 20006 for further processing. One or more environmental sensing devices may transmit environmental information to the surgical hub 20006. For example, the environmental sensing device may include a camera 20021 for detecting the position of the HCP's hand / body. The environmental sensing device may include a microphone 20022 for measuring ambient noise in the operating room. Other environmental sensing devices may include devices such as a thermometer for measuring temperature and a hygrometer for measuring the ambient humidity in the operating room. The surgical hub 20006 may, alone or in communication with a cloud computing system, use the surgeon biomarker measurement data and / or environmental sensing information to, for example, modify the control algorithm of a handheld instrument or the average latency of a robot interface to minimize tremors. In one example, the HCP sensing system 20020 may measure one or more surgeon biomarkers associated with the HCP and transmit the measurement data associated with the surgeon biomarkers to the surgical hub 20006.The HCP sensing system 20020 may use one or more of the RF protocols of Bluetooth (registered trademark), Bluetooth Low-Energy (BLE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low-Power Wireless Personal Area Network (6LoWPAN), and Wi-Fi to communicate with the surgical hub 20006. The surgeon biomarkers may include one or more of stress, heart rate, etc. The environmental measurements from the operating room may include ambient noise levels related to the movement of the surgeon or patient, surgeon and / or staff, the attention level of the surgeon and / or staff, etc.

[0057] The surgical hub 20006 may adaptively control one or more surgical instruments 20031 using the surgeon biomarker measurement data associated with the HCP. For example, the surgical hub 20006 may send a control program to the surgical instrument 20031 to control its actuator to limit or compensate for fatigue and the use of fine motor skills. The surgical hub 20006 may send the control program based on situation awareness and / or circumstances regarding the importance or criticality of the task. The control program may instruct the instrument to change its operation to provide more control when control is needed.

[0058] Figure 3 shows an exemplary surgical system 20002 having a surgical hub 20006. The surgical hub 20006 can be paired with a wearable sensing system 20011, an environmental sensing system 20015, a human interface system 20012, a robotic system 20013, and an intelligent instrument 20014 via a modular control unit. The hub 20006 includes a display 20048, an imaging module 20049, a generator module 20050, a communication module 20056, a processor module 20057, a storage array 20058, and an operating room mapping module 20059. In certain embodiments, as illustrated in Figure 3, the hub 20006 further includes a smoke evacuation module 20054 and / or a suction / irrigation module 20055. The various modules and systems can be connected directly or via the communication module 20056 to the modular control unit via a router. Operating room devices can be coupled to cloud computing resources and data storage via the modular control unit. The human interface system 20012 can include a display subsystem and a notification subsystem.

[0059] The modular control unit may be connected to a non-contact sensor module. The non-contact sensor module may use ultrasonic, laser type, and / or similar non-contact measurement devices to measure the dimensions of the operating room and generate a map of the operating room. Other distance sensors can be used to determine the boundaries of the operating room. In U.S. Provisional Patent Application No. 62 / 611,341, filed December 28, 2017, entitled "INTERACTIVE SURGICAL PLATFORM," which is hereby incorporated by reference in its entirety, an ultrasonic-based non-contact sensor module, as described under the heading "Surgical Hub Spatial Awareness Within an Operating Room" in that document, can scan the operating room by transmitting an ultrasonic burst and receiving the echo when the ultrasonic burst is reflected from the outer wall of the operating room. The sensor module may be configured to determine the size of the operating room and adjust the Bluetooth pairing distance limit. A laser-based non-contact sensor module can scan the operating room, for example, by transmitting a laser light pulse, receiving the laser light pulse reflected from the outer wall of the operating room, comparing the phase of the transmitted pulse with the received pulse to determine the size of the operating room, and adjusting the Bluetooth pairing distance limit.

[0060] During surgery, applying energy to tissue for sealing and / or cutting is generally associated with smoke evacuation, aspiration of excess fluid, and / or perfusion of tissue. Fluid lines, power lines, and / or data lines from different sources often become entangled during surgery. Valuable time can be lost in addressing this problem during surgery. To untangle the lines, it may be necessary to unplug them from their corresponding modules, which may require resetting the modules. The hub module type enclosure 20060 provides an integrated environment for managing power lines, data lines, and fluid lines, reducing the frequency of such line entanglements. Aspects of the present disclosure present a surgical hub 20006 for use in surgeries involving applying energy to tissue at a surgical site. The surgical hub 20006 includes a hub enclosure 20060 and a combined generator module slidably receivable within a docking station of the hub enclosure 20060. The docking station includes data contacts and power contacts. The combined generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component housed within a single unit. In one aspect, the combined generator module also includes a smoke evacuation component, at least one energy supply cable for connecting the combined generator module to a surgical instrument, at least one smoke evacuation component configured to discharge smoke, fluid, and / or particulates generated by applying therapeutic energy to tissue, and a fluid line extending from a remote surgical site to the smoke evacuation component. In one aspect, the fluid line may be a first fluid line, and a second fluid line may extend from a remote surgical site to a suction and perfusion module 20055 slidably receivable within the hub enclosure 20060. In one aspect, the hub enclosure 20060 may include a fluid interface. Certain surgeries may require applying two or more energy types to tissue.One type of energy may be more beneficial for cutting tissue, while a different type of energy may be more beneficial for sealing tissue. For example, a bipolar generator can be used to seal tissue, while an ultrasonic generator can be used to cut the sealed tissue. Aspects of the present disclosure present a solution where the hub module enclosure 20060 is configured to house different generators and facilitate bidirectional communication between them. One advantage of the hub module enclosure 20060 is that it allows for the quick removal and / or replacement of various modules. Aspects of the present disclosure present a modular surgical enclosure for use in surgical procedures involving the application of energy to tissue. The modular surgical enclosure includes a first energy generator module configured to generate a first energy for application to tissue, and a first docking station having a first docking port including first data and power contacts, wherein the first energy generator module is slidably movable to engage electrically with the power and data contacts, and the first energy generator module is also slidably movable to disengage from the electrical engagement with the first power and data contacts. In addition to the above, the modular surgical enclosure also includes a second energy generator module configured to generate a second energy different from the first energy for application to tissue, and a second docking station having a second docking port including second data contacts and second power contacts, wherein the second energy generator module is slidably movable to engage electrically with the power and data contacts, and the second energy generator module is also slidably movable to disengage from the electrical engagement with the second power and data contacts. Additionally, the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port configured to facilitate communication between the first energy generator module and the second energy generator module.Referring to FIG. 3, aspects of the present disclosure are presented regarding a hub module type enclosure 20060 that enables modular integration of a generator module 20050, a smoke exhaust module 20054, and a suction / irrigation module 20055. The hub module type enclosure 20060 further facilitates two-way communication between module 20059, module 20054, and module 20055. The generator module 20050 may include integrated monopolar components, bipolar components, and ultrasonic components supported within a single housing unit slidably insertable into the hub's module type enclosure 20060. The generator module 20050 may be configured to connect to a monopolar device 20051, a bipolar device 20052, and an ultrasonic device 20053. Alternatively, the generator module 20050 may include a series of monopolar generator modules, bipolar generator modules, and / or ultrasonic generator modules that interact via the hub module type enclosure 20060. The hub module type enclosure 20060 can be configured to facilitate the insertion of multiple generators and two-way communication between the generators docked to the hub module type enclosure 20060 such that the multiple generators function as a single generator.

[0061] FIG. 4 illustrates a surgical data network having a set of communication hubs configured to connect to a cloud, a set of sensing systems, environmental sensing systems, and a set of other modular devices disposed in one or more operating rooms, patient recovery rooms, or rooms within a medical facility specially equipped for surgery within a medical facility, according to at least one aspect of the present disclosure.

[0062] As illustrated in FIG. 4, the surgical hub system 20060 may include a modular communication hub 20065 configured to connect modular devices disposed within a medical facility to a cloud-based system (e.g., a cloud computing system 20064 that may include a remote server 20067 connected to a remote storage 20068). The modular communication hub 20065 and the devices may be connected in a room within a medical facility specially equipped for surgical procedures. In one aspect, the modular communication hub 20065 may include a network hub 20061 and / or a network switch 20062 that communicate with a network router 20066. The modular communication hub 20065 may also be coupled to a local computer system 20063 and provide local computer processing and data manipulation.

[0063] The computer system 20063 may include a processor and a network interface 20100. The processor may be coupled via a system bus to a communication module, storage, memory, non-volatile memory, and an input / output (I / O) interface. The system bus may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures, examples of which include a 9-bit bus, Industry Standard Architecture (ISA), Micro-Charmel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), USB, Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), Small Computer Systems Interface (SCSI), or any other proprietary bus, but is not limited thereto.

[0064] The processor may be any single-core or multi-core processor, such as those known by the product name ARM Cortex by Texas Instruments. In one aspect, the processor may be, for example, the LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments. This processor core includes on-chip memory of 256KB single-cycle flash memory or other non-volatile memory with a maximum of 40MHz, a prefetch buffer for improving performance beyond 40MHz, 32KB single-cycle serial random access memory (SRAM), internal read-only memory (ROM) with StellarisWare (registered trademark) software, 2KB electrically erasable programmable read-only memory (EEPROM) and / or one or more pulse width modulation (PWM) modules, one or more quadrature encoder input (QEI) analogs, and one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, the details of which are available in the product datasheet.

[0065] In one example, the processor may include a safety controller with two controller-based families such as TMS570 and RM4x, also known by the product name Hercules ARM Cortex R4 from Texas Instruments. The safety controller may be configured specifically for safety-critical applications of IEC61508 and ISO26262, among others, while providing scalable performance, connectivity, and memory options, and providing a high level of integrated safety mechanisms.

[0066] It should be understood that computer system 20063 may include software that functions as a medium between the described user and basic computer resources in a suitable operating environment. Such software can include an operating system. The operating system, which can be stored on disk storage, can function to control and allocate the resources of the computer system. System applications can utilize the resource management by the operating system via program modules and program data stored either in system memory or on disk storage. It should be understood that the various components described herein can be implemented with various operating systems or combinations of operating systems.

[0067] A user can input commands or information into the computer system 20063 via an input device connected to the I / O interface. Examples of input devices include, but are not limited to, pointing devices such as mice, trackballs, styli, touchpads, keyboards, microphones, joysticks, game pads, satellite broadcast receiving antennas, scanners, TV tuner cards, digital cameras, digital video cameras, webcams, etc. These and other input devices are connected to the processor 20102 through the system bus via an interface port. Examples of interface ports include serial ports, parallel ports, game ports, and USB. Output devices use some of the same type of ports as input devices. Thus, for example, a USB port may be used to provide input to the computer system 20063 and output information from the computer system 20063 to an output device. Output adapters may be provided to illustrate that among output devices that may require special adapters, there can be several output devices such as monitors, displays, speakers, and printers. Examples of output adapters include, but are not limited to, video and sound cards that provide connection means between the output device and the system bus. Note that other devices and / or systems of devices, such as remote computers, can provide both input and output functions.

[0068] The computer system 20063 can operate in a networked environment that uses logical connections to one or more remote computers, such as a cloud computer, or a local computer. The remote cloud computer can be, for example, a personal computer, a server, a router, a network PC, a workstation, a microprocessor-based device, a peer device, or other common network nodes, but typically includes many or all of the elements described with respect to the computer system. For simplicity, only a memory storage device is illustrated together with the remote computer. The remote computer can be logically connected to the computer system via a network interface and subsequently physically connected via a communication connection. The network interface can include communication networks such as a local area network (LAN) and a wide area network (WAN). Examples of LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet / IEEE802.3, Token Ring / IEEE802.5, etc. Examples of WAN technologies include circuit-switched networks such as point-to-point links, Integrated Services Digital Network (ISDN) and its variants, packet-switched networks, and Digital Subscriber Line (DSL), but are not limited thereto.

[0069] In various examples, computer system 20063 may include an image processor, an image processing engine, a media processor, or any special digital signal processor (DSP) used for processing digital images. The image processor can enhance speed and efficiency using parallel computing with single instruction, multiple data (SIMD), or multiple instruction, multiple data (MIMD) techniques. The digital image processing engine can perform various tasks. The image processor may be a system on a chip with a multi-core processor architecture.

[0070] The communication connection part may refer to the hardware / software used to connect a network interface to a bus. For exemplary clarity, the communication connection part is shown inside computer system 20063, but the communication connection part may be outside computer system 20063. For illustrative purposes only, the hardware / software required for connection to a network interface may include modems such as ordinary telephone grade modems, cable modems, fiber optic modems, and DSL modems, ISDN adapters, and internal and external technologies such as Ethernet cards. In some examples, the network interface may also be provided using an RF interface.

[0071] The surgical data network associated with the surgical hub system 20060 may be configured as passive, intelligent, or switching. A passive surgical data network functions as a conduit for data, enabling data to go from one device (or segment) to another device (or segment) and to cloud computing resources. An intelligent surgical data network enables traffic to pass through the surgical data network under surveillance and includes additional features that configure each port within the network hub 20061 or the network switch 20062. An intelligent surgical data network may be referred to as a manageable hub or switch. A switching hub reads the destination address of each packet and then forwards the packet to the correct port.

[0072] The modular devices 1a - 1n arranged in the operating room can be connected to the modular communication hub 20065. The network hub 20061 and / or the network switch 20062 can be connected to the network router 20066 to connect the devices 1a - 1n to the cloud computing system 20064 or the local computer system 20063. The data associated with the devices 1a - 1n may be transferred via the router to a cloud - based computer for remote data processing and operation. The data associated with the devices 1a - 1n can also be transferred to the local computer system 20063 for local data processing and operation. The modular devices 2a - 2m arranged in the same operating room may also be connected to the network switch 20062. The network switch 20062 can be connected to the network hub 20061 and / or the network router 20066 to connect the devices 2a - 2m to the cloud 20064. The data associated with the devices 2a - 2m can be transferred via the network router 20066 to the cloud computing system 20064 for data processing and operation. The data associated with the devices 2a - 2m may also be transferred to the local computer system 20063 for local data processing and operation.

[0073] The wearable sensing system 20011 may include one or more sensing systems 20069. The sensing system 20069 may include an HCP sensing system and / or a patient sensing system. One or more sensing systems 20069 may communicate directly through one of the network routers 20066 or through the network hub 20061 or network switching 20062 that communicates with the network router 20066 with the computer system 20063 of the surgical hub system 20060 or the cloud server 20067.

[0074] The sensing system 20069 can be coupled to a network router 20066 to connect the sensing system 20069 to a local computer system 20063 and / or a cloud computing system 20064. Data associated with the sensing system 20069 can be transferred via the network router 20066 to the cloud computing system 20064 for data processing and manipulation. Data associated with the sensing system 20069 may also be transferred to the local computer system 20063 for local data processing and manipulation.

[0075] As illustrated in FIG. 4, the surgical hub system 20060 can be extended by interconnecting a plurality of network hubs 20061 and / or a plurality of network switches 20062 with a plurality of network routers 20066. The modular communication hub 20065 can be housed within a modular control tower configured to receive a plurality of devices 1a - 1n / 2a - 2m. The local computer system 20063 may also be housed within the modular control tower. The modular communication hub 20065 can be connected to a display 20068 to display images obtained by some of the devices 1a - 1n / 2a - 2m, for example, during a surgical procedure. In various aspects, the devices 1a - 1n / 2a - 2m may include various modules such as an imaging module coupled to an endoscope, a generator module coupled to an energy-based surgical device, a smoke evacuation module, a suction / irrigation module, a communication module, a processor module, a storage array, a surgical device coupled to a display, and / or a non-contact sensor module, among other modular devices that can be connected to the modular communication hub 20065 of a surgical data network.

[0076] In one aspect, the surgical hub system 20060 illustrated in FIG. 4 may include a combination of a network hub(s), network switch, and network router(s) that connect devices 1a-1n / 2a-2m, or the sensing system 20069, to the cloud-based system 20064. One or more of the devices 1a-1n / 2a-2m or the sensing system 20069 connected to the network hub 20061 or network switch 20062 may collect data in real time and transfer the data to a cloud computer for data processing and operation. It will be understood that cloud computing relies on sharing computing resources rather than having local servers or personal devices to handle software applications. The term "cloud" may be used as a metaphor for the "Internet", but this term is not so limited. Thus, the term "cloud computing" may be used herein to refer to "one type of Internet-based computing", in which case various services such as servers, storage, and applications are delivered via the Internet to the modular communication hub 20065 and / or computer system 20063 located in an operating room (e.g., a fixed, mobile, temporary, or on-site operating room or space), and to devices connected to the modular communication hub 20065 and / or computer system 20063. The cloud infrastructure may be maintained by a cloud service provider. In this context, the cloud service provider may be an entity that coordinates the use and control of the devices 1a-1n / 2a-2m located in one or more operating rooms. Cloud computing services may perform a number of calculations based on data collected by smart surgical instruments, robots, sensing systems, and other computerized devices located in the operating room. The hub hardware enables multiple devices, sensing systems, and / or connections to connect to a computer that communicates with cloud computing resources and storage.

[0077] By applying cloud computing data processing technology to the data collected by devices 1a to 1n / 2a to 2m, the surgical data network can provide improvements in surgical outcomes, cost reduction, and patient satisfaction. After tissue sealing and cutting procedures, at least some of devices 1a to 1n / 2a to 2m can be used to observe the tissue state to evaluate leakage or perfusion of the sealed tissue. At least some of devices 1a to 1n / 2a to 2m can be used to examine data including images of samples of body tissues for diagnostic purposes using cloud-based computing to identify pathologies such as the effects of diseases. This can include tissue localization, margin confirmation, and phenotypes. At least some of devices 1a to 1n / 2a to 2m can be used to identify the anatomical structures of the body using various sensors integrated with the imaging device and techniques such as overlaying images captured by multiple imaging devices. The data collected by devices 1a to 1n / 2a to 2m, including image data, can be transferred to cloud computing system 20064 or local computer system 20063 or both for data processing and operations including image processing and manipulation. The data may be analyzed to improve the results of surgery by determining whether further treatments such as endoscopic interventions, emerging technologies, targeted radiation, targeted interventions, and precision robotics can be performed on tissue-specific sites and conditions. Such data analysis may further employ prognostic analysis processing, and using standardized methods can provide useful feedback either to confirm surgical treatment and surgeon behavior or to propose modifications to surgical treatment and surgeon behavior.

[0078] Applying cloud computer data processing technology to the measurement data collected by the sensing system 20069 can result in improved surgical outcomes, improved recovery outcomes, reduced costs, and improved patient satisfaction. At least some of the sensing system 20069 may be used to evaluate the physiological state of a surgeon operating on a patient, a patient being prepared for surgery, or a patient recovering after surgery. The cloud-based computing system 20064 can monitor biomarkers associated with a surgeon or patient in real time, generate a surgical plan based at least on the measurement data collected before surgery, supply control signals to surgical instruments during surgery, and be used to notify a patient of complications during the postoperative period.

[0079] The operating room devices 1a to 1n can be connected to the modular communication hub 20065 via a wired channel or a wireless channel according to the configuration of the devices 1a to 1n with respect to the network hub 20061. In one aspect, the network hub 20061 may be implemented as a local network broadcast device that functions on the physical layer of the Open System Interconnection (OSI) model. The network hub can provide connectivity to the devices 1a to 1n located within the same operating room network. The network hub 20061 can collect data in the form of packets and transmit them to the router in half-duplex mode. The network hub 20061 cannot store any media access control / Internet Protocol (MAC / IP) for transferring device data. Only one of the devices 1a to 1n can transmit data at a time via the network hub 20061. The network hub 20061 cannot have a routing table or intelligence regarding the destination of the information and broadcasts all network data across each connection and to the remote server 20067 of the cloud computing system 20064. The network hub 20061 can detect basic network errors such as collisions, but broadcasting all information to multiple ports can pose a security risk and cause bottlenecks.

[0080] The operating room devices 2a to 2m can be connected to the network switch 20062 via a wired channel or a wireless channel. The network switch 20062 functions within the data link layer of the OSI model. The network switch 20062 may be a multicast device for connecting the devices 2a to 2m arranged in the same operating room to the network. The network switch 20062 transmits data in the form of frames to the network router 20066 and can function in full-duplex mode. Multiple devices 2a to 2m can transmit data simultaneously via the network switch 20062. The network switch 20062 stores and uses the MAC addresses of the devices 2a to 2m for transferring data.

[0081] The network hub 20061 and / or the network switch 20062 can be connected to the network router 20066 for connecting to the cloud computing system 20064. The network router 20066 functions within the network layer of the OSI model. The network router 20066 creates a route for transmitting the data packets received from the network hub 20061 and / or the network switch 20062 to the cloud-based computer resources for further processing and operation of the data collected by any one or all of the devices 1a to 1n / 2a to 2m and the wearable sensing system 20011. The network router 20066 may be used to connect two or more different networks located at different positions, such as different operating rooms in the same medical facility or different networks in different operating rooms of different medical facilities. The network router 20066 transmits data in the form of packets to the cloud computing system 20064 and can function in full-duplex mode. Multiple devices can transmit data simultaneously. The network router 20066 can use IP addresses for transferring data.

[0082] In one example, the network hub 20061 may be implemented as a USB hub that enables connecting a plurality of USB devices to a host computer. The USB hub can expand a single USB port into several levels so that there are more available ports for connecting devices to the host system computer. The network hub 20061 may include a wired function or a wireless function for receiving information via a wired channel or a wireless channel. In one aspect, a wireless USB short-range high-bandwidth wireless communication protocol may be used for communication between devices 1a to 1n and devices 2a to 2m located within an operating room.

[0083] In an example, the operating room devices 1a - 1n / 2a - 2m and / or the sensing system 20069 may communicate with the modular communication hub 20065 via the Bluetooth wireless technology standard to exchange data over a short distance (using short - wavelength UHF radio waves in the 2.4 - 2.485 GHz ISM band) and to construct a personal area network (PAN). The operating room devices 1a - 1n / 2a - 2m and / or the sensing system 20069 may communicate with the modular communication hub 20065 via some wireless communication standards or wired communication standards or protocols, such as Bluetooth, Low - Energy Bluetooth, near - field communication (NFC), Wi - Fi (IEEE802.11 family), WiMAX (IEEE802.16 family), IEEE802.20, New Radio (NR), Long Term Evolution (LTE), and also Ev - DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, and Ethernet derivatives thereof, as well as 3G, 4G, 5G, and any other wireless and wired protocols designated in the future, but not limited to these. The computing module may include a plurality of communication modules. For example, the first communication module may be dedicated to short - distance wireless communication such as Wi - Fi and Bluetooth, Low - Energy Bluetooth, Bluetooth Smart, etc., and the second communication module may be dedicated to long - distance wireless communication such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev - DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, etc.

[0084] The modular communication hub 20065 functions as a central connection for one or more of the operating room devices 1a - 1n / 2a - 2m and / or the sensing system 20069 and can handle data types known as frames. The frames can carry data generated by the devices 1a - 1n / 2a - 2m and / or the sensing system 20069. When a frame is received by the modular communication hub 20065, the frame is amplified and sent to the network router 20066, which can transfer this data to the cloud computing system 20064 or the local computer system 20063 by using a number of wireless communication standards or wired communication standards or protocols as described herein.

[0085] The modular communication hub 20065 may be used as a stand - alone device or may be connected to compatible network hubs 20061 and network switches 20062 to form a larger network. The modular communication hub 20065 can be a good option for networking the operating room devices 1a - 1n / 2a - 2m as it is generally easy to install, configure, and maintain.

[0086] FIG. 5 illustrates a logic diagram of a control system 20220 for a surgical instrument or a surgical tool according to one or more aspects of the present disclosure. The surgical instrument or the surgical tool may be configurable. The surgical instrument may be at hand such as an imaging device, a surgical stapler, an energy device, an end cutter device, and may include surgical supplies specific to the procedure. For example, the surgical instrument may include any one of an electric stapler, an electric stapler generator, an energy device, a high energy device, a high energy Joe device, an end cutter clamp, an energy device generator, an in - operating room imaging system, a smoke evacuation device, a suction irrigation device, a pneumoperitoneum system, etc. The system 20220 may include a control circuit. The control circuit may include a microcontroller 20221 including a processor 20222 and a memory 20223. For example, one or more of the sensors 20225, 20226, 20227 provide real - time feedback to the processor 20222. A motor 20230 driven by a motor driver 20229 is operably connected to a longitudinally movable displacement member to drive an I - beam knife element. A tracking system 20228 may be configured to determine the position of the longitudinally movable displacement member. The position information may be provided to a processor 20222 that may be programmed or configured to determine the position of the longitudinally movable drive member as well as the position of the firing member, the firing bar, and the I - beam knife element. Additional motors may be provided to a tool driver interface to control the firing of the I - beam, the movement of the closure tube, the rotation of the shaft, and the articulation movement. A display 20224 may display various operating states of the instrument and may include a touch - screen function for data input. The information displayed on the display 20224 may be overlaid with an image acquired via an endoscope imaging module.

[0087] The microcontroller 20221 may be any single-core or multi-core processor, such as those known by the trade names of ARM Cortex made by Texas Instruments. In one aspect, the main microcontroller 20221 may be, for example, an on-chip memory of 256KB single-cycle flash memory or other non-volatile memory with a maximum of 40MHz, the details of which are available in the product datasheet, a prefetch buffer for improving performance beyond 40MHz, 32KB single-cycle SRAM, an internal ROM with StellarisWare® software, 2KB EEPROM, one or more PWM modules, one or more QEI analogs, and / or one or more 12-bit ADCs with 12 analog input channels, which may be the LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments.

[0088] The microcontroller 20221 may also include a safety controller with two controller-based families such as TMS570 and RM4x known by the trade names of Hercules ARM Cortex R4 also made by Texas Instruments. The safety controller may be configured specifically for safety-critical applications of IEC61508 and ISO26262, among others, while providing scalable performance, connectivity, and memory options, and providing advanced integrated safety mechanisms.

[0089] The microcontroller 20221 may be programmed to perform various functions such as precise control of the speed and position of the knife and the articulation movement system. In one aspect, the microcontroller 20221 may include a processor 20222 and a memory 20223. The electric motor 20230 may be a brushed direct current (DC) motor with a gearbox and a mechanical connection to the articulation movement section or the knife system. In one aspect, the motor driver 20229 may be an A3941 available from Allegro Microsystems, Inc. Other motor drivers may be readily substituted for use in the tracking system 20228 with an absolute positioning system. A detailed description of the absolute positioning system is described in U.S. Patent Application Publication No. 2017 / 0296213, published on October 19, 2017, titled "SYSTEMS AND METHODS FOR CONTROLLING A SURGICAL STAPLING AND CUTTING INSTRUMENT", which is hereby incorporated by reference in its entirety.

[0090] The microcontroller 20221 may be programmed to provide accurate control of the displacement member and the speed and position of the articulation movement system. The microcontroller 20221 may be configured to calculate a response within the software of the microcontroller 20221. The calculated response can be compared with the measured response of the actual system to obtain an "observed" response, which is used for actual feedback decision-making. The observed response may be a suitably adjusted value that balances the smooth and continuous nature of the simulated response with the measured response, which can detect external influences on the system.

[0091] The motor 20230 may be controlled by a motor driver 20229 and can also be used by a surgical instrument or a tool firing system. In various forms, the motor 20230 may be a brushed DC drive motor having a maximum rotational speed of about 25,000 RPM. In some examples, the motor 20230 may include a brushless motor, a cordless motor, a synchronous motor, a stepper motor, or any other suitable electric motor. The motor driver 20229 may include, for example, an H-bridge driver including field effect transistors (FETs). The motor 20230 may be powered by a power supply assembly removably attached to a handle assembly or a tool housing to supply control power to a surgical instrument or a tool. The power supply assembly may include a battery including a number of battery cells connected in series that can be used as a power supply to power a surgical instrument or a tool. In certain situations, the battery cells of the power supply assembly may be replaceable and / or rechargeable. In at least one example, the battery cells may be lithium-ion batteries that can be connectable to and separable from the power supply assembly.

[0092] The motor driver 20229 may be the A3941 available from Allegro Microsystems, Inc. The A3941 may be a full-bridge controller for use with an external N-channel power metal-oxide semiconductor field-effect transistor (MOSFET) specifically designed for inductive loads such as brushed DC motors. The driver 20229 may include an inherent charge pump regulator, which supplies a full (>10V) gate drive to a battery voltage up to 7V, enabling the A3941 to operate with a reduced gate drive up to 5.5V. A bootstrap capacitor may be used to supply the above battery supply voltage required for the N-channel MOSFET. The internal charge pump for high-side drive enables DC (100% duty cycle) operation. The full bridge can be driven in fast or slow decay mode using diodes or synchronous rectification. In slow decay mode, current recirculation is possible by either the high-side FET or the low-side FET. The power FET can be protected from shoot-through by a resistor-adjustable dead time. The integrated diagnostics indicate low voltage, over-temperature, and power bridge anomalies and can be configured to protect the power MOSFET under most short-circuit conditions. Other motor drivers may be easily substituted for use in the tracking system 20228 with an absolute positioning system.

[0093] Tracking system 20228 can include a controlled motor drive circuit configuration with a position sensor 20225 according to one aspect of the present disclosure. The position sensor 20225 for an absolute positioning system can supply a unique position signal corresponding to the position of a displacement member. In some examples, the displacement member can represent a longitudinally movable drive member having a rack of drive teeth for meshing engagement with a corresponding drive gear of a gear reduction assembly. In some examples, the displacement member can represent a firing member adapted and configured to include a rack of drive teeth. In some examples, the displacement member can represent a firing bar or an I-beam, each of which can be adapted and configured to include a rack of drive teeth. Thus, as used herein, the term displacement member can generally be used to refer to any movable member of a surgical instrument or tool, such as a drive member, a firing member, a firing bar, an I-beam, or any element that can be displaced. In one aspect, the longitudinally movable drive member can be coupled to a firing member, a firing bar, and an I-beam. Thus, the absolute positioning system can effectively track the linear displacement of the I-beam by tracking the linear displacement of the longitudinally movable drive member. In various aspects, the displacement member can be coupled to any position sensor 20225 suitable for measuring linear displacement. Thus, a longitudinally movable drive member, a firing member, a firing bar, or an I-beam, or a combination thereof, can be coupled to any suitable linear displacement sensor. The linear displacement sensor can include a contact displacement sensor or a non-contact displacement sensor.Linear displacement sensors may include a linear variable differential transformer (LVDT), a differential variable reluctance transducer (DVRT), a slide potentiometer, a magnetic sensing system including a movable magnet and a series of linearly arranged Hall effect sensors, a magnetic sensing system including a fixed magnet and a series of linearly arranged movable Hall effect sensors, an optical detection system including a movable light source and a series of linearly arranged photodiodes or photodetectors, an optical sensing system including a fixed light source and a series of linearly arranged movable photodiodes or photodetectors, or any combination thereof.

[0094] The electric motor 20230 may include a rotatable shaft that operably interfaces with a gear assembly attached to mesh with a set of drive teeth or a rack on a displacement member. The sensor element may be operably coupled to the gear assembly such that one rotation of the position sensor 20225 element corresponds to some linear longitudinal translation of the displacement member. The configuration of the gear ring and sensor may be connected to a linear actuator by a rack and pinion configuration or to a rotary actuator by spur gears or other connections. A power supply may supply power to the absolute positioning system, and an output indicator may display the output of the absolute positioning system. The displacement member may represent a longitudinally movable drive member having a rack of drive teeth formed thereon for meshing with a corresponding drive gear of a gear reduction assembly. The displacement member may represent a longitudinally movable emitter member, emitter bar, I-beam, or a combination thereof.

[0095] One rotation of the sensor element associated with the position sensor 20225 can correspond to a longitudinal linear displacement d1 of the displacement member, where d1 is the longitudinal linear distance that the displacement member moves from point "a" to point "b" after one rotation of the sensor element connected to the displacement member. The sensor device can be connected via a gear reduction device in which the position sensor 20225 completes one or more rotations with respect to the full stroke of the displacement member. The position sensor 20225 may complete multiple rotations with respect to the full stroke of the displacement member.

[0096] To provide a unique position signal for two or more rotations of the position sensor 20225, a series of switches (where n is an integer greater than 1) may be used alone or in combination with a gear reduction device. The state of the switch can be fed back to the microcontroller 20221, which applies logic to determine a unique position signal corresponding to the longitudinal linear displacement d1 + d2 +... dn of the displacement member. The output of the position sensor 20225 is supplied to the microcontroller 20221. The position sensor 20225 of the sensor device can comprise a magnetic sensor, an analog rotary sensor such as a potentiometer, or an array of analog Hall effect elements that output a unique combination of position signals or values.

[0097] The position sensor 20225 can comprise any number of magnetic sensing elements such as, for example, magnetic sensors classified by measuring the total magnetic field or vector components of the magnetic field. The technologies used to produce both types of magnetic sensors can encompass many aspects of physics and electronics. Technologies used for sensing magnetic fields include, among others, search coils, flux gates, optical pumping, nuclear precession, SQUIDs, Hall effect, anisotropic magnetoresistance, giant magnetoresistance, magnetic tunnel junctions, giant magnetoimpedance, magnetostrictive / piezoelectric composites, magnetic diodes, magnetic transistors, optical fibers, magnetooptics, and microelectromechanical system-based magnetic sensors.

[0098] The position sensor 20225 of the tracking system 20228 with an absolute positioning system may comprise a magnetic rotary absolute positioning system. The position sensor 20225 may be implemented as an AS5055EQFT single-chip magnetic rotary position sensor available from Austria Microsystems, AG. The position sensor 20225 is connected to the microcontroller 20221 to realize an absolute positioning system. The position sensor 20225 is a low-voltage and low-power component and may include four Hall effect elements in the area of the position sensor 20225 that can be arranged above the magnet. Also, a high-resolution ADC and a smart power management controller may be provided on the chip. A coordinate rotation digital computer (CORDIC) processor, also known as the digit-by-digit method and the border algorithm, may be provided to implement a concise and efficient algorithm for calculating hyperbolic and trigonometric functions that only require addition, subtraction, bit shift, and table reference operations. The angular position, alarm bit, and magnetic field information may be transmitted to the microcontroller 20221 via a standard serial communication interface such as a serial peripheral interface (SPI) interface. The position sensor 20225 may provide a resolution of 12 bits or 14 bits. The position sensor 20225 may be an AS5055 chip provided in a small QFN16-pin 4×4×0.85 mm package.

[0099] Tracking system 20228 with an absolute positioning system may include and / or be programmed to implement a feedback controller such as a PID, a state feedback, and an adaptive controller. The power supply converts a signal from the feedback controller into a physical input to the system, in this case a voltage. Other examples include PWM of voltage, current, and force. In addition to the position measured by position sensor 20225, other sensors (sensors) may be provided to measure physical parameters of the physical system. In some embodiments, other sensors (sensors) include those described in U.S. Patent No. 9,345,481, issued May 24, 2016, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety, U.S. Patent Application Publication No. 2014 / 0263552, published September 18, 2014, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety, and U.S. Patent Application No. 15 / 628,175, filed June 20, 2017, entitled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," which is incorporated herein by reference in its entirety, such as sensor arrangements. In a digital signal processing system, the absolute positioning system is coupled to a digital data acquisition system, where the output of the absolute positioning system has a finite resolution and sampling frequency. The absolute positioning system may include a comparison and combination circuit to combine the calculated response with the measured response using algorithms such as weighted averages and theoretical control loops that drive the calculated response towards the measured response. The calculated response of the physical system may take into account characteristics such as mass, inertia, viscous friction, and inductive resistance to predict how the state and output of the physical system will behave given the input.

[0100] An absolute positioning system can provide the absolute position of a displacement member upon power-up of the instrument without retracting or advancing the displacement member to a reset (zero or home) position, which may require a conventional rotary encoder that simply counts the number of forward or backward steps taken by a motor 20230 to estimate the position of a device actuator, drive bar, knife, etc.

[0101] For example, a sensor 20226, such as a strain gauge or a micro strain gauge, may be configured to measure one or more parameters of an end effector, such as, for example, the closing force applied to an anvil, or the amplitude of the strain exerted on the anvil during a clamping operation. The measured strain can be converted into a digital signal and provided to a processor 20222. Instead of, or in addition to, the sensor 20226, a sensor 20227, such as a load cell, for example, may measure the closing force applied to the anvil by a closing drive system. For example, a sensor 20227, such as a load cell, may measure the firing force applied to an I-beam during a firing stroke of a surgical instrument or tool. The I-beam is configured to engage a wedge thread, which is configured to cam a staple driver upward to eject staples into deformable contact with the anvil. The I-beam may also include a sharp cutting edge that can be used to cut tissue when the I-beam is advanced distally by a firing bar. Alternatively, a current sensor 20231 may be used to measure the current consumed by the motor 20230. The force required to advance the firing member can correspond, for example, to the current drawn by the motor 20230. The measured force can be converted into a digital signal and provided to the processor 20222.

[0102] For example, a strain gauge sensor 20226 may be used to measure the force applied to tissue by an end effector. To measure the force exerted by the end effector on the tissue being treated, the strain gauge may be coupled to the end effector. A system for measuring the force applied to tissue grasped by an end effector may comprise a strain gauge sensor 20226, such as a micro strain gauge, configured to measure one or more parameters of the end effector. In one aspect, the strain gauge sensor 20226 can measure the amplitude or magnitude of the strain exerted on the jaw members of the end effector during a clamping operation, which can indicate tissue compression. The measured strain can be converted into a digital signal and supplied to a processor 20222 of a microcontroller 20221. A load sensor 20227 may measure, for example, the force used to operate a knife element to cut tissue captured between an anvil and a staple cartridge. A magnetic field sensor can be used to measure the thickness of the captured tissue. The measurements of the magnetic field sensor may also be converted into a digital signal and provided to the processor 20222.

[0103] The measured values of tissue compression, tissue thickness, and / or the force required to close the end effector on the tissue, respectively measured by sensors 20226, 20227, may be used by the microcontroller 20221 to characterize corresponding values of a selected position of a firing member and / or the speed of the firing member. In one embodiment, the memory 20223 may store techniques, equations, and / or look-up tables that may be used by the microcontroller 20221 during evaluation.

[0104] The control system 20220 of the surgical instrument or tool may also comprise a wired or wireless communication circuit for communicating with a surgical hub 20065 as shown in FIG. 4.

[0105] FIG. 6 illustrates an exemplary surgical system 20280 according to the present disclosure, which may include a surgical instrument 20282 that can communicate with a console 20294 or a portable device 20296 through a local area network 20292 and / or a cloud network 20293 via a wired and / or wireless connection. The console 20294 and the portable device 20296 may be any suitable computing device. The surgical instrument 20282 may include a handle 20297, an adapter 20285, and a loading unit 20287. The adapter 20285 is releasably coupled to the handle 20297, and the loading unit 20287 is releasably coupled to the adapter 20285 such that the adapter 20285 transmits force from a drive shaft to the loading unit 20287. The adapter 20285 or the loading unit 20287 may include a force gauge (not explicitly shown) disposed therein for measuring the force applied to the loading unit 20287. The loading unit 20287 may include an end effector 20289 having a first jaw 20291 and a second jaw 20290. The loading unit 20287 may be an in-vivo loading unit, i.e., a multi-firing loading unit (MFLU), that allows a clinician to fire multiple fasteners multiple times without removing the loading unit 20287 from the surgical site to reload the loading unit 20287.

[0106] The first jaw 20291 and the second jaw 20290 may be configured to clamp tissue therebetween, fire a fastener through the clamped tissue, and cut the clamped tissue. The first jaw 20291 may be configured to fire at least one fastener multiple times, or may be configured to include a replaceable multi-firing fastener cartridge that can fire multiple fasteners (e.g., staples, clips, etc.) more than once before being replaced. The second jaw 20290 may include an anvil that deforms or otherwise secures the fastener as the fastener is ejected from the multi-firing fastener cartridge.

[0107] The handle 20297 may include a motor coupled to the drive shaft so as to act on the rotation of the drive shaft. The handle 20297 may include a control interface for selectively activating the motor. The control interface may include buttons, switches, levers, sliders, touchscreens, and any other suitable input mechanism or user interface, which may be engaged by a clinician to activate the motor.

[0108] The control interface of the handle 20297 may communicate with a controller 20298 of the handle 20297 to selectively activate the motor and act on the rotation of the drive shaft. The controller 20298 may be disposed within the handle 20297 and may be configured to receive inputs from the control interface and adapter data from the adapter 20285 or loading unit data from the loading unit 20287. The controller 20298 may analyze inputs from the control interface and data received from the adapter 20285 and / or the loading unit 20287 to selectively activate the motor. The handle 20297 may also include a display visible to the clinician during use of the handle 20297. The display may be configured to display portions of the adapter or loading unit data before, during, or after firing of the instrument 20282.

[0109] The adapter 20285 may include an adapter identification device 20284 disposed therein, and the loading unit 20287 may include a loading unit identification device 20288 disposed therein. The adapter identification device 20284 may communicate with the controller 20298, and the loading unit identification device 20288 may communicate with the controller 20298. It will be understood that the loading unit identification device 20288 may communicate with the adapter identification device 20284 that relays or passes communication from the loading unit identification device 20288 to the controller 20298.

[0110] Adapter 20285 may also include a plurality of sensors 20286 (one shown) disposed therearound for detecting various states of the adapter 20285 or the environment (e.g., whether the adapter 20285 is connected to the loading unit, whether the adapter 20285 is connected to the handle, whether the drive shaft is rotating, the torque of the drive shaft, the strain of the drive shaft, the temperature within the adapter 20285, the number of firings of the adapter 20285, the peak force of the adapter 20285 during firing, the total amount of force applied to the adapter 20285, the peak recoil force of the adapter 20285, the number of rest periods of the adapter 20285 during firing, etc.). The plurality of sensors 20286 may provide an input to the adapter identification device 20284 in the form of a data signal. The data signals of the plurality of sensors 20286 may be stored within the adapter identification device 20284 or may be used to update the adapter data stored within the adapter identification device 20284. The data signals of the plurality of sensors 20286 may be analog or digital. The plurality of sensors 20286 may include a force gauge for measuring the force exerted on the loading unit 20287 during firing.

[0111] The handle 20297 and the adapter 20285 may be configured to interconnect the adapter identification device 20284 and the loading unit identification device 20288 with the controller 20298 via an electrical interface. The electrical interface may be a direct electrical interface (i.e., including electrical contacts that engage each other to transmit energy and signals therebetween). Additionally or alternatively, the electrical interface may be a non-contact electrical interface for wirelessly transmitting (e.g., inductively transmitting) energy and signals therebetween. It is also contemplated that the adapter identification device 20284 and the controller 20298 may wirelessly communicate with each other via a wireless connection separate from the electrical interface.

[0112] The handle 20297 may include a transceiver 20283 configured to transmit instrument data from the controller 20298 to other components of the system 20280 (e.g., the LAN 20292, the cloud 20293, the console 20294, or the portable device 20296). The controller 20298 may also transmit instrument data and / or measurement data associated with one or more sensors 20286 to the surgical hub. The transceiver 20283 may receive data (e.g., cartridge data, loading unit data, adapter data, or other notifications) from the surgical hub 20270. The transceiver 20283 may receive data (e.g., cartridge data, loading unit data, or adapter data) from other components of the system 20280. For example, the controller 20298 may transmit instrument data including the serial number of a mounting adapter (e.g., the adapter 20285) attached to the handle 20297, the serial number of a loading unit (e.g., the loading unit 20287) attached to the adapter 20285, and the serial numbers of a plurality of firing fastener cartridges loaded into the loading unit to the console 20294. Subsequently, the console 20294 may return data (e.g., cartridge data, loading unit data, or adapter data) associated with the attached cartridge, loading unit, and adapter, respectively, to the controller 20298. The controller 20298 may display a message on a local instrument display or, alternatively, transmit a message via the transceiver 20283 to the console 20294 or the portable device 20296 to display the message on the display 20295 or the portable device screen, respectively.

[0113] FIG. 7 illustrates a diagram of a situation awareness surgical system 5100 according to at least one aspect of the present disclosure. The data source 5126 can include, for example, a modular device 5102 (which can include sensors configured to detect parameters associated with a patient, an HCP, and the environment, and / or the modular device itself), a database 5122 (e.g., an EMR database including patient records), a patient monitoring device 5124 (e.g., a blood pressure (BP) monitor and an electrocardiography (EKG) monitor), an HCP monitoring device 35510, and / or an environmental monitoring device 35512. The surgical hub 5104 can be configured to derive context information regarding a surgical procedure from the data, for example, based on a particular combination of the received data or the particular order in which data is received from the data source 5126. The context information inferred from the received data can include, for example, the type of surgical procedure being performed, a particular step of the surgical procedure that the surgeon is performing, the type of tissue being operated on, or the body cavity that is the subject of the procedure. This function according to some aspects of the surgical hub 5104 for deriving or inferring information regarding a surgical procedure from the received data can be referred to as "situation awareness." For example, the surgical hub 5104 can incorporate a situation awareness system, which is the hardware and / or programming associated with the surgical hub 5104 that derives context information regarding a surgical procedure from received data and / or surgical planning information received from an edge computing system 35514 or a corporate cloud server 35516.

[0114] The situation recognition system of the surgical hub 5104 can be configured to derive context information from data received from various different data sources 5126. For example, the situation recognition system can include a pattern recognition system, or a machine learning system (such as an artificial neural network) trained with training data to correlate various inputs (e.g., data from database 5122, patient monitoring device 5124, modular device 5102, HCP monitoring device 35510, and / or environmental monitoring device 35512) with corresponding context information regarding the surgical procedure. The machine learning system can be trained to accurately derive context information regarding the surgical procedure from the provided inputs. In an example, the situation recognition system can include a lookup table that stores pre-characterized context information regarding the surgical procedure in association with one or more inputs (or a range of inputs) corresponding to that context information. In response to a query with one or more inputs, the lookup table can return the corresponding context information of the situation recognition system to control the modular device 5102. In an example, the context information received by the situation recognition system of the surgical hub 5104 can be associated with a specific control adjustment, or a series of control adjustments, of one or more modular devices 5102. In an example, the situation recognition system can include a further machine learning system, lookup table, or other such system that generates or reads one or more control adjustments of one or more modular devices 5102 when the context information is provided as an input.

[0115] The surgical hub 5104 incorporating the situation awareness system can provide many advantages to the surgical system 5100. One advantage can include providing improved interpretation of sensed and collected data, which can improve the processing accuracy during the surgical procedure and / or the use of the data. Returning to the previous example, the situation awareness surgical hub 5104 can determine which type of tissue is being operated on, and thus, if an unexpectedly high force is detected to close the end effector of the surgical instrument, the situation awareness surgical hub 5104 can correctly accelerate or decelerate the motor of the surgical instrument according to the tissue type.

[0116] The type of tissue being operated on can affect the adjustments made to the compression speed and load threshold of a surgical stapling and cutting instrument for specific tissue gap measurements. The situation awareness surgical hub 5104 can infer whether the surgical procedure being performed is a thoracic procedure or an abdominal procedure, whereby the surgical hub 5104 can determine whether the tissue clamped by the end effector of the surgical stapling and cutting instrument is lung tissue (in the case of a thoracic procedure) or stomach tissue (in the case of an abdominal procedure). The surgical hub 5104 can then appropriately adjust the compression speed and load threshold of the surgical stapling and cutting instrument according to the type of tissue.

[0117] The type of body cavity being operated on during a insufflation procedure can affect the function of the smoke evacuation device. The situation awareness surgical hub 5104 can determine whether the surgical site is under pressure (by determining that the surgical procedure utilizes insufflation) and can determine the type of procedure. Generally, since a certain type of procedure can be performed within a specific body cavity, the surgical hub 5104 can appropriately control the motor speed of the smoke evacuation device according to the body cavity being operated on. Thus, the situation awareness surgical hub 5104 can provide a consistent amount of smoke evacuation for both thoracic and abdominal procedures.

[0118] The type of procedure being performed can affect the energy level optimal for the operation of an ultrasonic surgical instrument or a radio frequency (RF) electrosurgical instrument. For example, in arthroscopic procedures, the end effector of an ultrasonic surgical instrument or an RF electrosurgical instrument is immersed in fluid, which may require a higher energy level. The Situational Awareness Surgical Hub 5104 can determine whether the surgical procedure is an arthroscopic procedure. The Surgical Hub 5104 can then adjust the RF power level of the generator or the ultrasonic amplitude (e.g., "energy level") to compensate for the fluid-filled environment. In relation, the type of tissue being operated on can affect the energy level optimal for the operation of an ultrasonic surgical instrument or an RF electrosurgical instrument. The Situational Awareness Surgical Hub 5104 can determine which type of surgical procedure is being performed and then customize the energy level of the ultrasonic surgical instrument or the RF electrosurgical instrument, respectively, according to the tissue shape expected for the surgical procedure. Further, the Situational Awareness Surgical Hub 5104 can be configured to adjust the energy level of the ultrasonic surgical instrument or the RF electrosurgical instrument not only for each procedure but also over the course of the surgical procedure. The Situational Awareness Surgical Hub 5104 can determine which step of the surgical procedure is being performed or will be performed next and then update the control algorithm of the generator and / or the ultrasonic surgical instrument or the RF electrosurgical instrument to set the energy level to an appropriate value for the type of tissue expected according to the steps of the surgical procedure.

[0119] In an example, the surgical hub 5104 can derive data from an additional data source 5126 to improve the conclusions drawn from one data source 5126. The situation awareness surgical hub 5104 can enhance the data received from the modular device 5102 with context information constructed regarding the surgical procedure from other data sources 5126. For example, the situation awareness surgical hub 5104 can be configured to determine whether hemostasis has occurred (e.g., whether bleeding at the surgical site has stopped) according to video or image data received from a medical imaging device. The surgical hub 5104 can be further configured to compare physiological measurements (e.g., blood pressure sensed by a BP monitor communicatively connected to the surgical hub 5104) with visual or image data of hemostasis (e.g., from a medical imaging device communicatively coupled to the surgical hub 5104) to make a determination regarding the integrity of staple lines or tissue welds. The situation awareness system of the surgical hub 5104 can provide additional context when analyzing visualization data considering physiological measurement data. The additional context can be useful when the visualization data may not be conclusive or may be incomplete by itself.

[0120] For example, if it is determined in a subsequent step of a procedure that the use of an instrument is required, the situation awareness surgical hub 5104 can actively activate the generator to which an RF electrosurgical instrument is connected. Actively activating the energy source can make it possible to have the instrument ready for use as soon as the preceding steps of the procedure are completed.

[0121] The situation-aware surgical hub 5104 can determine whether the current step or subsequent steps of a surgical procedure require different views or magnifications on the display according to the features of the surgical site that the surgeon is expected to view. The surgical hub 5104 can actively change the displayed view (e.g., supplied from a medical imaging device for a visualization system) as appropriate, whereby the display automatically adjusts throughout the surgical procedure.

[0122] The situation-aware surgical hub 5104 can determine which step of a surgical procedure is being performed or will be performed next and whether specific data or a comparison between data is required for that step of the surgical procedure. The surgical hub 5104 can be configured to automatically call up a data screen based on the step of the surgical procedure being performed without waiting for the surgeon to request specific information.

[0123] During the setup of a surgical procedure or during the surgical procedure itself, errors can be checked. For example, the Situation Awareness Surgical Hub 5104 can determine whether the operating room is properly or optimally set up for the surgical procedure to be performed. The Surgical Hub 5104 can determine the type of surgical procedure being performed, read the corresponding checklist, product locations, or setup requirements (e.g., from memory), and then be configured to compare the current operating room layout to the standard layout for the type of surgical procedure that the Surgical Hub 5104 has determined is being performed. In some examples, the Surgical Hub 5104 can compare a list of items for the procedure and / or a list of devices paired with the Surgical Hub 5104 to the recommended or expected manifest of items and / or devices for a given surgical procedure. If there is any discrepancy between the lists, the Surgical Hub 5104 can provide an alert indicating that a particular modular device 5102, patient monitoring device 5124, HCP monitoring device 35510, environmental monitoring device 35512, and / or other surgical supplies are missing. In some examples, the Surgical Hub 5104 can determine, for example, the relative distance or relative position of the modular device 5102 and the patient monitoring device 5124 via a proximity sensor. The Surgical Hub 5104 can compare the relative position of the devices to the layout recommended or expected for a particular surgical procedure. If there is any discrepancy between the layouts, the Surgical Hub 5104 can be configured to provide an alert indicating that the current layout of the surgical procedure deviates from the recommended layout.

[0124] The situation awareness surgical hub 5104 can determine whether a surgeon (or other HCP) is making a mistake or deviating from a series of actions expected during a surgical procedure. For example, the surgical hub 5104 can determine the type of surgical procedure being performed, read a corresponding list of steps or order of device use (e.g., from memory), and then compare the steps being taken or devices being used during the surgical procedure to the steps or devices expected for the type of surgical procedure that the surgical hub 5104 has determined is being performed. The surgical hub 5104 can provide an alert indicating that an unexpected action is being taken at a particular step in the surgical procedure or that an unexpected device is being utilized.

[0125] Surgical instruments (and other modular devices 5102) can be adjusted to suit the specific context of each surgical procedure (such as adjustment to different tissue types) and can verify actions during the surgical procedure. The next steps, data, and display adjustments can be provided to the surgical instruments (and other modular devices 5102) in the operating room according to the specific context of the procedure.

[0126] Surgical autonomous systems, devices, and methods can include aspects of integration with other medical devices, data sources, processes, and institutions. Surgical autonomous systems, devices, and methods can include, for example, aspects of integration with a computer-implemented bidirectional surgical system and / or one or more elements of a computer-implemented bidirectional surgical system. Surgical systems, surgical autonomous systems, and autonomous surgical systems can be compatible as described herein.

[0127] Referring to FIG. 8, an overview of the surgical autonomous system 49000 can be provided. The surgical instrument A 49005 and / or the surgical instrument B 49035 can be used in a surgical operation as part of the surgical system 49000. The surgical hub 4900 can also be configured to regulate the information flow to the display of the surgical instrument. For example, the surgical hub may be described in U.S. Patent Application Publication No. 2019 / 0200844(A1) (U.S. Patent Application No. 16 / 209,385) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY" filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety. Exemplary surgical instruments suitable for use with the surgical system 49000 are described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019 / 0200844(A1) (U.S. Patent Application No. 16 / 209,385) filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety.

[0128] FIG. 8 shows an example of the surgical autonomous system 49000. The system 49000 can be used to perform a surgical operation on a patient lying on an operating table in an operating room. The robotic system can be used as part of the surgical system in a surgical operation. For example, the robotic system may be described in U.S. Patent Application Publication No. 2019 / 0200844(A1) (U.S. Patent Application No. 16 / 209,385) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY" filed on December 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety. The robotic hub can be used to process an image of the surgical site and then display it to the surgeon through the surgeon's console.

[0129] Other types of robotic systems can be readily adapted to be used with the surgical system 49000. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019 / 0201137 (A1) (U.S. Patent Application No. 16 / 209,407) entitled "METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL", filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety.

[0130] Various examples of cloud-based analysis methods implemented by the cloud and suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0206569 (A1) (U.S. Patent Application No. 16 / 209,403) entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB", filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety.

[0131] In various aspects, the imaging device may be used in a surgical system and may include at least one image sensor and one or more optical components. Suitable image sensors can include, but are not limited to, Charge-Coupled Device (CCD) sensors and Complementary Metal-Oxide Semiconductor (CMOS) sensors.

[0132] The optical components of the imaging device may include one or more light sources and / or one or more lenses. The one or more light sources can be directed to illuminate a portion of the surgical field. The one or more image sensors can receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0133] One or more illumination sources may be configured to irradiate electromagnetic energy in the visible spectrum as well as in the invisible spectrum. The visible spectrum is sometimes also referred to as the optical spectrum or emission spectrum and is a portion of the electromagnetic spectrum that is visible to the human eye (e.g., detectable by the human eye) and may be referred to as visible light or simply light. A typical human eye responds to wavelengths of approximately 380 nm to approximately 750 nm in air.

[0134] The invisible spectrum (e.g., non-emission spectrum) is a portion of the electromagnetic spectrum that is located below and above the visible spectrum (i.e., wavelengths less than approximately 380 nm and greater than approximately 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than approximately 750 nm are longer than the red visible spectrum and these become invisible infrared (IR), microwaves, and radio electromagnetic radiation. Wavelengths less than approximately 380 nm are shorter than the violet spectrum and these become invisible ultraviolet, x-rays, and gamma ray electromagnetic radiation.

[0135] In various aspects, the imaging device may be configured for use in minimally invasive procedures. Examples of imaging devices suitable for use with the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, choledochoscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophagogastroduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngo-ureteroscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0136] The imaging device may employ multispectral monitoring to distinguish topography from the underlying structure. A multispectral image captures image data within a specific wavelength range from across the electromagnetic spectrum. The wavelengths can be separated by filters or by using instruments having sensitivity to specific wavelengths including frequencies beyond the visible light range, e.g., IR, and light from ultraviolet. Spectral imaging enables extraction of additional information that cannot be captured by the red, green, and blue receptors of the human eye. The use of multispectral imaging is described in more detail under the heading “Advanced Imaging Acquisition Module” in U.S. Patent Application Publication No. 2019-0200844(A1) (U.S. Patent Application No. 16 / 209,385), filed on Dec. 4, 2018, the disclosure of which is hereby incorporated by reference in its entirety. Multispectral monitoring can be a useful tool for repositioning the surgical field after a surgical task for performing one or more of the above-described tests on the treated tissue has been completed. It is understood that strict sterilization of the operating room and surgical instruments is required during any surgical procedure. The strict hygiene and sterilization conditions required in the “operating room,” i.e., the operating room or treatment room, require the highest possible sterility of all medical devices and instruments. Part of the sterilization process is the need to sterilize anything that comes into contact with the patient or enters the sterile field, including the imaging device and its accessories and components. It will be understood that the sterile field can be considered a specific area considered to be free of microorganisms, such as within a tray or on a sterile towel, or the sterile field can be considered the area immediately surrounding a patient prepared for surgery. The sterile field can include properly attired and scrubbed team members, as well as all equipment and fixtures within that area.

[0137] Surgical instrument A 49005 and / or surgical instrument B 49035 may have one or more capabilities. (For example, surgical instrument A has capabilities B, Z, and D, and surgical instrument B has capabilities C, F, and E.) The capabilities may be associated with functions that the surgical instruments can perform (e.g., function A 49010 associated with surgical instrument A 49005 and function B 49030 associated with surgical instrument B 49035). Examples of functions may be described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019 / 0200844 (A1) (U.S. Patent Application No. 16 / 209,385) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY" filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. For example, if the surgical instrument is an end cutter, one of the capabilities may be to excise tissue (e.g., the tissue surrounding the colon when a surgeon performs a colectomy). The capabilities may include surgical tasks as described with respect to FIG. 9. For example, the ability to excise tissue may include control of the energy source, cutting, stapling, knob orientation, body orientation, body position, anvil jaw force, reload alignment slot management, and / or the like.

[0138] Data may be generated based on the performance of surgical instrument A 49005 and / or B 49035 (e.g., by a monitoring module located in surgical hub 49040 or locally by the surgical instrument). The data may be related to how the surgical instrument is operating with respect to performance. For example, the data may be associated with physical measurements, physiological measurements, and / or the like. The measurements are described in more detail under the heading "Monitoring Of Adjusting A Surgical Parameter Based On Biomarker Measurements" in U.S. Patent Application No. US 17 / 156,28 filed on November 10, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0139] The indication of data may be sent, for example, to a surrogical hub 49040 where it can be evaluated. In an example, the indication may be sent to a cloud service (such as Amazon Web Services) as described herein. The data may be used as an input in an analysis module.

[0140] FIG. 9 shows an example of an overview of a conforming interconnection surgical system. As shown in FIG. 9, one or more surgical instruments may be used to perform a surgical task. For example, device A 49045 may be associated with a surgical instrument. Device B 49090 may be associated with a second surgical instrument. The surgical instrument can include one or more functions. For example, device A 49045 and / or device B 49090 may be associated with one or more functions (such as device A functions 1, 2, and 3 49050, and device B functions 1, 2, and 3 49085). One or more functions may be functions that are executed autonomously (e.g., during surgery). For example, the function of device A 49045 may be the orientation of device A 49045. When this function is activated, as shown in FIG. 9, the orientation of device A 49045 may be autonomously determined (e.g., controlled) by the system.

[0141] If the function is not activated, the function may be manually executed by the surgeon. The function associated with Device B 49090 may be a clamp. If this function is activated, the clamp for Device B 49090 may be executed autonomously. If this function is not activated, the clamp may be executed by the surgeon performing the surgical task. Enabling and disabling one or more functions associated with each of Device A 49045 and Device B 49090 may be dynamic. For example, while transitioning through stages during the execution of a surgical task, the surgical hub 49095 may disable and / or enable one or more of the functions associated with Device A 49045 and / or Device B 49090 based on the introduction or removal of a surgical instrument into or from the surgical OR, or based on the introduction of a new surgical system into the surgical OR.

[0142] Enabling and / or disabling may be determined based on metrics of the surgical instrument (e.g., Device A metric 49055 and Device B metric 49080) (e.g., how Device A 49045 executed its function, whether it is enabled and in autonomous mode, or disabled and in manual mode). In such cases, the metric may be generated based on one or more of the following sources that may be placed on the surgical instrument, namely, sensors, actuators, robotic metrics, patient biomarkers, force, surgeon biomarkers, surgical context, number of staff, change in the number of staff, etc. The patient and / or surgeon biomarkers may be associated with measurements obtained from wearable devices that the patient and / or surgeon may wear during the execution of the surgical task. The actuator may include one or more servo motors placed on the surgical instrument. The metric data associated with these classifications may be sent to the surgical hub 49095 via a message (e.g., a JSON message). The message may pass through the surgical application interface. The surgical hub 49095 may receive this data and may process the data via the surgical hub processor 49110.

[0143] One or more functions may be based on and / or enabled based on the available resources associated with the system. For example, the available resources may be stored in the resource management subsystem 49115 of the surgical hub. Device A 49045 and device B 49090 can use the resources from the resource management subsystem 49115 to execute one or more of their functions (e.g., autonomously). In the example, the surgical hub 49095 can determine which functions to enable for the surgical instrument based on the available resources. The surgical hub 49095 may query the resource management subsystem 49115 to determine the available resources (e.g., the total available resources), which can be described with respect to FIGS. 5 and 6. Determining which functions to enable may involve the use of the rules engine 49105. The rules engine 49105 may be disposed within the surgical hub 49095. In the example, the rules engine 49105 may be disposed on a third-party service. The functions may be collected by the surgical hub 49095 and used to generate a matrix (e.g., using the matrix generator 49100) as described with respect to FIG. 11.

[0144] For example, function 1 of device A 49045 may be associated with (e.g., used with) a specific amount of bandwidth and a specific amount of power. This information may be used by the surgical hub 49095 to generate a matrix, which may be a data structure that links the functions of each surgical device to the resources (e.g., resource ranges) required to execute that function. The rules engine 49105 may be tasked with evaluating different variations of allocating resources to functions and may optimize which functions to enable based on the evaluation in the surgical hub 49095. The functions determined to be enabled may be transmitted to the surgical instrument via a message. In the example, the resources and / or functions of the surgical instrument may be manually input by a medical staff member 49120.

[0145] Figure 10 shows an example of the relationship between a function set 49125, a surgical hub 49150, and resource management. The function set 49125 may be included in the system. The function set 49125 may be manually input by a user 49160 such as a surgeon or a medical staff member. In the example, the function set 49125 may be a learning set determined using a machine learning algorithm.

[0146] One or more functions may be associated with a surgical instrument. For example, as described with respect to FIG. 2, one or more functions may be functions that the surgical instrument may be capable of autonomously performing. When a function is enabled, this may propose that the surgical instrument autonomously perform this function, for example, clamp.

[0147] One or more functions may be associated with one or more resources 49145. For example, function A 49135 may be associated with resources 49145 A, B, and C. In the example, the function may be associated with a resource range 49140. The resource range 49140 may be a range within which the surgical instrument can autonomously perform a function (for example, the range may include values indicating the expected performance of the function, which may be based on past data and the surgical context). For example, a minimum resource and a maximum resource may exist. In the example, an optimal resource may exist.

[0148] One or more functions linked to resource 49145 and / or resource range 49140 may be sent to surgical hub 49150. Surgical hub 49150 can store and analyze such values within resource management subsystem 49155, which can include one or more different data structures for holding and querying functions, resource 49145, and resource range 49140. The functions to be activated can be adjusted based on the introduction or removal of one or more surgical instruments or surgical systems (e.g., by changing the level of autonomy associated with the function). A higher level of autonomy may be associated with more tasks of the function to be executed autonomously, and a lower level of autonomy may be associated with fewer tasks of the function to be executed autonomously.

[0149] In an example, a second surgical hub may be introduced into the surgical OR. The second surgical hub may provide, for example, more allowable resources 49145 used by surgical instruments that already exist within the surgical OR. In such a case, more functions may be activated due to the increased allowable resources 49145 as determined by the rules engine of surgical hub 49150.

[0150] The determination of whether to activate and / or deactivate one or more functions can be performed by a rules engine, as described with respect to FIG. 6. When an instrument (e.g., a new instrument) or system (e.g., a new system) is introduced, it can be aligned with surgical hub 49150. Surgical hub 49150 may configure the system or surgical instrument and may notify other surgical instruments and / or systems already present within the surgical OR. A surgical instrument may have identification information associated therewith, similar to a surgical system that may be assigned by surgical hub 49150 at the time of registration.

[0151] A device (e.g., a new device) and / or a system may request (e.g., send a request message) the surgical hub 49150 for permission to connect. The request message may indicate which resources 49145 the system (e.g., a new system) is available for use. A surgical instrument (e.g., a new surgical instrument) may send a request message for connection to the surgical hub 49150. The surgical instrument may indicate in the request message which functions it can perform autonomously, along with the number of resources 49145 for the functions.

[0152] Based on the current setup, the hub 49150 may determine whether to allow the surgical system and / or the new surgical instrument to connect. After connection, the surgical hub 49150 may decide whether to adjust which functions are enabled and / or disabled for the current system, and may send such a decision to the surgical instrument via a message. Based on receiving the message, the surgical instrument can follow the functions indicated by the surgical hub 49150 and enable and / or disable them.

[0153] Enabling and / or disabling functions can take into account performance metrics such as latency. The surgical hub 49150 may include an optimization module that can perform a cost analysis by enabling and / or disabling function variations when determining the optimal set of functions for the in - hand surgical task.

[0154] FIG. 11 shows an example of a matrix generator 49170 and function determination. As shown in FIG. 11, the function resource 49165 may be associated with a surgical instrument, such as Device A. The function resource 49165 may be associated with the matrix generated with respect to FIG. 9. For example, the function resource 49165 may show the generated matrix showing the link between the function of Device A and the resource (e.g., function 1 of Device A) as shown in FIG. 11. When the device is connected to the surgical hub, the device may send the function together with the resources associated with the device to the surgical hub as described with respect to FIG. 10. The resources may be resources used to autonomously execute the function of the surgical instrument. As shown in FIG. 11, function 1 of Device A may be associated with bandwidth X. The matrix generator 49170 can combine the resources of the secondary device introduced into the surgical hub, execute optimization techniques, and figure out which functions should be enabled to best execute the in - hand surgical task.

[0155] For example, Function 1 of Device A can be associated with bandwidth resource X (e.g., can be associated with bandwidth, power, payload effort, data packet size, transmission frequency, etc.). Matrix generator 49170 can input the bandwidth Z of input device B. The rule engine (e.g., obtaining this combination from the matrix generator) can determine the cost of activating Function 1 using this combination. In a second variant form, matrix generator 49170 can use the bandwidth of Function X together with another device having a bandwidth of T, and the rule engine can determine the cost associated with this combination. In the example, matrix generator 49170 can use the bandwidth of Function 1 of X to be combined with a newly introduced device with a bandwidth of L in the OR, and the rule engine can determine the cost of this combination. After evaluating these different variations, the surgical hub can determine (e.g., via the rule engine) whether to activate Function 1 with a particular tested combination of resources based on an analysis that can be included in the cost analysis module. When the surgical system is introduced into the OR, the bandwidth of Function 1 of X can be used in matrix generator 49170 together with the bandwidth (e.g., total bandwidth) of the surgical system to determine whether Function 1 can be activated.

[0156] As shown in FIG. 11, another resource that can be used to determine whether function 1, 2, or 3 is enabled can be power. The function 1 of device A can be kept constant, but the introduced devices associated with the power resource can be used in matrix generator 49170, the cost associated with each combination is determined, and it can be used to determine whether function 1 should be enabled. This technique can be used for functions 2 and 3. For example, to autonomously execute function 2 for device A, a bandwidth of N may be used. Matrix generator 49170 may combine different variations of the bandwidths of other devices and keep the bandwidth constant while determining whether function 2 should be enabled. After the function is determined by the surgical hub or third-party service, the surgical hub may send a message to each of the surgical instruments that enable the function and may also send a notification to the user (e.g., the surgeon). In the example, the surgical hub may send a message to the user instructing to manually disable one or more functions based on the introduction or removal of the surgical instrument or surgical system.

[0157] The cost analysis performed by the rule engine may include measures of the execution of surgical tasks. For example, the surgical hub may perform simulations (e.g., locally or remotely via a cloud service) using each of the combinations of resources generated by the matrix. The simulation framework may be described in "Method for Surgical Simulation" of U.S. Patent Application No. 17 / 332,593 filed on May 27, 2021, the disclosure of which is incorporated herein by reference in its entirety. Performance metrics may be collected based on the simulation. For example, CPI may be determined similar to latency and throughput. An overall score may be generated for each of the combinations, and the surgical hub may determine the maximum score and use the combination of functions that resulted in the maximum score for the surgical task simulation. In the example, determining which combination should be used may involve the rule engine assigning weights to performance metrics and other considerations associated with the surgical task. Determining the rules may be based on one or more of the surgical context, type of function, past data, or type of resource as described with respect to FIG. 12. For example, situation awareness may provide the system with additional context not related to the device (e.g., each device) that may enable the system to optimally adapt. Having awareness of the surgical context may provide an advantage for the system to be appropriately adjusted. Past data may indicate that the treatment risk is higher for a particular patient. Past data can include global and / or patient-specific information.

[0158] FIG. 12 shows a flowchart of a conforming interconnected surgical system. The surgical hub may be used for autonomous determination of surgical functions in a surgical environment. At 49175, the surgical hub may detect a first surgical device within the surgical environment (e.g., may include a processor configured to detect). The first surgical device may include a first plurality of functions. The surgical hub may include a plurality of resources.

[0159] At 49180, the surgical hub can detect a second surgical device within the surgical environment. The second surgical device may include a second plurality of functions.

[0160] At 49185, one or more functions from the first plurality of functions may be determined for the first surgical device to execute based on a plurality of resources. One or more functions from the second plurality of functions may be determined for the second surgical device to execute based on a plurality of resources.

[0161] In an example, the surgical hub can include a plurality of ports. The first surgical device may be connected to the surgical hub via a first port from the plurality of ports. The second surgical device may be connected to the surgical hub via a second port from the plurality of ports.

[0162] Determining one or more functions from the first plurality of functions for the first surgical device to execute and one or more functions from the second plurality of functions for the second surgical device to execute may include using an optimization (e.g., Newton's method) associated with one or more rules. The optimization associated with one or more rules may be based on one or more of a surgical context, a type of the first plurality of functions, a type of the second plurality of functions, past data, or a plurality of resources.

[0163] In an example, the surgical hub can determine a surgical context associated with the surgical environment. The surgical hub can determine one or more functions from the first plurality of functions for the first surgical device to execute and one or more functions from the second plurality of functions for the second surgical device to execute based on a plurality of resources and the surgical context.

[0164] The surgical hub may send instructions for one or more determined functions to the user of the first surgical device and the user of the second surgical device. The surgical hub can detect a third surgical device within the surgical environment. The third surgical device may include a third plurality of functions. The surgical hub may adjust one or more functions from the first plurality of functions for the first surgical device to perform and one or more functions from the second plurality of functions for the second surgical device to perform based on a plurality of resources and the third plurality of functions. One or more functions from the third plurality of functions for the third surgical device to perform may be determined based on the plurality of resources.

[0165] The surgical hub may determine one or more optional functions from the first plurality of functions for the first surgical device to perform and one or more optional functions from the second plurality of functions for the second surgical device to perform based on the plurality of resources. The optional functions may be selected by the user of the first surgical instrument and the user of the second surgical instrument. The surgical hub may determine one or more resources from the plurality of resources for the first surgical device and one or more resources from the plurality of resources for the second surgical device. The determined one or more resources may be used to perform determining one or more functions.

[0166] Figure 13 shows an example of an overview of the rules engine 49195. In the example, the rules engine 49195 may be included in a third-party system, and the surgical hub may access the third-party system (e.g., using a security key). The rules engine 49195 may be linked to a matrix generator as described with respect to FIG. 9. The rules engine may assign weights based on the importance of the functions 49200 of the surgical instrument that can be executed autonomously and on the resources associated with the surgical instrument. For example, function 1 described with respect to FIG. 11 may be given a higher weight compared to function 2. Function 2 described with respect to FIG. 11 of device A may be given a higher weight than function 3. The weights may be determined based on the surgical context and / or past data. The weights may be determined based on the results collected from the surgical simulation.

[0167] Weights may be assigned to performance metrics. Weights may be assigned to resources for each of the functions 49205. For example, in the case of function 1, as described with respect to FIG. 11, bandwidth may be assigned a higher weight compared to power. As described with respect to FIG. 11, power for function 2 of device A may be assigned a higher weight than bandwidth. This may depend on the type of function 49200 that is being executed autonomously. For example, power may be assigned a higher weight for executing orientation autonomously compared to the power used for executing a clamp autonomously.

[0168] Based on the weights, priorities 49210 may be assigned to the resources associated with each of the functions 49200. High and / or low priorities 49210 may be determined. Functions 49200 associated with high priority 49210 may be selected to be enabled more than functions 49200 associated with low priority 49210. When the surgical task 49220 is being executed, the weights may be dynamic based on the surgical context. The rules engine 49195 may determine updated weights for each of the functions 49200 of the surgical instrument. The weights may be updated based on a new surgical instrument or a new system introduced into the surgical operating room (OR).

[0169] An autonomous system interaction may be provided. The adaptation of autonomous options and / or processes may be based on the automatic identification and / or connection between the system and a secondary system. The adaptation of automated functional options can be based on the interconnection of the system. For example, a surgical hub may be connected to one or more secondary systems and, when establishing the connection, may adjust the operations that can be performed autonomously. The adjustment may include setting up, updating the operating system, specifying communication priority or processor load priority, and / or directly establishing communication between secondary systems (e.g., two secondary systems).

[0170] Self-regulatory communication establishment and / or registration may be provided. The surgical hub and / or instrument may be associated with setup and / or configuration upon startup or access to the smart network. The surgical hub and / or instrument setup and / or configuration upon startup or access to the smart network may be described herein. In an example, when a device requests and / or establishes a connection to a hub-defined network, the hub may define for the device the type of communication, frequency, system security level (e.g., cause the device to adjust its security level), flagging technology, software revision, etc. During this definition of the communication exchange, the hub may adjust the expected level of mutual communication to the situation it is monitoring based on procedures, network backlog / latency, capacity level, and / or procedures. The hub may define that multiple systems within its network communicate directly (e.g., using network bandwidth) and / or communicate through the hub (e.g., enabling the recording and adaptation of communication based on additional data), which may depend on the hub's capabilities and capacity for that time and specific procedures. For example, the hub may determine the mode of communication (e.g., best mode) between devices (e.g., two devices) and tell the devices how to communicate with each other. The hub may, for example, send flagging information to the device to check in with the hub based on a particular change. The hub may require the device to adjust the frequency at which the device operates, the magnitude at which the system communicates (e.g., in dB), and / or the protocol used by the device based on the noise and / or amount of data detected within the OR by the hub. The hub can adjust a number of system communication commands based on the introduction of the device, the priority of its data to other systems within the network, or the risk in its function, to provide the highest priority or highest risk to the data that uses the maximum capacity for the conversation. It may define the communication interaction between systems (e.g., two systems) within its system to adjust their communication.For example, the hub can detect that the signal of the device is decreasing, or that the next step of the surgery requires communicating an appropriate amount of data using a higher bandwidth. The hub can enable an antenna (e.g., an additional antenna) to attempt to increase the signal strength, or the hub can communicate to the device to increase the signal output strength.

[0171] Connecting to an available network and / or registering as a participant within a hub control matrix can be provided. Automatic identification (ID) and / or registration can be based on the detected aspects of the network and / or device. Unconnected connectable surgical devices can recognize the network surrounding them. Recognition can enable selective interrogation of possible connection options. The options can be compared to an existing table or hierarchical order of networks that should have a preference for connection. The device can announce its intention to connect, and the network can provide information about which of the networks are expecting its arrival and / or networks that the network does not want to interfere with or attempt to connect to. This can be based on the surgical step, the progress of the surgery, the instrument in communication (e.g., currently in communication), the arrival of duplicate or incompatible devices, and / or other situational uses. Automatic adaptation of system functions can be based on the parameters for establishing communication between systems. The parameters can be steps of the procedure. For example, when establishing a connection, the device can adapt the system if it occurs within a specific step in the surgery (e.g., only in that case).

[0172] Conformance can be verified using software and / or functional tests that automatically change and / or update the assembled configuration (e.g., after initial assembly). The software can update the configuration settings. Verification of the update may be used. The functional check of the configuration may include one or more of the following. This may include verification of the transducer function. This may include verification of torque. This may include the maximum joint angle value. This may include re - establishing the position and / or home of Device 0. Shipping issues can cause the device to deviate from calibration. For example, energy devices can be announced and / or detected by the hub. The hub can know the next steps and / or use cases for the devices announced in the procedure (e.g., based on factory - set configuration, surgeon selection / modification / machine, or clinical database learning). The hub can automatically send the configuration settings to the device for its activation.

[0173] Selective automatic establishment of communication between systems within the same operating room (OR) network may be provided, which may include distributed analysis, communication, and data verification by co-networked OR devices. For example, two-way communication of operating parameters between a device and a hub that controls the surgical plan and / or flow may include one or more of the following. A surgical stapling device may be pre-set at a cutting speed of 15 mm / second by default at the factory. The stapler may be energized within the OR and establish communication with the hub. The hub may be able to recognize the device. The hub may analyze a database and determine that the optimal speed for this surgery is 5 mm / second and the pre-compression delay is 20 seconds. The hub may communicate with the device and send an update of the cutting speed and compression time. The device may use the parameters and check that the values are within the operating boundaries of the device system. The device may reply to the hub that 5 mm / second is at the lower limit of the system motor performance, which may potentially result in insufficient staple line delivery and / or device performance degradation. The hub may obtain the updated information and determine the risk profile at a higher cutting speed. An acceptable increase may be determined and sent back to the device. The device may verify the parameters and confirm to the hub that it is ready. One or more of the techniques described herein may be performed autonomously. If an acceptable solution cannot be achieved, the hub may be able to warn the OR personnel. Based on the patient, device, and treatment risk levels, the system may be able to set the overall treatment risk level. Depending on the calculated risk level, the system may autonomously complete the task, or if the risk is considered too high, the system may wait to be confirmed by the surgeon before completing the surgical step. A mobile flexible endoscope robot may be rolled into the OR for transbronchial imaging and for part of the incision and / or mobilization.When the system recognizes that it is operating in proximity to patients of other smart or digital ecosystem devices (e.g., laparotomy retractors, staplers, energy devices, etc.), the system may establish communication with the device when controlling the system in proximity to other systems to assist in avoiding collision or unexpected tissue tension situations.

[0174] The autonomous identification of trackable and / or identifiable elements may be within the system. The automatic identification of a wearable system may include one or more of the following. This may include the automatic determination of the active user of the wearable device, which may include using previous data sets to determine the user and / or the automatic verification and / or confirmation of identification (e.g., using the fingerprint of a previous biomarker to verify the user). This can include the classification of user characteristics such as jobs (e.g., skill sets, ability levels, trained equipment, etc.), the determination of the use of monitored data, and / or the prioritization of monitored data relative to other data sources simultaneously processed by the system. The automatic definition of reporting frequencies, registrations, and / or data collection parameters. Instrument, tool, and medical stock identification may be provided and may include the detection of the manufacturer, model, and / or serial number. The automatic determination of system limitations or reach may include the systems permitted to be controlled by the hub, the tolerance for communication establishment, and / or the physical and / or functional boundaries for tracking capabilities.

[0175] The following is a numbered list of embodiments that may or may not be claimed. 1. A surgical hub for the autonomous determination of surgical device functions to be activated for a surgical procedure, the surgical hub comprising a processor, the processor comprising detecting a first surgical device within the surgical environment, the first surgical device comprising a first plurality of functions, the surgical hub comprising a plurality of resources, Detecting a second surgical device within the surgical environment, the second surgical device having a second plurality of functions; Receiving or deriving an indication of a surgical procedure to be performed, the surgical procedure including a plurality of surgical tasks; Determining one or more functions from a first plurality of functions to enable a first surgical device to perform one of the plurality of surgical tasks, and determining one or more functions from a second plurality of functions to enable a second surgical device to perform one of the plurality of surgical tasks, the determination of which functions to activate being based on a plurality of resources, a surgical hub configured to perform the determination.

[0176] The plurality of resources of the hub are finite, and the enabled instrument functions require the use of the hub's resources to perform one or more surgical tasks. The hub advantageously autonomously adjusts the allocation of finite hub resources to the instrument functions necessary to perform one or more tasks of the surgical procedure to enable those instrument functions.

[0177] The plurality of resources may comprise computer resources that may include power and bandwidth. The functions may be associated with a range of resources (e.g., a power and / or bandwidth range) within which a surgical instrument can autonomously perform a surgical task. In other words, the resources may be the computer processing requirements for the functions for autonomously performing a surgical task.

[0178] 2. The processor is further configured to Transmit an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device, or Transmit a signal to the first device and / or the second device to enable the determined one or more functions, the surgical hub according to Embodiment 1.

[0179] Advantageously, in addition to determining the functions to be enabled, the hub can send signals to the first device and the second device to enable the determined functions, thereby preparing the devices and their functions for the surgical procedure or a task of the surgical procedure (since the enabling of the functions can be dynamic based on the stage of the procedure). Alternatively, the hub can send a message to the user of the device indicating which functions should be enabled.

[0180] 3. The surgical hub according to embodiment 1 or embodiment 2, wherein the enabled function is a function for autonomously performing a surgical task.

[0181] The surgical hub is configured to determine which tasks should be autonomously performed by the instrument functions and which should be manually performed by the surgeon, and the resource allocation is a factor for determining which tasks should be autonomously or manually performed, and thus which functions should be enabled or disabled.

[0182] 4. The surgical hub according to any one of embodiments 1 to 3, wherein the function is a function executable by the device.

[0183] 5. The first device and / or the second device is an end cutter, and the first and / or second plurality of functions include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil jaw force, reload alignment slot management, according to the surgical hub of embodiment 4.

[0184] 6. The surgical hub according to any one of embodiments 1 to 5, wherein the processor is further configured to receive or determine instructions for the first plurality of functions and the second plurality of functions.

[0185] The user can manually input the functions of the device. Alternatively or additionally, the hub may be able to determine the functions by knowing the device. For example, the device may be connected to the hub and share an identifier that identifies the device, and through this identifier, the hub may be able to determine the functions of the device (e.g., via a lookup table stored in memory or via an external memory / server).

[0186] 7. The processor is configured to detect the first device and the second device when the first device and the second device send a request to connect to the hub, or when the first device and the second device are connected to the hub, for the surgical hub according to any one of Embodiments 1 to 6.

[0187] 8. The surgical hub includes a plurality of ports, the first surgical device is connected to the surgical hub via a first port from the plurality of ports, and the second surgical device is connected to the surgical hub via a second port from the plurality of ports, for the surgical hub according to any one of Embodiments 1 to 7.

[0188] 9. Determining which functions should be enabled includes using optimizations associated with one or more rules, for the surgical hub according to any one of Embodiments 1 to 8.

[0189] 10. The optimizations associated with one or more rules are based on at least one of a surgical context, a first plurality of function types, a second plurality of function types, past data, or a plurality of resources, for the surgical hub according to Embodiment 9.

[0190] Some functions may be given higher priority, for example, based on the complexity of the tasks they perform or based on the surgical outcomes of the surgical tasks when performed manually versus automatically (which can be based on historical data from past surgeries including the tasks performed by those functions). For example, the historical data may indicate that the surgical outcomes of the surgical tasks are improved when the tasks are autonomously performed by the functions, and thus, optimization may prioritize enabling that function to autonomously perform the surgical tasks. By optimizing in this way, the results of the surgical procedure can generally be improved.

[0191] As an example, the results of tissue resection can correspond to the integrity of the seal line, and a seal with less leakage is equivalent to a more positive result than a seal with more leakage. The results of the procedure can also correspond to the number of complications that occur during the surgery, which can include, for example, malfunction of the instrument, leakage of the seal line, misfiring of the staple line, etc.

[0192] 11. The processor is further configured to detect a third surgical device within the surgical environment, wherein the third surgical device comprises a third plurality of functions, and adjust one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device based on the plurality of resources and the third plurality of functions, a surgical hub according to any of embodiments 1 to 10.

[0193] Advantageously, the surgical hub can dynamically determine which functions of the devices should be enabled or disabled as the surgical procedure progresses and / or as devices are brought into the surgical environment and / or connected to the hub.

[0194] 12. The surgical hub according to embodiment 11, wherein the adjustment includes disabling a function that was previously enabled.

[0195] By dynamically enabling and disabling functions, it is possible to reallocate resources when a new device is detected or when a surgical procedure is in progress.

[0196] 13. The processor is further configured to detect a third surgical device within the surgical environment, the third surgical device comprising a third plurality of functions, and determine, based on a plurality of resources, one or more functions from the third plurality of functions to be executed by the third surgical device, the surgical hub according to any one of Embodiments 1 to 12.

[0197] 14. The processor is further configured to determine, based on a plurality of resources, one or more optional functions from the first plurality of functions to be executed by the first surgical device and one or more optional functions from the second plurality of functions to be executed by the second surgical device, the surgical hub according to any one of Embodiments 1 to 13.

[0198] 15. The surgical hub of Embodiment 14, wherein the optional function is selected by the user of the first surgical instrument and the user of the second surgical instrument.

[0199] 16. A method for autonomous determination of surgical device functions enabled for a surgical procedure, the method comprising detecting a first surgical device within the surgical environment, the first surgical device comprising a first plurality of functions and the surgical hub comprising a plurality of resources, and detecting a second surgical device within the surgical environment, the second surgical device comprising a second plurality of functions, and receiving or deriving an indication of the surgical procedure to be performed, the surgical procedure comprising a plurality of surgical tasks, and To enable a first surgical device to perform one of a plurality of surgical tasks, determining one or more functions from a first plurality of functions, and to enable a second surgical device to perform one of a plurality of surgical tasks, determining one or more functions from a second plurality of functions, the determining being based on a plurality of resources, and determining.

[0200] The plurality of resources of the hub are finite, and the enabled instrument functions require the use of the hub's resources to perform one or more surgical tasks. The method advantageously autonomously adjusts the allocation of finite hub resources to the instrument functions required to perform one or more tasks of a surgical procedure and enables those instrument functions.

[0201] The plurality of resources may comprise computer resources that may include power and bandwidth. The functions may be associated with a resource range (e.g., a power and / or bandwidth range) within which the surgical instrument can autonomously perform a surgical task. In other words, the resources may be the computer processing requirements for the functions for autonomously performing a surgical task.

[0202] 17. Transmitting an indication of the determined one or more functions to the user of the first surgical device and the user of the second surgical device, or Transmitting a signal to the first device and / or the second device to enable the determined one or more functions. The method according to embodiment 16 further includes.

[0203] Advantageously, in addition to determining the function to be enabled, the method includes transmitting signals to the first device and the second device to enable the determined function, thereby preparing the devices and their functions for a surgical procedure or a task of that surgical procedure (since the enabling of the function can be dynamic based on the stage of the procedure). Alternatively, the method includes transmitting a message to the user of the device indicating which function is to be enabled.

[0204] 18. The method according to embodiment 16 or embodiment 17, wherein the enabled function is a function for autonomously performing a surgical task.

[0205] The method is configured to determine which tasks should be autonomously performed by the instrument function and which should be manually performed by the surgeon, and the resource allocation is a factor for determining which tasks should be autonomously or manually performed, and thus which functions should be enabled or disabled.

[0206] 19. The method according to any one of embodiments 16 to 18, wherein the function is a function that the device can perform.

[0207] 20. The first device and / or the second device is an end cutter, and the plurality of first and / or second functions include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil jaw force, reload alignment slot management, according to the method of embodiment 19.

[0208] 21. The method according to any one of embodiments 16 to 20, further comprising receiving or determining an indication of the plurality of first functions and the plurality of second functions.

[0209] 22. Detecting the first device and the second device includes detecting when the first device and the second device send a request to connect to the hub, or detecting when the first device and the second device are coupled to the hub, the method according to any one of embodiments 16 to 21.

[0210] 23. The surgical hub has a plurality of ports, the first surgical device is connected to the surgical hub via a first port from the plurality of ports, and the second surgical device is connected to the surgical hub via a second port from the plurality of ports, the method according to any one of embodiments 16 to 22.

[0211] 24. Determining which functions should be enabled includes using optimizations associated with one or more rules, the method according to any one of embodiments 16 to 23.

[0212] 25. The optimizations associated with one or more rules are based on at least one of a surgical context, a first plurality of function types, a second plurality of function types, past data, or a plurality of resources, the method according to embodiment 24.

[0213] Some functions may be given a higher priority, for example, based on the complexity of the tasks performed by those functions, or based on the surgical results of surgical tasks when performed manually or automatically (which can be based on past data from past surgeries including that task performed by that function). For example, the past data may indicate that the surgical results of the surgical task are improved when the task is autonomously performed by the function, and thus the optimization may prioritize enabling that function to autonomously perform the surgical task. By optimizing in this way, the results of the surgical procedure can generally be improved.

[0214] As an example, the results of tissue resection can correspond to the integrity of the seal line, and a seal with less leakage is equivalent to a more positive result than a seal with more leakage. The results of the procedure can also correspond to the number of intraoperative complications that can include, for example, device malfunction, leakage of the seal line, misfiring of the staple line, and the like.

[0215] 26. Detecting a third surgical device within the surgical environment, the third surgical device having a third plurality of functions, and Based on a plurality of resources and the third plurality of functions, adjusting one or more functions from a first plurality of functions for execution by the first surgical device and one or more functions from a second plurality of functions for execution by the second surgical device; The method according to any one of Embodiments 16 to 25, further comprising:

[0216] Advantageously, the method includes dynamically determining which functions of the device should be enabled or disabled as the surgery progresses and / or as the device is brought into the surgical environment and / or connected to the hub.

[0217] 27. The method according to Embodiment 26, wherein the adjustment includes disabling a previously enabled function.

[0218] By dynamically enabling and disabling functions, it is possible to reallocate resources when a new device is detected or as the surgery progresses.

[0219] 28. Detecting a third surgical device within the surgical environment, the third surgical device having a third plurality of functions, and Based on a plurality of resources, determining one or more functions from a third plurality of functions for execution by the third surgical device; The method according to any one of Embodiments 16 to 27, further comprising:

[0220] 29. Based on a plurality of resources, further comprising determining one or more optional functions from a first plurality of functions for a first surgical device to perform, and one or more optional functions from a second plurality of functions for a second surgical device to perform, the method according to any one of Embodiments 1 to 28.

[0221] 30. The method according to Embodiment 29, wherein the optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0222] 31. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of Embodiments 16 to 30.

[0223] 32. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of Embodiments 16 to 30.

[0224] The following are numbered aspects of the present disclosure, which may or may not be claimed. 1. A surgical hub for autonomous determination of surgical functions in a surgical environment, the surgical hub comprising a processor, the processor being configured to detect a first surgical device within the surgical environment, the first surgical device comprising a first plurality of functions, the surgical hub comprising a plurality of resources; detect a second surgical device within the surgical environment, the second surgical device comprising a second plurality of functions; and determine one or more functions from a first plurality of functions for the first surgical device to perform and one or more functions from a second plurality of functions for the second surgical device to perform based on the plurality of resources.

[0225] 2. The surgical hub includes a plurality of ports, the first surgical device is connected to the surgical hub via a first port from the plurality of ports, and the second surgical device is connected to the surgical hub via a second port from the plurality of ports, the surgical hub according to aspect 1.

[0226] 3. Determining one or more functions from a first plurality of functions for the first surgical device to perform and one or more functions from a second plurality of functions for the second surgical device to perform includes using optimizations associated with one or more rules, the surgical hub according to aspect 1.

[0227] 4. The optimizations associated with one or more rules are based on at least one of a surgical context, a type of the first plurality of functions, a type of the second plurality of functions, past data, or a plurality of resources, the surgical hub according to aspect 3.

[0228] 5. The processor is further configured to determine a surgical context associated with the surgical environment, and determine one or more functions from a first plurality of functions for the first surgical device to perform and one or more functions from a second plurality of functions for the second surgical device to perform based on the plurality of resources and the surgical context, the surgical hub according to aspect 1.

[0229] 6. The processor is further configured to send instructions for the determined one or more functions to the user of the first surgical device and the user of the second surgical device, the surgical hub according to aspect 1.

[0230] 7. The processor is detecting a third surgical device within the surgical environment, the third surgical device having a third plurality of functions, detecting Based on a plurality of resources and a third plurality of functions, adjusting one or more functions from a first plurality of functions for execution by a first surgical device and one or more functions from a second plurality of functions for execution by a second surgical device, the surgical hub according to aspect 1, further configured to perform.

[0231] 8. The processor is detecting a third surgical device in the surgical environment, the third surgical device having a third plurality of functions, detecting; Based on a plurality of resources, determining one or more functions from a third plurality of functions for execution by the third surgical device, the surgical hub according to aspect 2, further configured to perform.

[0232] 9. The processor is Based on a plurality of resources, further configured to determine one or more optional functions from a first plurality of functions for execution by the first surgical device and one or more optional functions from a second plurality of functions for execution by the second surgical device, the surgical hub according to aspect 1.

[0233] 10. The optional functions are selected by a user of the first surgical instrument and a user of the second surgical instrument, the surgical hub according to aspect 9.

[0234] 11. The processor is determining one or more resources from a plurality of resources for the first surgical device and one or more resources from a plurality of resources for the second surgical device, the determined one or more resources being used to perform determining one or more functions, the surgical hub according to aspect 1, further configured to determine.

[0235] 12. A method for autonomous determination of surgical functions in a surgical environment, the method comprising Detecting a first surgical device within a surgical environment, the first surgical device having a first plurality of functions and the surgical hub having a plurality of resources; Detecting a second surgical device within the surgical environment, the second surgical device having a second plurality of functions; Determining, based on the plurality of resources, one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device, the method comprising:

[0236] 13. The surgical hub comprises a plurality of ports, the first surgical device is coupled to the surgical hub via a first port from the plurality of ports, and the second surgical device is coupled to the surgical hub via a second port from the plurality of ports, the method according to aspect 12.

[0237] 14. Determining one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device comprises using optimization associated with one or more rules, the method according to aspect 12.

[0238] 15. The optimization associated with one or more rules is based on at least one of a surgical context, a type of the first plurality of functions, a type of the second plurality of functions, past data, or the plurality of resources, the method according to aspect 14.

[0239] 16. Determining a surgical context associated with the surgical environment; Further comprising determining, based on the plurality of resources and the surgical context, one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device, the method according to aspect 12.

[0240] 17. The method according to aspect 12, further comprising transmitting an instruction of one or more determined functions to a user of a first surgical device and a user of a second surgical device.

[0241] 18. Detecting a third surgical device within the surgical environment, wherein the third surgical device has a third plurality of functions, and Based on a plurality of resources and the third plurality of functions, adjusting one or more functions from a first plurality of functions for execution by the first surgical device and one or more functions from a second plurality of functions for execution by the second surgical device. The method according to aspect 12, further comprising.

[0242] 19. Detecting a third surgical device within the surgical environment, wherein the third surgical device has a third plurality of functions, and Based on a plurality of resources, determining one or more functions from a third plurality of functions for execution by the third surgical device. The method according to aspect 13, further comprising.

[0243] 20. Based on a plurality of resources, further comprising determining one or more optional functions from a first plurality of functions for execution by the first surgical device and one or more optional functions from a second plurality of functions for execution by the second surgical device. The method according to aspect 12, further comprising.

[0244] 21. The method according to aspect 20, wherein the optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0245] 〔Embodiment〕 (1) A surgical hub for autonomous determination of surgical device functions activated for a surgical procedure, wherein the surgical hub Comprises a processor, and the processor Detecting a first surgical device within the surgical environment, wherein the first surgical device has a first plurality of functions and the surgical hub has a plurality of resources; Detecting a second surgical device within the surgical environment, wherein the second surgical device has a second plurality of functions; Receiving or deriving an instruction for the surgical procedure to be performed, wherein the surgical procedure includes a plurality of surgical tasks; Determining one or more functions from the first plurality of functions to enable the first surgical device to perform one of the plurality of surgical tasks, and determining one or more functions from the second plurality of functions to enable the second surgical device to perform one of the plurality of surgical tasks, wherein the determination of which functions to activate is based on the plurality of resources, and a surgical hub configured to perform the foregoing. (2) The processor is configured to: Transmit an instruction for the determined one or more functions to the user of the first surgical device and the user of the second surgical device, or The surgical hub according to Embodiment 1, further configured to transmit a signal to the first device and / or the second device to activate the determined one or more functions. (3) The surgical hub according to Embodiment 1 or 2, wherein the activated function is a function for autonomously performing the surgical task. (4) The surgical hub according to any one of Embodiments 1 to 3, wherein the function is a function that the device can perform. (5) The first device and / or the second device is an end cutter, and the first plurality of functions and / or the second plurality of functions include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil joe force, and reload alignment slot management. The surgical hub according to Embodiment 4.

[0246] (6) The surgical hub according to any one of Embodiments 1 to 5, wherein the processor is further configured to receive or determine instructions for the first plurality of functions and the second plurality of functions. (7) The surgical hub according to any one of Embodiments 1 to 6, wherein the processor is configured to detect the first device and the second device when the first device and the second device send a request to couple to the hub or when the first device and the second device couple to the hub. (8) The surgical hub according to any one of Embodiments 1 to 7, wherein the surgical hub includes a plurality of ports, the first surgical device is connected to the surgical hub via a first port from the plurality of ports, and the second surgical device is connected to the surgical hub via a second port from the plurality of ports. (9) The surgical hub according to any one of Embodiments 1 to 8, wherein determining which function should be activated includes using optimization associated with one or more rules. (10) The surgical hub according to Embodiment 9, wherein the optimization associated with one or more rules is based on at least one of a surgical context, a type of the first plurality of functions, a type of the second plurality of functions, past data, or the plurality of resources.

[0247] (11) The processor is detecting a third surgical device within the surgical environment, the third surgical device comprising a third plurality of functions, and Based on the plurality of resources and the third plurality of functions, adjusting one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device, and further configured to perform, the surgical hub according to any one of Embodiments 1 to 10. (12) The surgical hub according to Embodiment 11, wherein the adjustment includes disabling a previously enabled function. (13) The processor detecting a third surgical device within the surgical environment, the third surgical device having a third plurality of functions; and further configured to determine one or more functions from the third plurality of functions for execution by the third surgical device based on the plurality of resources, the surgical hub according to any one of Embodiments 1 to 12. (14) The processor and further configured to determine one or more optional functions from the first plurality of functions for execution by the first surgical device and one or more optional functions from the second plurality of functions for execution by the second surgical device based on the plurality of resources, the surgical hub according to any one of Embodiments 1 to 13. (15) The surgical hub according to Embodiment 14, wherein the optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0248] (16) A method for autonomous determination of surgical device functions enabled for a surgical procedure, the method comprising: detecting a first surgical device within the surgical environment, the first surgical device having a first plurality of functions and the surgical hub having a plurality of resources; Detecting a second surgical device within the surgical environment, wherein the second surgical device comprises a second plurality of functions; Receiving or deriving an instruction for the surgical operation to be performed, wherein the surgical operation includes a plurality of surgical tasks; Determining one or more functions from the first plurality of functions to enable the first surgical device to perform one of the plurality of surgical tasks, and determining one or more functions from the second plurality of functions to enable the second surgical device to perform one of the plurality of surgical tasks, wherein the determination is based on the plurality of resources; (17) Transmitting an instruction for the determined one or more functions to the user of the first surgical device and the user of the second surgical device, or Further comprising transmitting a signal to the first device and / or the second device to activate the determined one or more functions, the method according to embodiment 16; (18) The activated function is a function for autonomously performing the surgical task, the method according to embodiment 16 or embodiment 17; (19) The function is a function that the device can perform, the method according to any one of embodiments 16 to 18; (20) The first device and / or the second device is an end cutter, and the first plurality of functions and / or the second plurality of functions include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil jo force, reload alignment slot management, the method according to embodiment 19;

[0249] (21) Further comprising receiving or determining an instruction for the first plurality of functions and the second plurality of functions, the method according to any one of embodiments 16 to 20; (22) Detecting the first device and the second device includes detecting when the first device and the second device send a request to couple to the hub, or detecting when the first device and the second device are coupled to the hub, according to any of embodiments 16 to 21. (23) The surgical hub includes a plurality of ports. The first surgical device is coupled to the surgical hub via a first port from the plurality of ports, and the second surgical device is coupled to the surgical hub via a second port from the plurality of ports, according to any of embodiments 16 to 22. (24) Determining which functions should be enabled includes using optimizations associated with one or more rules, according to any of embodiments 16 to 23. (25) The optimization associated with one or more rules is based on at least one of a surgical context, the type of the first plurality of functions, the type of the second plurality of functions, past data, or the plurality of resources, according to the method of embodiment 24.

[0250] (26) Detecting a third surgical device within the surgical environment, the third surgical device having a third plurality of functions, and adjusting, based on the plurality of resources and the third plurality of functions, one or more functions from the first plurality of functions for the first surgical device to perform and one or more functions from the second plurality of functions for the second surgical device to perform, according to any of embodiments 16 to 25. (27) The adjustment includes disabling a previously enabled function, according to the method of embodiment 26. (28) Detecting a third surgical device within the surgical environment, the third surgical device having a third plurality of functions, and determining, based on the plurality of resources, one or more functions from the third plurality of functions for execution by the third surgical device, the method according to any of embodiments 16 to 27, further comprising. (29) The method according to any of embodiments 1 to 28, further comprising determining one or more optional functions from the first plurality of functions for execution by the first surgical device and one or more optional functions from the second plurality of functions for execution by the second surgical device, based on the plurality of resources. (30) The method according to embodiment 29, wherein the optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0251] (31) A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to any of embodiments 16 to 30. (32) A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any of embodiments 16 to 30.

Claims

1. A surgical hub for autonomous determination of a surgical device function enabled for surgery, the surgical hub comprising a processor, the processor being configured to detect a first surgical device within the surgical environment, the first surgical device comprising a first plurality of functions, the surgical hub comprising a plurality of resources; detect a second surgical device within the surgical environment, the second surgical device comprising a second plurality of functions; receive or derive an indication of the surgery to be performed, the surgery comprising a plurality of surgical tasks; determine, based on the plurality of resources, one or more functions from the first plurality of functions to enable the first surgical device to perform one of the plurality of surgical tasks, and one or more functions from the second plurality of functions to enable the second surgical device to perform one of the plurality of surgical tasks, the determination of which functions to enable being based on the plurality of resources.

2. The processor is further configured to transmit an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device, or transmit a signal to the first device and / or the second device to enable the determined one or more functions.

3. The enabled function is a function for autonomously performing the surgical task.

4. The function is a function that the device is capable of performing.

5. The first device and / or the second device is an end cutter, and the first plurality of functions and / or the second plurality of functions include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil jaw force, and reload alignment slot management. The surgical hub according to claim 4.

6. The surgical hub according to claim 1, wherein the processor is further configured to receive or determine instructions for the first plurality of functions and the second plurality of functions.

7. The surgical hub according to claim 1, wherein the processor is configured to detect the first device and the second device when the first device and the second device send a request to couple to the hub, or when the first device and the second device couple to the hub.

8. The surgical hub includes a plurality of ports. The first surgical device is coupled to the surgical hub via a first port from the plurality of ports, and the second surgical device is coupled to the surgical hub via a second port from the plurality of ports. The surgical hub according to claim 1.

9. Determining which functions should be activated includes using optimizations associated with one or more rules. The surgical hub according to claim 1.

10. The surgical hub according to claim 9, wherein the optimizations associated with one or more rules are based on at least one of a surgical context, the type of the first plurality of functions, the type of the second plurality of functions, past data, or the plurality of resources.

11. The processor is Detecting a third surgical device within the surgical environment, the third surgical device comprising a third plurality of functions; Based on the plurality of resources and the third plurality of functions, adjusting one or more functions from the first plurality of functions for execution by the first surgical device and one or more functions from the second plurality of functions for execution by the second surgical device; The surgical hub according to claim 1, further configured to perform. **Claim 12** The adjustment includes invalidating a previously enabled function; The surgical hub according to claim 11. **Claim 13** The processor is Detecting a third surgical device within the surgical environment, the third surgical device comprising a third plurality of functions; Based on the plurality of resources, determining one or more functions from the third plurality of functions for execution by the third surgical device; The surgical hub according to claim 1, further configured to perform. **Claim 14** The processor is Based on the plurality of resources, further configured to determine one or more optional functions from the first plurality of functions for execution by the first surgical device and one or more optional functions from the second plurality of functions for execution by the second surgical device; The surgical hub according to claim 1. **Claim 15** The optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument; The surgical hub according to claim 14. **Claim 16** A method for autonomous determination of surgical device functions enabled for a surgical procedure, the method comprising Detecting a first surgical device within the surgical environment, wherein the first surgical device comprises a first plurality of functions and the surgical hub comprises a plurality of resources; Detecting a second surgical device within the surgical environment, wherein the second surgical device comprises a second plurality of functions; Receiving or deriving an instruction for the surgical procedure to be performed, wherein the surgical procedure comprises a plurality of surgical tasks; Determining one or more functions from the first plurality of functions to enable the first surgical device to perform one of the plurality of surgical tasks, and determining one or more functions from the second plurality of functions to enable the second surgical device to perform one of the plurality of surgical tasks, wherein the determination is based on the plurality of resources. A method comprising:

17. Transmitting an instruction for the determined one or more functions to a user of the first surgical device and a user of the second surgical device, or Further comprising transmitting a signal to the first device and / or the second device to activate the determined one or more functions. The method according to claim 16.

18. The method according to claim 16 or claim 17, wherein the activated function is a function for autonomously performing the surgical task.

19. The method according to claim 16, wherein the function is a function that the device can perform.

20. The first device and / or the second device is an end cutter, and the first plurality of functions and / or the second plurality of functions include one or more of control of an energy source, cutting, stapling, knob orientation, body orientation, body position, clamping, anvil jaw force, reload alignment slot management. The method according to claim 19.

21. The method according to claim 16, further comprising receiving or determining an instruction for the first plurality of functions and the second plurality of functions.

22. Detecting the first device and the second device includes detecting when the first device and the second device send a request to be coupled to the hub, or detecting when the first device and the second device are coupled to the hub. The method according to claim 16.

23. The surgical hub includes a plurality of ports, the first surgical device is coupled to the surgical hub via a first port from the plurality of ports, and the second surgical device is coupled to the surgical hub via a second port from the plurality of ports. The method according to claim 16.

24. Determining which function should be activated includes using optimization associated with one or more rules. The method according to claim 16.

25. The optimization associated with one or more rules is based on at least one of a surgical context, a type of the first plurality of functions, a type of the second plurality of functions, past data, or the plurality of resources. The method according to claim 24.

26. Detecting a third surgical device within the surgical environment, the third surgical device having a third plurality of functions, and Adjusting the one or more functions from the first plurality of functions for execution by the first surgical device and the one or more functions from the second plurality of functions for execution by the second surgical device based on the plurality of resources and the third plurality of functions. The method according to claim 16, further comprising.

27. The adjustment includes deactivating a previously activated function. The method according to claim 26.

28. Detecting a third surgical device within the surgical environment, wherein the third surgical device comprises a plurality of third functions; Further comprising determining, based on the plurality of resources, one or more functions from the plurality of third functions to be performed by the third surgical device, the method according to claim 16.

29. Further comprising determining, based on the plurality of resources, one or more optional functions from the plurality of first functions to be performed by the first surgical device and one or more optional functions from the plurality of second functions to be performed by the second surgical device, the method according to claim 16.

30. The method according to claim 29, wherein the optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument.

31. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to claim 16.

32. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to claim 16.

Images