Adaptive autonomous functionality and system interconnection

Abstract

Systems, methods, and instruments for adaptive autonomous functionality and system interconnection are described herein. The surgical hub may detect a first surgical device in a surgical environment. The first surgical device may include a first plurality of functions. The surgical hub may include a plurality of resources. The surgical hub may detect a second surgical device in the surgical environment. The second surgical device may include a second plurality of functions. The surgical hub may determine, based on the plurality of resources, one or more of the first plurality of functions for execution by the first surgical device and one or more of the second plurality of functions for execution by the second surgical device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is related to the following concurrently filed patent applications, the contents of each of which are incorporated herein by reference:

[0003] U.S. Patent Application Titled “METHOD OF CONTROLLING AUTONOMOUSOPERATIONS IN A SURGICAL SYSTEM,” Attorney Docket No. END9430USNP1.

[0004] U.S. Patent Application Titled “AUTONOMOUS ADAPTATION OF SURGICALDEVICE ALGORITHM,” Attorney Docket No. END9430USNP5. Background Art

[0005] Surgical procedures are typically performed in a surgical suite or operating room at a medical facility, such as, for example, a hospital. Various surgical devices and systems are utilized in performing surgical procedures. In the digital and information age, medical systems and facilities are often slow to implement systems or procedures that utilize newer and improved technologies due to patient safety and a general desire to maintain traditional practices. Summary of the invention

[0006] Systems, methods and apparatus for adaptive autonomous functions and system interconnection are described herein. A surgical hub may detect a first surgical device in a surgical environment. The first surgical device may include a first plurality of functions. The surgical hub may include a plurality of resources. The surgical hub may detect a second surgical device in the surgical environment. The second surgical device may include a second plurality of functions. The surgical hub may determine one or more functions of the first plurality of functions for the first surgical device to perform and one or more functions of the second plurality of functions for the second surgical device to perform based on the plurality of resources.

[0007] A surgical hub for autonomously determining surgical device functions to be enabled for a surgical procedure is described. The surgical hub includes a processor. The processor is configured to be able to detect a first surgical device in a surgical environment. The first surgical device includes a first plurality of functions. The surgical hub includes a plurality of resources. The processor is further configured to be able to: detect a second surgical device in the surgical environment, wherein the second surgical device includes a second plurality of functions. The processor is further configured to be able to: receive or derive an indication of the surgical procedure to be performed, the surgical procedure including a plurality of surgical tasks. The processor is further configured to be able to: determine one or more functions of the first plurality of functions of the first surgical device to be enabled to perform a surgical task in the plurality of surgical tasks, and determine one or more functions of the second plurality of functions of the second surgical device to be enabled to perform a surgical task in the plurality of surgical tasks. The determination of which functions to enable is based on the plurality of resources. The plurality of resources of the hub are limited, and the enabled instrument functions require the use of the hub's resources in order to perform one or more surgical tasks. The hub advantageously autonomously coordinates the allocation of limited hub resources to instrument functions required to perform one or more tasks of the surgical procedure to enable those instrument functions. The plurality of resources may include computer resources, which may include power and bandwidth. Functionality may be associated with a resource range (e.g., a power and / or bandwidth range) under which the surgical instrument is able to autonomously perform the surgical task. In other words, a resource may be a computer processing requirement for a function to autonomously perform the surgical task.

[0008] The processor is further configured to be able to: send an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device; or transmit a signal to the first device and / or the second device to enable the determined one or more functions. Advantageously, in addition to determining the functions to be enabled, the hub is also capable of transmitting a signal to the first device and the second device to enable the determined functions, thereby preparing the devices and their functions for the surgical procedure or the task of the surgical procedure (because the enabling of functions can be done dynamically based on the stage in the surgery). Alternatively, the hub can transmit a message to the user of the device indicating which functions to enable.

[0009] The enabled functions may be functions for autonomously performing surgical tasks. The surgical hub may be configured to be able to determine which tasks are to be performed autonomously by instrument functions and which tasks are to be performed manually by the surgeon, wherein the allocation of resources is a factor used to determine which tasks should be performed autonomously or manually and therefore which functions should be enabled or disabled.

[0010] These functions may be functions that the device is capable of performing.

[0011] The first device and / or the second device can be an endoscopic cutting and closing device, and the first plurality of functions and / or the second plurality of functions may include one or more of the following: controlling energy source, cutting, suturing, knob orientation, body orientation, body positioning, clamping, anvil jaw force, and reloading alignment slot management.

[0012] The processor may be further configured to: receive or determine an indication of the first plurality of functions and the second plurality of functions. A user may manually input the function of the device. Alternatively or in addition, the hub may be able to determine the function by knowing the device. For example, a device may be coupled to the hub and share an identifier that identifies the device, and from the identifier, the hub may be able to determine the function of the device (e.g., via a lookup table stored in a memory or via an external memory / server).

[0013] The processor may be further configured to detect the first device and the second device when the first device and the second device send a request to couple to the hub, or when the first device and the second device are coupled to the hub.

[0014] The surgical hub may include a plurality of ports. A first surgical device may be connected to the surgical hub via a first port of the plurality of ports. A second surgical device may be connected to the surgical hub via a second port of the plurality of ports.

[0015] Determining which features should be enabled may include using optimizations associated with one or more rules.

[0016] The optimization associated with one or more rules may be based on at least one of the following: surgical context, the type of the first plurality of functions, the type of the second plurality of functions, historical data, or the plurality of resources. Some functions may be given higher priority, for example, based on the complexity of the tasks performed by those functions or based on the surgical results of the surgical tasks when performed manually and automatically (this may be based on historical data from past operations including that task performed by that function). For example, historical data may show that the surgical results of the surgical task are improved when the function performs the task autonomously, and therefore, the optimization may give priority to enabling the function to perform the surgical task autonomously. By optimizing in this way, the results of the surgical operation can generally be improved. For example, the results of tissue resection may correspond to the integrity of the sealing line, where less sealing leakage is equivalent to a more positive result compared to more sealing leakage. The results of the operation also correspond to the number of difficulties that occurred during the surgical operation, which may include, for example, instrument failure, sealing line leakage, nail line misfire, etc.

[0017] The processor may be further configured to detect a third surgical device in the surgical environment. The third surgical device may include a third plurality of functions. The processor may be further configured to adjust the one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform based on the plurality of resources and the third plurality of functions. Advantageously, the surgical hub may dynamically determine which functions of a device should be enabled or disabled while a surgical procedure is being performed and / or when a device is brought into the surgical environment and / or connected to the hub.

[0018] The adjustment may include disabling previously enabled functions. By dynamically enabling and disabling functions, it is possible to reallocate resources when a new device is detected or while a surgical procedure is ongoing.

[0019] The processor may be further configured to detect a third surgical device in the surgical environment. The third surgical device may include a third plurality of functions. The processor may be further configured to determine one or more functions of the third plurality of functions for the third surgical device to perform based on the plurality of resources.

[0020] The processor may be further configured to determine, based on the plurality of resources, one or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform.

[0021] The optional functions may be selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0022] A method for autonomously determining surgical device functions to be enabled for a surgical procedure is described. The method includes detecting a first surgical device in a surgical environment. The first surgical device includes a first plurality of functions. The surgical hub includes a plurality of resources. The method also includes: detecting a second surgical device in the surgical environment. The second surgical device includes a second plurality of functions. The method also includes: receiving or deriving an indication of the surgical procedure to be performed, the surgical procedure including a plurality of surgical tasks. The method also includes: determining one or more functions of the first plurality of functions of the first surgical device to be enabled to perform a surgical task in the plurality of surgical tasks, and determining one or more functions of the second plurality of functions of the second surgical device to be enabled to perform a surgical task in the plurality of surgical tasks, wherein the determination is based on the plurality of resources. The plurality of resources of the hub may be limited, and the enabled instrument functions require the use of the resources of the hub in order to perform one or more surgical tasks. The method advantageously autonomously coordinates the allocation of limited hub resources to instrument functions required to perform one or more tasks of the surgical procedure to enable those instrument functions. The plurality of resources may include computer resources, which may include power and bandwidth. Functionality may be associated with a resource range (eg, power and / or bandwidth range) within which the surgical instrument is able to autonomously perform a surgical task. In other words, a resource may be a computer processing requirement for a function to autonomously perform a surgical task.

[0023] The method may also include: sending an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device; or transmitting a signal to the first device and / or the second device to enable the determined one or more functions. Advantageously, in addition to determining the functions to be enabled, the method also includes transmitting a signal to the first device and the second device to enable the determined functions, thereby preparing the devices and their functions for the surgical procedure or the task of the surgical procedure (because the enabling of functions can be done dynamically based on the stage in the surgery). Alternatively, the method includes transmitting a message to the user of the device indicating which functions are to be enabled.

[0024] The enabled function may be a function for autonomously performing a surgical task. The method is configured to be able to determine which tasks are to be performed autonomously by the instrument function and which tasks are to be performed manually by the surgeon, wherein the allocation of resources is a factor used to determine which tasks should be performed autonomously or manually and therefore which functions should be enabled or disabled.

[0025] These functions may be functions that the device is capable of performing.

[0026] The first device and / or the second device can be an endoscopic cutting and closing device, and the multiple first functions and / or the multiple second functions may include one or more of the following: controlling energy source, cutting, suturing, knob orientation, body orientation, body positioning, clamping, anvil jaw force, and reloading alignment slot management.

[0027] The method may also include receiving or determining an indication of the first plurality of functions and the second plurality of functions.

[0028] Detecting the first device and the second device may include detecting when the first device and the second device send a request to couple to the hub, or detecting when the first device and the second device are already coupled to the hub.

[0029] The surgical hub may include a plurality of ports. A first surgical device may be connected to the surgical hub via a first port of the plurality of ports. A second surgical device may be connected to the surgical hub via a second port of the plurality of ports.

[0030] Determining which features should be enabled may include using optimizations associated with one or more rules.

[0031] The optimization associated with one or more rules may be based on at least one of the following: surgical context, the type of the first plurality of functions, the type of the second plurality of functions, historical data, or the plurality of resources. Some functions may be given higher priority, for example, based on the complexity of the tasks performed by those functions or based on the surgical results of the surgical tasks when performed manually and automatically (this may be based on historical data from past operations including that task performed by that function). For example, historical data may show that the surgical results of the surgical task are improved when the function performs the task autonomously, and therefore, the optimization may give priority to enabling the function to perform the surgical task autonomously. By optimizing in this way, the results of the surgical operation can generally be improved. For example, the results of tissue resection may correspond to the integrity of the sealing line, where less sealing leakage is equivalent to a more positive result compared to more sealing leakage. The results of the operation also correspond to the number of difficulties that occurred during the surgical operation, which may include, for example, instrument failure, sealing line leakage, nail line misfire, etc.

[0032] The method may further include detecting a third surgical device in the surgical environment. The third surgical device may include a third plurality of functions. The method may further include adjusting the one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform based on the plurality of resources and the third plurality of functions. Advantageously, the method includes dynamically determining which functions of the device should be enabled or disabled while the surgical procedure is being performed and / or when the device is brought into the surgical environment and / or connected to the hub.

[0033] The adjustment may include disabling previously enabled functions. By dynamically enabling and disabling functions, it is possible to reallocate resources when a new device is detected or while a surgical procedure is ongoing.

[0034] The method may also include: detecting a third surgical device in the surgical environment. The third surgical device may include a third plurality of functions. The method may also include: determining one or more functions of the third plurality of functions for the third surgical device to perform based on the plurality of resources.

[0035] The method may further include determining, based on the plurality of resources, one or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform.

[0036] The optional functions may be selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0037] A computer program is described, comprising instructions which, when executed by a computer, cause the computer to perform any of the methods previously described.

[0038] A computer readable medium is described, comprising instructions which, when executed by a computer, cause the computer to perform any of the previously described methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a block diagram of a computer-implemented surgical system.

[0040] Figure 2 An exemplary surgical system is shown in a surgical operating room.

[0041] Figure 3 Exemplary surgical hubs that pair with various systems are illustrated.

[0042] Figure 4 A surgical data network having a set of communicating surgical hubs configured to be able to connect to a set of sensing systems, an environmental sensing system, a set of devices, etc. is illustrated.

[0043] Figure 5 A logic diagram of a control system for a surgical instrument is illustrated.

[0044] Figure 6 An exemplary surgical system is shown that includes a handle having a controller and a motor, an adapter releasably coupled to the handle, and a loading unit releasably coupled to the adapter.

[0045] Figure 7 An exemplary situational awareness surgical system is shown.

[0046] Figure 8 An example of a surgical autonomous system is shown.

[0047] Fig. 9 An example of an overview of an adaptive interconnected surgical system is shown.

[0048] Fig.10 An example of the relationship between feature sets, surgical hubs, and resource management is shown.

[0049] Fig.11 An example of a matrix generator and function determination is shown.

[0050] Fig.12 A flow chart of an adaptively interconnected surgical system is shown.

[0051] Fig.13 An example of an overview of a rules engine is shown. DETAILED DESCRIPTION

[0052] Figure 1 20000. An exemplary surgical system such as surgical system 20000 may include one or more surgical systems (e.g., surgical subsystems) 20002, 20003, and 20004. For example, surgical system 20002 may include a computer-implemented interactive surgical system. For example, surgical system 20002 may include a surgical hub 20006 and / or a computing device 20016 that communicates with a cloud computing system 20008, such as Figure 2 The cloud computing system 20008 may include at least one remote cloud server 20009 and at least one remote cloud storage unit 20010. Exemplary surgical systems 20002, 20003, or 20004 may include a wearable sensing system 20011, an environmental sensing system 20015, a robotic system 20013, one or more intelligent instruments 20014, a human-machine interface system 20012, and the like. The human-machine interface system is also referred to herein as a human-machine interface device. The wearable sensing system 20011 may include one or more HCP sensing systems and / or one or more patient sensing systems. The environmental sensing system 20015 may include, for example, one or more devices for measuring one or more environmental attributes, such as Figure 2 As further described. The robotic system 20013 may include a plurality of devices for performing a surgical procedure, such as Figure 2 Further described.

[0053] The surgical system 20002 can communicate with a remote server 20009, which can be part of a cloud computing system 20008. In one example, the surgical system 20002 can communicate with the remote server 20009 via a cable / FIOS networking node of an Internet service provider. In one example, the patient sensing system can communicate directly with the remote server 20009. The surgical system 20002 and / or components therein can communicate with the remote server 20009 via a cellular transmit / receive point (TRP) or base station using one or more of the following cellular protocols: GSM / GPRS / EDGE (2G), UMTS / HSPA (3G), Long Term Evolution (LTE) or 4G, Advanced LTE (LTE-A), New Radio (NR), or 5G.

[0054] The surgical hub 20006 can cooperate and interact with one of the multiple devices that display images from the laparoscope and information from one or more other smart devices and one or more sensing systems 20011. The surgical hub 20006 can interact with one or more sensing systems 20011, one or more smart devices, and multiple displays. The surgical hub 20006 can be configured to collect measurement data from one or more sensing systems 20011 and send notification or control messages to the one or more sensing systems 20011. The surgical hub 20006 can send information including notification information to the human-machine interface system 20012 and / or receive information including notification information from the human-machine interface system. The human-machine interface system 20012 may include one or more human-machine interface devices (HIDs). The surgical hub 20006 can send and / or receive notification information or control information to convert into audio, display and / or control information to various devices communicating with the surgical hub.

[0055] For example, the sensing system 20001 may include a wearable sensing system 20011 (which may include one or more HCP sensing systems and one or more patient sensing systems) and an environmental sensing system 20015, such as Figure 1 The one or more sensing systems 20001 can measure data related to various biomarkers. The one or more sensing systems 20001 can use one or more sensors such as light sensors (e.g., photodiodes, photoresistors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, pyroelectric sensors, infrared sensors, etc. to measure biomarkers. The one or more sensors can use one or more of the following sensing technologies to measure biomarkers as described herein: photoplethysmography, electrocardiography, electroencephalography, colorimetry, impedance tracing, potentiometric measurement, amperometric measurement, etc.

[0056] Biomarkers measured by the one or more sensing systems 20001 may include, but are not limited to, sleep, core body temperature, maximum oxygen uptake, physical activity, alcohol consumption, respiratory rate, oxygen saturation, blood pressure, blood glucose, heart rate variability, blood pH, hydration status, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, coughing and sneezing, gastrointestinal motility, gastrointestinal imaging, respiratory bacteria, edema, psychiatric factors, sweat, circulating tumor cells, autonomic nervous tone, circadian rhythm and / or menstrual cycle.

[0057] Biomarkers may be related to physiological systems, which may include, but are not limited to, behavior and psychology, cardiovascular system, renal system, skin system, nervous system, gastrointestinal system, respiratory system, endocrine system, immune system, tumor, musculoskeletal system and / or reproductive system. Information from biomarkers can be determined and / or used by, for example, a computer-implemented patient and surgical system 20000. Information from biomarkers can be determined and / or used by a computer-implemented patient and surgical system 20000, for example, to improve the system and / or improve patient outcomes. One or more sensing systems 20001, biomarkers 20005 and physiological systems are described in more detail in U.S. application 17 / 156,287 (Agent File No. END9290USNP1), filed on January 22, 2021, entitled "METHODOF ADJUSTING A SURGICAL PARAMETER BASED ON BIOMARKER MEASUREMENTS", the disclosure of which is incorporated herein by reference in its entirety.

[0058] Figure 2 An example of a surgical system 20002 in a surgical operating room is shown. Figure 2 As illustrated, the patient undergoes surgery by one or more healthcare professionals (HCPs). The HCP is monitored by one or more HCP sensing systems 20020 worn by the HCP. The HCP and the environment around the HCP may also be monitored by one or more environmental sensing systems, which may include, for example, a set of cameras 20021, a set of microphones 20022, and other sensors that may be deployed in an operating room. The HCP sensing system 20020 and the environmental sensing system may communicate with a surgical hub 20006, which in turn may communicate with one or more cloud servers 20009 of a cloud computing system 20008, such as Figure 1 The environmental sensing system may be used to measure one or more environmental attributes, such as the position of the HCP in the operating room, HCP movement, ambient noise in the operating room, temperature / humidity in the operating room, etc.

[0059] like Figure 2As illustrated, the main display 20023 and one or more audio output devices (e.g., speakers 20019) are positioned in the sterile field to be visible to the operator at the operating table 20024. In addition, the visualization / notification tower 20026 is positioned outside the sterile field. The visualization / notification tower 20026 may include a first non-sterile human-machine interaction device (HID) 20027 and a second non-sterile HID 20029 that are separated from each other. The HID can be a display or a display with a touch screen that allows a person to directly interface with the HID. The human-machine interface system guided by the surgical hub 20006 can be configured to be able to coordinate the information flow of operators inside and outside the sterile field using HIDs 20027, 20029, and 20023. In one example, the surgical hub 20006 can cause the HID (e.g., the main HID 20023) to display notifications and / or information about patients and / or surgical procedures. In one example, the surgical hub 20006 can prompt for input and / or receive input from personnel in the sterile or non-sterile field. In one example, the surgical hub 20006 can cause the HID to display a snapshot of the surgical site recorded by the imaging device 20030 on the non-sterile HID 20027 or 20029 while maintaining a real-time feed of the surgical site on the main HID 20023. For example, the snapshot on the non-sterile display 20027 or 20029 can allow a non-sterile operator to perform diagnostic steps related to the surgical procedure.

[0060] In one aspect, the surgical hub 20006 can be configured to be able to route diagnostic input or feedback entered by a non-sterile operator at the visualization tower 20026 to the primary display 20023 within the sterile field, where the sterile operator at the operating table can view the diagnostic input or feedback. In one example, the input can be a modified form of a snapshot displayed on the non-sterile display 20027 or 20029, which can be routed to the primary display 20023 via the surgical hub 20006.

[0061] refer to Figure 2, surgical instrument 20031 is used in a surgical procedure as part of surgical system 20002. Hub 20006 can be configured to coordinate the flow of information to the display of surgical instrument 20031. For example, described in U.S. Patent Application Publication No. US2019-0200844A1 (U.S. Patent Application No. 16 / 209,385) filed on December 4, 2018, entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY", the disclosure of which is incorporated herein by reference in its entirety. Diagnostic input or feedback entered by a non-sterile operator at visualization tower 20026 can be routed by hub 20006 to a surgical instrument display within the sterile field, where the operator of surgical instrument 20031 can view the diagnostic input or feedback. For example, exemplary surgical instruments suitable for use with surgical system 20002 are described under the heading “Surgical Instrument Hardware” in U.S. patent application publication No. US 2019-0200844A1 (U.S. patent application No. 16 / 209,385), entitled “METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY,” filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety.

[0062] Figure 2 An example of a surgical system 20002 for performing a surgical procedure on a patient lying flat on an operating table 20024 in a surgical operating room 20035 is illustrated. A robotic system 20034 may be used in a surgical procedure as part of the surgical system 20002. The robotic system 20034 may include a surgeon's console 20036, a patient side cart 20032 (surgical robot), and a surgical robot hub 20033. The patient side cart 20032 may manipulate at least one removably coupled surgical tool 20037 through a minimally invasive incision in the patient's body while the surgeon observes the surgical site through the surgeon's console 20036. An image of the surgical site may be obtained through a medical imaging device 20030, which may be manipulated by the patient side cart 20032 to orient the imaging device 20030. The robotic hub 20033 may be used to process the image of the surgical site for subsequent display to the surgeon through the surgeon's console 20036.

[0063] Other types of robotic systems may be readily adapted for use with the surgical system 20002. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application No. US2019-0201137A1, filed on December 4, 2018, entitled “METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL” (U.S. Patent Application No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety.

[0064] Various examples of cloud-based analytics performed by cloud computing system 20008 and suitable for use with the present disclosure are described in U.S. patent application publication No. US2019-0206569 A1 (U.S. patent application No. 16 / 209,403), entitled “METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB,” filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety.

[0065] In various aspects, the imaging device 20030 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, charge coupled device (CCD) sensors and complementary metal oxide semiconductor (CMOS) sensors.

[0066] The optical components of the imaging device 20030 may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate multiple portions of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0067] One or more illumination sources may be configured to radiate electromagnetic energy in the visible spectrum as well as in the invisible spectrum. The visible spectrum (sometimes referred to as the optical spectrum or the luminescent spectrum) is that portion of the electromagnetic spectrum that is visible to the human eye (i.e., detectable by the human eye) and may be referred to as visible light or simply light. A typical human eye will respond to wavelengths in the range of about 380 nm to about 750 nm in air.

[0068] The invisible spectrum (e.g., non-luminescent spectrum) is the portion of the electromagnetic spectrum below and above the visible spectrum (i.e., wavelengths below about 380 nm and above about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, and they become invisible infrared (IR), microwaves, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, and they become invisible ultraviolet, x-ray, and gamma-ray electromagnetic radiation.

[0069] In various aspects, the imaging device 20030 is configured for use in minimally invasive surgery. Examples of imaging devices suitable for use in the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, choledochoscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophago-duodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngeal-renal endoscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0070] The imaging device can use multispectral monitoring to distinguish the topography and the underlying structure. A multispectral image is an image that captures image data within a specific wavelength range across the electromagnetic spectrum. The wavelengths can be separated by filters or by using instruments that are sensitive to specific wavelengths, including light from frequencies outside the visible light range, such as IR and ultraviolet. Spectral imaging allows the extraction of additional information that the human eye fails to capture with its red, green, and blue receptors. The use of multispectral imaging is described in more detail under the title "Advanced Imaging Acquisition Module" in U.S. Patent Application Publication No. US2019-0200844 A1 (U.S. Patent Application No. 16 / 209,385) filed on December 4, 2018, entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY", the disclosure of which is incorporated herein by reference in its entirety. After completing the surgical task to perform one or more of the previously described tests on the treated tissue, multispectral monitoring can be a useful tool for relocating the surgical site. It is self-evident that during any surgical operation, the operating room and surgical equipment need to be strictly sterilized. The strict sanitation and sterilization conditions required in the "surgical operating room" (i.e., operating room or treatment room) require the highest possible sterility of all medical devices and equipment. Part of this sterilization process is the need to sterilize any material that contacts the patient or penetrates the sterile field, including the imaging device 20030 and its attachments and components. It should be understood that the sterile field can be considered as a designated area that is considered to be free of microorganisms, such as in a tray or in a sterile towel, or the sterile field can be considered as the area around the patient prepared for surgery. The sterile field can include properly dressed scrubbed team members, as well as all equipment and fixtures in the area.

[0071] Figure 1 The illustrated wearable sensing system 20011 may include one or more sensing systems, such as Figure 2HCP sensing system 20020 shown. HCP sensing system 20020 may include a sensing system for monitoring and detecting a set of physical states and / or a set of physiological states of a healthcare provider (HCP). The HCP may typically be a surgeon or one or more healthcare providers assisting the surgeon or other healthcare providers. In one example, sensing system 20020 may measure a set of biomarkers to monitor the heart rate of the HCP. In one example, a sensing system 20020 (e.g., a watch or wristband) worn on the surgeon's wrist may use an accelerometer to detect hand motion and / or shaking and determine the amplitude and frequency of the tremor. Sensing system 20020 may send measurement data associated with the set of biomarkers and data associated with the surgeon's physical state to surgical hub 20006 for further processing. One or more environmental sensing devices may send environmental information to surgical hub 20006. For example, an environmental sensing device may include a camera 20021 for detecting the hand / body position of the HCP. The environmental sensing device may include a microphone 20022 for measuring ambient noise in the operating room. Other environmental sensing devices may include devices such as thermometers for measuring temperature and hygrometers for measuring humidity of the environment in the operating room. A surgical hub 20006, either alone or in communication with a cloud computing system, may use surgeon biomarker measurement data and / or environmental sensing information to modify the average latency of a control algorithm or a robotic interface of a handheld instrument, for example, to minimize tremor. In one example, the HCP sensing system 20020 may measure one or more surgeon biomarkers associated with the HCP and send measurement data associated with the surgeon biomarkers to the surgical hub 20006. The HCP sensing system 20020 may communicate with the surgical hub 20006 using one or more of the following RF protocols: Bluetooth, Bluetooth Low Energy (BLE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low Power Wireless Personal Area Network (6LoWPAN), Wi-Fi. Surgeon biomarkers may include one or more of the following: stress, heart rate, etc. Environmental measurements from the operating room may include ambient noise levels associated with surgeons or patients, surgeon and / or personnel movement, surgeon and / or personnel attention levels, etc.

[0072] The surgical hub 20006 can use the surgeon biomarker measurement data associated with the HCP to adaptively control one or more surgical instruments 20031. For example, the surgical hub 20006 can send a control program to the surgical instrument 20031 to control its actuator to limit or compensate for fatigue and the use of fine motor skills. The surgical hub 20006 can send the control program based on situational awareness and / or context about the importance or criticality of the task. When control is needed, the control program can instruct the instrument to change operation to provide more control.

[0073] Figure 3 An exemplary surgical system 20002 with a surgical hub 20006 is shown. The surgical hub 20006 can be paired with a wearable sensing system 20011, an environmental sensing system 20015, a human-machine interface system 20012, a robotic system 20013, and a smart device 20014 via modular controls. The hub 20006 includes a display 20048, an imaging module 20049, a generator module 20050, a communication module 20056, a processor module 20057, a storage array 20058, and an operating room mapping module 20059. In certain aspects, as Figure 3 As illustrated, the hub 20006 also includes a smoke evacuation module 20054 and / or a suction / irrigation module 20055. Various modules and systems can be connected to the modular controls directly via a router or via a communication module 20056. The operating room devices can be coupled to cloud computing resources and data storage devices via the modular controls. The human-machine interface system 20012 can include a display subsystem and a notification subsystem.

[0074] The modular control may be coupled to a non-contact sensor module. The non-contact sensor module may use an ultrasonic, laser-type and / or similar non-contact measuring device to measure the size of the operating room and generate a mapping of the surgical room. Other distance sensors may be used to determine the boundaries of the operating room. The ultrasonic-based non-contact sensor module may scan the operating room by emitting a burst of ultrasound and receiving an echo when it bounces back from the walls of the operating room, as described in the title "Surgical Hub Spatial Awareness Within an Operating Room" in U.S. Provisional Patent Application Serial No. 62 / 611,341, entitled "INTERACTIVE SURGICAL PLATFORM" filed on December 28, 2017, which is incorporated herein by reference in its entirety. The sensor module may be configured to be able to determine the size of the operating room and adjust the Bluetooth pairing distance limit. The laser-based non-contact sensor module may scan the operating room by emitting a laser pulse, receiving a laser pulse that bounces back from the walls of the operating room, and comparing the phase of the emitted pulse with the received pulse to determine the size of the operating room and adjust the Bluetooth pairing distance limit.

[0075] During surgery, the application of energy to tissue for sealing and / or cutting is often associated with smoke evacuation, suction of excess fluid, and / or flushing of tissue. Fluid lines, power lines, and / or data lines from different sources are often tangled during surgery. Solving this problem during surgery can waste valuable time. Disconnecting a line may require disconnecting the line from its corresponding module, which may require resetting the module. The hub modular housing 20060 provides a unified environment for managing power lines, data lines, and fluid lines, which reduces the frequency of tangling between such lines. Various aspects of the present disclosure provide a surgical hub 20006 for use in surgery involving the application of energy to tissue at a surgical site. The surgical hub 20006 includes a hub housing 20060 and a combined generator module slidably accommodated in a docking base of the hub housing 20060. The docking base includes data contacts and power contacts. The combined generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component seated in a single unit. In one aspect, the combined generator module also includes a smoke exhaust component for connecting the combined generator module to at least one energy delivery cable of a surgical instrument, at least one smoke exhaust component configured to exhaust smoke, fluid, and / or particles generated by applying therapeutic energy to tissue, and a fluid line extending from a remote surgical site to the smoke exhaust component. In one aspect, the fluid line can be a first fluid line, and a second fluid line can extend from the remote surgical site to a suction and flushing module 20055 slidably accommodated in a hub housing 20060. In one aspect, the hub housing 20060 may include a fluid interface. Certain surgical procedures may require more than one energy type to be applied to tissue. One energy type may be more conducive to cutting tissue, while another different energy type may be more conducive to sealing tissue. For example, a bipolar generator may be used to seal tissue, while an ultrasonic generator may be used to cut sealed tissue. Aspects of the present disclosure provide a solution in which a hub modular housing 20060 is configured to accommodate different generators and facilitate interactive communication between them. One of the advantages of the hub modular housing 20060 is that it enables rapid removal and / or replacement of various modules. Aspects of the present disclosure provide a modular surgical housing for use in surgical procedures involving application of energy to tissue. The modular surgical housing includes a first energy generator module, the first energy generator module is configured to generate a first energy for application to tissue, and a first docking base, the first docking base includes a first docking port, the first docking port includes first data and power contacts, wherein the first energy generator module is slidably movable into electrical engagement with the power and data contacts, and wherein the first energy generator module is slidably movable out of electrical engagement with the first power and data contacts.Further describing the above, the modular surgical shell also includes a second energy generator module, the second energy generator module is configured to generate a second energy different from the first energy for application to tissue, and a second docking base, the second docking base includes a second docking port, the second docking port includes second data and power contacts, wherein the second energy generator module is slidably movable into electrical engagement with the power and data contacts, and wherein the second energy generator is slidably movable out of electrical contact with the second power and data contacts. In addition, the modular surgical shell also includes a communication bus between the first docking port and the second docking port, which is configured to facilitate communication between the first energy generator module and the second energy generator module. Reference. Figure 3 , aspects of the present disclosure are presented as a hub modular housing 20060 that allows modular integration of a generator module 20050, a smoke evacuation module 20054, and a suction / irrigation module 20055. The hub modular housing 20060 also facilitates interactive communication between the modules 20059, 20054, 20055. The generator module 20050 may have integrated monopolar components, bipolar components, and ultrasonic components supported in a single housing unit that is slidably inserted into the hub modular housing 20060. The generator module 20050 may be configured to be connectable to a monopolar device 20051, a bipolar device 20052, and an ultrasonic device 20053. Alternatively, the generator module 20050 may include a series of monopolar generator modules, bipolar generator modules, and / or ultrasonic generator modules that interact through the hub modular housing 20060. The hub modular housing 20060 can be configured to facilitate the insertion of multiple generators and interactive communication between the generators docked into the hub modular housing 20060 so that the generators will act as a single generator.

[0076] Figure 4 A surgical data network having a set of communication hubs according to at least one aspect of the present disclosure is illustrated, the communication hubs being configured to connect a set of sensing systems, environmental sensing systems, and a set of other modular devices located in one or more operating rooms, patient recovery rooms, or rooms in the medical facility specifically equipped for surgical operations to a cloud.

[0077] like Figure 4As illustrated, the surgical hub system 20060 may include a modular communication hub 20065 configured to connect modular devices located in a medical facility to a cloud-based system (e.g., a cloud computing system 20064, which may include a remote server 20067 coupled to a remote storage device 20068). The modular communication hub 20065 and devices may be connected in a room in a medical facility that is specifically equipped for surgical operations. In one aspect, the modular communication hub 20065 may include a network hub 20061 and / or a network switch 20062 that communicates with a network router 20066. The modular communication hub 20065 may be coupled to a local computer system 20063 to provide local computer processing and data manipulation.

[0078] The computer system 20063 may include a processor and a network interface 20100. The processor may be coupled to a communication module, a storage device, a memory, a non-volatile memory, and an input / output (I / O) interface via a system bus. The system bus may be any of several types of bus structures including a memory bus or a memory controller, a peripheral bus or an external bus, and / or a local bus using any of a variety of available bus architectures, including but not limited to a 9-bit bus, an Industry Standard Architecture (ISA), a Micro Charmel Architecture (MSA), an Extended ISA (EISA), Intelligent Drive Electronics (IDE), a VESA Local Bus (VLB), a Peripheral Component Interconnect (PCI), a USB, an Advanced Graphics Port (AGP), a Personal Computer Memory Card International Association bus (PCMCIA), a Small Computer System Interface (SCSI), or any other peripheral bus.

[0079] The controller may be any single-core or multi-core processor, such as those offered by Texas Instruments under the trade name ARM Cortex. In one aspect, the processor may be a commercially available LM4F230H5QR ARM Cortex-M4F processor core, such as Texas Instruments, which includes 256KB of single-cycle flash or other non-volatile memory (up to 40MHz) on-chip memory, a pre-fetch buffer for improved execution above 40MHz, 32KB of single-cycle sequential random access memory (SRAM), a Stellaris Internal read-only memory (ROM) for software, 2KB electrically erasable programmable read-only memory (EEPROM), and / or one or more pulse width modulation (PWM) modules, one or more quadrature encoder input (QEI) simulations, one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, details of which can be found in the product data sheet.

[0080] In one example, the processor may include a safety controller including two families based on controllers such as TMS570 and RM4x, also known by Texas Instruments under the trade name Hercules ARMCortex R4. The safety controller may be configured to be capable of being specifically used for IEC 61508 and ISO 26262 safety-critical applications, among others, to provide advanced integrated safety features while delivering scalable execution, connectivity, and memory options.

[0081] It should be understood that the computer system 20063 may include software that acts as an intermediary between a user and the basic computer resources described in a suitable operating environment. Such software may include an operating system. An operating system that may be stored on a disk storage device may be used to control and allocate the resources of a computer system. System applications may utilize the operating system to manage resources through program modules and program data stored in system memory or disk storage devices. It should be understood that the various components described herein may be implemented using various operating systems or combinations of operating systems.

[0082] The user can input commands or information into the computer system 20063 through an input device coupled to the I / O interface. The input device may include, but is not limited to, a pointing device, such as a mouse, a touch ball, a stylus, a touch pad, a keyboard, a microphone, a joystick, a game pad, a satellite dish, a scanner, a TV tuner card, a digital camera, a digital video camera, a web camera, etc. These and other input devices are connected to the processor 20102 through the system bus via an interface port. The interface port includes, for example, a serial port, a parallel port, a game port, and a USB. The output device uses the same type of port as the input device. Therefore, for example, a USB port can be used to provide input to the computer system 20063 and output information from the computer system 20063 to the output device. An output adapter is provided to illustrate that some output devices, such as monitors, displays, speakers, and printers, may need special adapters in other output devices. The output adapter may include, by way of example, but is not limited to, a video and sound card that provides a connection device between the output device and the system bus. It should be noted that other devices or systems such as remote computers can provide both input capabilities and output capabilities.

[0083] Computer system 20063 can be operated in a networked environment using a logical connection with one or more remote computers (such as cloud computers) or local computers. A remote cloud computer can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor-based appliance, a peer device or other public network node, etc., and generally includes many or all of the elements described relative to the computer system. For simplicity, only a memory storage device with a remote computer is illustrated. A remote computer can be logically connected to a computer system via a network interface, and then physically connected via a communication connection. A network interface can cover communication networks such as a local area network (LAN) and a wide area network (WAN). LAN technology can include a fiber distributed data interface (FDDI), a copper distributed data interface (CDDI), Ethernet / IEEE 802.3, a token ring / IEEE 802.5, etc. WAN technology can include, but is not limited to, point-to-point links, circuit switching networks such as integrated services digital networks (ISDN) and variants thereof, packet switching networks, and digital subscriber lines (DSL).

[0084] In various examples, computer system 20063 may include an image processor, an image processing engine, a media processor, or any dedicated digital signal processor (DSP) for processing digital images. The image processor may employ parallel computing with single instruction, multiple data (SIMD) or multiple instructions, multiple data (MIMD) techniques to improve speed and efficiency. The digital image processing engine may perform a range of tasks. The image processor may be a system on a chip with a multi-core processor architecture.

[0085] The communication connection may refer to the hardware / software used to connect the network interface to the bus. Although the communication connection is shown for exemplary clarification within the computer system 20063, it may also be located outside the computer system 20063. The hardware / software necessary to connect to the network interface may include internal and external technologies for exemplary purposes only, such as modems, including conventional telephone-grade modems, cable modems, fiber optic modems and DSL modems, ISDN adapters, and Ethernet cards. In some examples, an RF interface may also be used to provide a network interface.

[0086] The surgical data network associated with the surgical hub system 20060 can be configured as passive, intelligent, or switched. A passive surgical data network acts as a conduit for data, enabling it to be transferred from one device (or segment) to another device (or segment) and to cloud computing resources. An intelligent surgical data network includes additional features to enable monitoring of traffic through the surgical data network and configuration of each port in the network hub 20061 or network switch 20062. An intelligent surgical data network may be referred to as a manageable hub or switch. A switching hub reads the destination address of each packet and then forwards the packet to the correct port.

[0087] Modular devices 1a-1n located in an operating room may be coupled to a modular communication hub 20065. A network hub 20061 and / or a network switch 20062 may be coupled to a network router 20066 to connect devices 1a-1n to a cloud computing system 20064 or a local computer system 20063. Data associated with devices 1a-1n may be transmitted to a cloud-based computer via a router for remote data processing and manipulation. Data associated with devices 1a-1n may also be transmitted to a local computer system 20063 for local data processing and manipulation. Modular devices 2a-2m located in the same operating room may also be coupled to a network switch 20062. A network switch 20062 may be coupled to a network hub 20061 and / or a network router 20066 to connect devices 2a-2m to a cloud 20064. Data associated with devices 2a-2m may be transmitted to a cloud computing system 20064 via a network router 20066 for data processing and manipulation. Data associated with the devices 2a-2m may also be transmitted to the local computer system 20063 for local data processing and manipulation.

[0088] The wearable sensing system 20011 may include one or more sensing systems 20069. The sensing system 20069 may include an HCP sensing system and / or a patient sensing system. The one or more sensing systems 20069 may communicate with the computer system 20063 or cloud server 20067 of the surgical hub system 20060 directly via one of the network routers 20066 or via a network hub 20061 or a network switch 20062 that communicates with the network router 20066.

[0089] The sensing system 20069 may be coupled to a network router 20066 to connect the sensing system 20069 to a local computer system 20063 and / or a cloud computing system 20064. Data associated with the sensing system 20069 may be transmitted to the cloud computing system 20064 for data processing and manipulation via the network router 20066. Data associated with the sensing system 20069 may also be transmitted to the local computer system 20063 for local data processing and manipulation.

[0090] like Figure 4 As illustrated, the surgical hub system 20060 can be expanded by interconnecting multiple network hubs 20061 and / or multiple network switches 20062 with multiple network routers 20066. A modular communication hub 20065 can be included in a modular control tower that is configured to accommodate multiple devices 1a-1n / 2a-2m. A local computer system 20063 can also be included in the modular control tower. The modular communication hub 20065 can be connected to a display 20068 to display images obtained by some of the devices 1a-1n / 2a-2m during a surgical procedure, for example. In various aspects, devices 1a-1n / 2a-2m may include, for example, various modules, such as an imaging module coupled to an endoscope, a generator module coupled to an energy-based surgical device, a smoke evacuation module, a suction / irrigation module, a communication module, a processor module, a storage array, a surgical device connected to a display, and / or a non-contact sensor module in other modular devices that can be connected to a modular communication hub 20065 of a surgical data network.

[0091] In one aspect, Figure 4The illustrated surgical hub system 20060 may include a combination of a network hub, a network switch, and a network router that connects the device 1a-1n / 2a-2m or the sensing system 20069 to the cloud base system 20064. One or more of the devices 1a-1n / 2a-2m or the sensing system 20069 coupled to the network hub 20061 or the network switch 20062 can collect data in real time and transmit the data to a cloud computer for data processing and operation. It should be understood that cloud computing relies on shared computing resources rather than using local servers or personal devices to process software applications. The term "cloud" can be used as a metaphor for the "Internet", although the term is not so limited. Thus, the term "cloud computing" may be used herein to refer to a "type of Internet-based computing" in which different services (such as servers, storage devices, and applications) are delivered to a modular communication hub 20065 and / or computer system 20063 located in an operating room (e.g., a fixed, mobile, temporary, or on-site operating room or space) and devices connected to the modular communication hub 20065 and / or computer system 20063 via the Internet. The cloud infrastructure may be maintained by a cloud service provider. In this case, the cloud service provider may be an entity that coordinates the use and control of devices 1a-1n / 2a-2m located in one or more operating rooms. Cloud computing services can perform a large number of calculations based on data collected by smart surgical instruments, robots, sensing systems, and other computerized devices located in the operating room. The hub hardware enables multiple devices, sensing systems, and / or connections to be connected to a computer that communicates with cloud computing resources and storage devices.

[0092] By applying cloud computer data processing technology to the data collected by the devices 1a-1n / 2a-2m, the surgical data network can provide improved surgical results, reduced costs and improved patient satisfaction. At least some of the devices 1a-1n / 2a-2m can be used to observe tissue status to evaluate the leakage or perfusion of the sealed tissue after tissue sealing and cutting surgery. At least some of the devices 1a-1n / 2a-2m can be used to identify pathology, such as the effects of disease, using cloud-based computing to examine data including images of body tissue samples for diagnostic purposes. This may include positioning and edge confirmation of tissue and phenotype. At least some of the devices 1a-1n / 2a-2m can be used to identify the anatomical structure of the body using various sensors integrated with imaging devices and technologies (such as overlapping images captured by multiple imaging devices). The data (including image data) collected by the devices 1a-1n / 2a-2m can be transmitted to the cloud computing system 20064 or the local computer system 20063 or both for data processing and manipulation, including image processing and manipulation. The data can be analyzed to improve surgical outcomes by determining whether further treatment can be pursued, such as endoscopic interventions, emerging technologies, targeted radiation, targeted interventions, and the application of precision robotics to tissue-specific sites and conditions. Such data analysis can be further processed using outcome analysis, and using standardized methods can provide helpful feedback to confirm surgical treatments and surgeon behavior, or suggest modifications to surgical treatments and surgeon behavior.

[0093] Applying cloud computer data processing techniques to the measurement data collected by the sensing systems 20069, the surgical data network can provide improved surgical results, improved recovery results, reduced costs, and improved patient satisfaction. At least some of the sensing systems 20069 can be used to assess the physiological condition of a surgeon operating on a patient, or a patient preparing for surgery, or a patient recovering after surgery. The cloud-based computing system 20064 can be used to monitor biomarkers associated with the surgeon or patient in real time, and can be used to generate a surgical plan based at least on the measurement data collected prior to surgery, provide control signals to surgical instruments during surgery, and notify patients of complications during the post-operative period.

[0094] The operating room devices 1a-1n can be connected to the modular communication hub 20065 through a wired channel or a wireless channel, depending on the configuration of the devices 1a-1n to the network hub 20061. In one aspect, the network hub 20061 can be implemented as a local network broadcast device that works on the physical layer of the open system interconnection (OSI) model. The network hub can provide connections with devices 1a-1n located in the same operating room network. The network hub 20061 can collect data in the form of packets and send it to the router in half-duplex mode. The network hub 20061 may not store any media access control / Internet protocol (MAC / IP) for transmitting device data. Only one device in the device 1a-1n can send data through the network hub 20061 at a time. The network hub 20061 may not have a routing table or intelligence about where to send information and broadcast all network data on each connection and broadcast all network data to the remote server 20067 of the cloud computing system 20064. Network hub 20061 can detect basic network errors such as collisions, but broadcasting all information to multiple ports can pose a security risk and cause bottlenecks.

[0095] The operating room devices 2a-2m may be connected to the network switch 20062 via a wired channel or a wireless channel. The network switch 20062 operates in the data link layer of the OSI model. The network switch 20062 may be a multicast device for connecting the devices 2a-2m located in the same operating room to the network. The network switch 20062 may send data to the network router 20066 in the form of a frame and may operate in full-duplex mode. Multiple devices 2a-2m may simultaneously send data via the network switch 20062. The network switch 20062 stores and uses the MAC addresses of the devices 2a-2m to transmit data.

[0096] The network hub 20061 and / or the network switch 20062 may be coupled to a network router 20066 to connect to the cloud computing system 20064. The network router 20066 operates in the network layer of the OSI model. The network router 20066 generates routes for transmitting data packets received from the network hub 20061 and / or the network switch 20062 to cloud-based computer resources to further process and manipulate data collected by any or all of the devices 1a-1n / 2a-2m and the wearable sensing system 20011. The network router 20066 may be employed to connect two or more different networks located in different locations, such as, for example, different operating rooms in the same medical facility or different networks located in different operating rooms in different medical facilities. The network router 20066 may send data to the cloud computing system 20064 in the form of packets and operate in full-duplex mode. Multiple devices may send data simultaneously. The network router 20066 may use IP addresses to transmit data.

[0097] In one example, the network hub 20061 may be implemented as a USB hub that allows multiple USB devices to be connected to a host computer. A USB hub may expand a single USB port to multiple levels so that more ports are available to connect devices to a host system computer. The network hub 20061 may include wired or wireless capabilities for receiving information through a wired channel or a wireless channel. In one aspect, a wireless USB short-range, high-bandwidth wireless radio communication protocol may be used for communication between devices 1a-1n and devices 2a-2m located in an operating room.

[0098] In an example, the operating room devices 1a-1n / 2a-2m and / or the sensing system 20069 can communicate with the modular communication hub 20065 via the Bluetooth wireless technology standard for exchanging data from fixed devices and mobile devices over short distances (using short wavelength UHF radio waves from 2.4GHz to 2.485GHz in the ISM band) and building a personal area network (PAN). The operating room devices 1a-1n / 2a-2m and / or the sensing system 20069 can communicate with the modular communication hub 20065 via a variety of wireless or wired communication standards or protocols, including but not limited to Bluetooth, Low-Energy Bluetooth, Near Field Communication (NFC), Wi-Fi (IEEE802.11 series), WiMAX (IEEE 802.16 series), IEEE 802.20, New Radio (NR), Long Term Evolution (LTE) and Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT and its Ethernet derivatives, and any other wireless and wired protocols designated as 3G, 4G, 5G and above. The computing module may include multiple communication modules. For example, the first communication module can be dedicated to shorter-range wireless communications, such as Wi-Fi and Bluetooth Low Energy, Bluetooth Smart, while the second communication module can be dedicated to longer-range wireless communications, such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, etc.

[0099] Modular communication hub 20065 may serve as a central connection for one or more of operating room devices 1a-1n / 2a-2m and / or sensing systems 20069 and may process data types referred to as frames. Frames may carry data generated by devices 1a-1n / 2a-2m and / or sensing systems 20069. When modular communication hub 20065 receives a frame, the frame may be amplified and / or sent to a network router 20066, which may transmit the data to a cloud computing system 20064 or a local computer system 20063 using a plurality of wireless or wired communication standards or protocols, as described herein.

[0100] Modular communication hub 20065 can be used as a standalone device or connected to a compatible network hub 20061 and network switch 20062 to form a larger network. Modular communication hub 20065 is generally easy to install, configure, and maintain, making it a good option for networking operating room devices 1a-1n / 2a-2m.

[0101] Figure 5 A logic diagram of a control system 20220 of a surgical instrument or surgical tool according to one or more aspects of the present disclosure is illustrated. The surgical instrument or surgical tool may be configurable. The surgical instrument may include a surgical fixation device dedicated to the upcoming surgery, such as an imaging device, a surgical stapler, an energy device, an endoscopic cutting and closing device, etc. For example, the surgical instrument may include any of an electric stapler, a power stapler generator, an energy device, a pre-energy device, a pre-energy jaw device, an endoscopic cutting and closing clamp, an energy device generator, an operating room imaging system, a smoke evacuator, a suction-irrigation device, an insufflation system, etc. The system 20220 may include a control circuit. The control circuit may include a microcontroller 20221, which includes a processor 20222 and a memory 20223. For example, one or more of sensors 20225, 20226, 20227 provide real-time feedback to the processor 20222. A motor 20230 driven by a motor driver 20229 is operably coupled to a longitudinally movable displacement member to drive an I-beam blade element. A tracking system 20228 may be configured to determine the position of the longitudinally movable displacement member. Position information may be provided to a processor 20222, which may be programmed or configured to determine the position of the longitudinally movable drive member and the position of the firing member, firing rod, and I-beam blade element. Additional motors may be provided at the tool driver interface to control I-beam firing, closure tube travel, shaft rotation, and joint motion. A display 20224 may display a variety of operating conditions of the instrument and may include a touch screen function for data input. The information displayed on the display 20224 may be superimposed with an image acquired via an endoscopic imaging module.

[0102] The microcontroller 20221 can be any single-core or multi-core processor, such as those provided by Texas Instruments under the trade name ARM Cortex. In one aspect, the main microcontroller 20221 can be a LM4F230H5QR ARM Cortex-M4F processor core available from, for example, Texas Instruments, which includes 256KB of single-cycle flash memory or other non-volatile memory (up to 40MHz) on-chip memory, a pre-fetch buffer for improving performance above 40MHz, 32KB of single-cycle SRAM, a Stellaris Internal ROM for software, 2KB EEPROM, one or more PWM modules, one or more QEI analog and / or one or more 12-bit ADC with 12 analog input channels, details of which can be found in the product datasheet.

[0103] The microcontroller 20221 may include a safety controller including two controller-based families such as TMS570 and RM4x, also known as Hercules ARM CortexR4 produced by Texas Instruments. The safety controller may be configured to be specifically used for IEC 61508 and ISO 26262 safety-critical applications, etc., to provide advanced integrated safety features while delivering scalable execution, connectivity and memory options.

[0104] The microcontroller 20221 can be programmed to perform various functions, such as precise control of the speed and position of the knife and articulation system. In one aspect, the microcontroller 20221 may include a processor 20222 and a memory 20223. The electric motor 20230 may be a brushed DC (DC) motor having a gearbox and a mechanical link to the articulation or knife system. In one aspect, the motor driver 20229 may be an A3941 available from Allegro Microsystems, Inc. Other motor drivers can be easily replaced for use in a tracking system 20228 including an absolute positioning system. A detailed description of the absolute positioning system is described in U.S. Patent Application Publication No. 2017 / 0296213, entitled "SYSTEMS AND METHODS FOR CONTROLLING ASURGICAL STAPLING AND CUTTING INSTRUMENT," published on October 19, 2017, which is incorporated herein by reference in its entirety.

[0105] The microcontroller 20221 can be programmed to provide precise control of the speed and position of the displacement member and articulation system. The microcontroller 20221 can be configured to be able to calculate the response in the software of the microcontroller 20221. The calculated response can be compared with the measured response of the actual system to obtain an "observed" response, which is used for actual feedback decisions. The observed response can be a favorable tuning value that balances the smooth continuous nature of the simulated response with the measured response, which can detect external influences on the system.

[0106] The motor 20230 may be controlled by a motor driver 20229 and may be employed by a firing system of a surgical instrument or tool. In various forms, the motor 20230 may be a brushed DC drive motor having a maximum rotational speed of about 25,000 RPM. In some examples, the motor 20230 may include a brushless motor, a cordless motor, a synchronous motor, a stepper motor, or any other suitable electric motor. The motor driver 20229 may include, for example, an H-bridge driver including a field effect transistor (FET). The motor 20230 may be powered by a power assembly that may be releasably mounted to a handle assembly or a tool housing for supplying control power to a surgical instrument or tool. The power assembly may include a battery that may include a plurality of battery cells connected in series that may be used as a power source to provide power to a surgical instrument or tool. In some cases, the battery cells of the power assembly may be replaceable and / or rechargeable. In at least one example, the battery cell may be a lithium-ion battery that may be coupled to the power assembly and detachable from the power assembly.

[0107] The motor driver 20229 may be an A3941 available from Allegro Microsystems, Inc. The A3941 may be a full-bridge controller for use with an external N-channel power metal oxide semiconductor field effect transistor (MOSFET) specifically designed for inductive loads such as brushed DC motors. The driver 20229 may include a unique charge pump regulator that provides full (>10V) gate drive for battery voltages as low as 7V and allows the A3941 to operate at a reduced gate drive as low as 5.5V. A bootstrap capacitor may be used to provide the above-mentioned battery supply voltage required for the N-channel MOSFET. The internal charge pump of the high-side driver may allow DC (100% duty cycle) operation. A diode or synchronous rectification may be used to drive the full bridge in fast decay mode or slow decay mode. In slow decay mode, current recirculation may pass through the high-side FET or the low-side FET. The power FET may be protected from breakdown by a resistor-adjustable dead time. Comprehensive diagnostics provide indications of undervoltage, overtemperature, and power bridge faults, and can be configured to protect the power MOSFETs under most short circuit conditions.Other motor drivers can be easily substituted for use in tracking system 20228 including an absolute positioning system.

[0108] The tracking system 20228 may include a controlled motor drive circuit arrangement including a position sensor 20225 according to one aspect of the present disclosure. The position sensor 20225 for the absolute positioning system can provide a unique position signal corresponding to the position of the displacement member. In some examples, the displacement member can represent a longitudinally movable drive member, which includes a rack of drive teeth for meshing engagement with a corresponding drive gear of the gear reducer assembly. In some examples, the displacement member can represent a firing member, which can be adapted and configured to include a rack of drive teeth. In some examples, the displacement member can represent a firing rod or an I-beam, each of which can be adapted and configured to include a rack of drive teeth. Therefore, as used herein, the term displacement member can be generally used to refer to any movable member of a surgical instrument or tool such as a drive member, a firing member, a firing rod, an I-beam, or any element that can be displaced. In one aspect, the longitudinally movable drive member can be coupled to the firing member, the firing rod, and the I-beam. Therefore, the absolute positioning system can actually track the linear displacement of the I-beam by tracking the linear displacement of the longitudinally movable drive member. In various aspects, the displacement member can be coupled to any position sensor 20225 suitable for measuring linear displacement. Therefore, the longitudinally movable drive member, firing member, firing rod or I-beam or a combination thereof can be coupled to any suitable linear displacement sensor. The linear displacement sensor may include a contact displacement sensor or a non-contact displacement sensor. The linear displacement sensor may include a linear variable differential transformer (LVDT), a differential variable reluctance transducer (DVRT), a sliding potentiometer, a magnetic sensing system including a movable magnet and a series of linearly arranged Hall effect sensors, a magnetic sensing system including a fixed magnet and a series of movable linearly arranged Hall effect sensors, an optical sensing system including a movable light source and a series of linearly arranged photodiodes or photodetectors, an optical sensing system including a fixed light source and a series of movable linearly arranged photodiodes or photodetectors, or any combination thereof.

[0109] The electric motor 20230 may include a rotatable shaft operably docked with a gear assembly that is mounted on a displacement member in meshing engagement with a set of drive teeth or a rack of drive teeth. The sensor element may be operably coupled to the gear assembly so that a single rotation of the position sensor 20225 element corresponds to a certain linear longitudinal translation of the displacement member. The arrangement of the transmission and sensor may be connected to a linear actuator via a rack and pinion arrangement, or to a rotary actuator via a spur gear or other connection. The power source may supply power to the absolute positioning system, and the output indicator may display the output of the absolute positioning system. The displacement member may represent a longitudinally movable drive member that includes a rack of drive teeth formed thereon for meshing engagement with a corresponding drive gear of a gear reducer assembly. The displacement member may represent a longitudinally movable firing member, a firing rod, an I-beam, or a combination thereof.

[0110] A single rotation of the sensor element associated with the position sensor 20225 may be equivalent to a longitudinal linear displacement d1 of the displacement member, where d1 is the longitudinal linear distance the displacement member moves from point "a" to point "b" following a single rotation of the sensor element coupled to the displacement member. The sensor arrangement may be connected via a gear reduction that enables the position sensor 20225 to complete one or more rotations for a full stroke of the displacement member. The position sensor 20225 may complete multiple rotations for a full stroke of the displacement member.

[0111] A series of switches (where n is an integer greater than one) may be employed, alone or in combination with gear reduction, to provide unique position signals for more than one rotation of the position sensor 20225. The state of the switches may be fed back to the microcontroller 20221, which applies logic to determine a unique position signal corresponding to the longitudinal linear displacement d1+d2+...dn of the displacement member. The output of the position sensor 20225 is provided to the microcontroller 20221. The position sensor 20225 of this sensor arrangement may include an array of magnetic sensors, analog rotation sensors (such as potentiometers), or analog Hall effect elements, which output a unique combination of position signals or values.

[0112] Position sensor 20225 may include any number of magnetic sensing elements, such as, for example, magnetic sensors classified according to whether they measure the total magnetic field or the vector components of the magnetic field. The technology used to produce the above two types of magnetic sensors can cover many aspects of physics and electronics. The technology used for magnetic field sensing may include probe coils, fluxgates, optical pumps, nuclear spins, superconducting quantum interference devices (SQUIDs), Hall effects, anisotropic magnetoresistance, giant magnetoresistance, magnetic tunnel junctions, giant magnetoimpedance, magnetostrictive / piezoelectric composites, magnetosensitive diodes, magnetosensitive transistors, optical fibers, magneto-optics, and magnetic sensors based on microelectromechanical systems, etc.

[0113] The position sensor 20225 for the tracking system 20228 including the absolute positioning system may include a magnetic rotation absolute positioning system. The position sensor 20225 may be implemented as an AS5055EQFT monolithic magnetic rotation position sensor available from Austria Microsystems, AG. The position sensor 20225 interfaces with the microcontroller 20221 to provide an absolute positioning system. The position sensor 20225 may be a low voltage and low power component and may include four Hall effect elements in the area of ​​the position sensor 20225 that may be located above the magnet. A high resolution ADC and an intelligent power management controller may also be provided on the chip. A coordinate rotation digital computer (CORDIC) processor (also known as the bitwise method and Volder algorithm) may be provided to perform a simple and efficient algorithm to calculate hyperbolic and trigonometric functions, which requires only addition, subtraction, bit shifting, and table lookup operations. The angular position, alarm bit, and magnetic field information may be transmitted to the microcontroller 20221 via a standard serial communication interface, such as a serial peripheral interface (SPI) interface. The position sensor 20225 may provide 12 or 14 bit resolution. The position sensor 20225 may be an AS5055 chip provided in a small QFN 16-pin 4×4×0.85 mm package.

[0114] The tracking system 20228 including an absolute positioning system may include and / or be programmed to implement feedback controllers such as PID, state feedback, and adaptive controllers. The power source converts the signal from the feedback controller into a physical input to the system: in this case, voltage. Other examples include PWM of voltage, current, and force. In addition to the position measured by the position sensor 20225, other sensors may be provided to measure physical parameters of the physical system. In some aspects, one or more other sensors may include sensor arrangements such as those described in U.S. Patent No. 9,345,481, entitled “STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM,” issued on May 24, 2016, which is incorporated herein by reference in its entirety; U.S. Patent Application Publication No. 2014 / 0263552, entitled “STAPLE CARTRIDGE TISSUETHICKNESS SENSOR SYSTEM,” issued on September 18, 2014, which is incorporated herein by reference in its entirety; and U.S. Patent Application Serial No. 15 / 628,175, entitled “TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT,” filed on June 20, 2017, which is incorporated herein by reference in its entirety. In a digital signal processing system, the absolute positioning system is coupled to a digital data acquisition system, where the output of the absolute positioning system will have a finite resolution and sampling frequency. The absolute positioning system may include comparison and combination circuits to combine the calculated response with the measured response using an algorithm (such as weighted average and theoretical control loop) that drives the calculated response toward the measured response. The calculated response of the physical system may take into account characteristics such as mass, inertia, viscous friction, inductance and resistance to predict the state and output of the physical system by knowing the input.

[0115] Thus, the absolute positioning system can provide the absolute position of the displacement member when the instrument is powered on, and does not retract or advance the displacement member to a reset (zero or home) position as may be required by conventional rotary encoders, which simply count the number of forward or backward steps taken by the motor 20230 to infer the position of the device actuator, drive rod, knife, etc.

[0116] Sensor 20226 (such as, for example, a strain gauge or micro strain gauge) can be configured to measure one or more parameters of the end effector, such as, for example, the amplitude of the strain applied to the anvil during the clamping operation, which can indicate the closing force applied to the anvil. The measured strain can be converted into a digital signal and provided to the processor 20222. Alternatively or in addition to sensor 20226, sensor 20227 (such as, for example, a load sensor) can measure the closing force applied to the anvil by the closed drive system. Sensor 20227 (such as, for example, a load sensor) can measure the firing force applied to the I-beam during the firing stroke of the surgical instrument or tool. The I-beam is configured to engage a wedge-shaped sliding member, which is configured to enable the nail driver to cam upward to push the nail out to deform and contact the anvil. The I-beam may also include a sharp cutting edge, which can be used to cut tissue when the I-beam is pushed distally by the firing rod. Alternatively, a current sensor 20231 can be employed to measure the current drawn by the motor 20230. For example, the force required to advance the firing member can correspond to the current drawn by the motor 20230. The measured force can be converted into a digital signal and provided to the processor 20222.

[0117] For example, a strain gauge sensor 20226 can be used to measure the force applied to the tissue by the end effector. A strain gauge can be coupled to the end effector to measure the force on the tissue processed by the end effector. A system for measuring the force applied to the tissue grasped by the end effector may include a strain gauge sensor 20226, such as a micro strain gauge, which can be configured to measure, for example, one or more parameters of the end effector. In one aspect, the strain gauge sensor 20226 can measure the amplitude or magnitude of the strain applied to the jaw member of the end effector during the clamping operation, which can indicate tissue compression. The measured strain can be converted into a digital signal and provided to the processor 20222 of the microcontroller 20221. The load sensor 20227 can measure the force used to operate the knife element, for example, to cut the tissue captured between the anvil and the nail magazine. A magnetic field sensor can be used to measure the thickness of the captured tissue. The measurement results of the magnetic field sensor can also be converted into a digital signal and provided to the processor 20222.

[0118] The microcontroller 20221 can characterize the corresponding values ​​of the selected position of the firing member and / or the velocity of the firing member using the measurements of tissue compression, tissue thickness, and / or the force required to close the end effector on the tissue measured by the sensors 20226, 20227, respectively. In one embodiment, the memory 20223 can store techniques, formulas, and / or lookup tables that can be used by the microcontroller 20221 in the evaluation.

[0119] The control system 20220 of the surgical instrument or tool may also include wired or wireless communication circuitry to communicate with the surgical hub 20065, such as Figure 4 shown.

[0120] Figure 6 An exemplary surgical system 20280 according to the present disclosure is shown and may include a surgical instrument 20282 that communicates with a console 20294 or a portable device 20296 via a wired and / or wireless connection through a local area network 20292 and / or a cloud network 20293. The console 20294 and the portable device 20296 may be any suitable computing device. The surgical instrument 20282 may include a handle 20297, an adapter 20285, and a loading unit 20287. The adapter 20285 is releasably coupled to the handle 20297, and the loading unit 20287 is releasably coupled to the adapter 20285 such that the adapter 20285 transmits force from a drive shaft to the loading unit 20287. The adapter 20285 or the loading unit 20287 may include a force gauge (not explicitly shown) disposed therein to measure the force applied to the loading unit 20287. The loading unit 20287 may include an end effector 20289 having a first jaw 20291 and a second jaw 20290. The loading unit 20287 may be an in situ loading or multi-fire loading unit (MFLU) that allows the clinician to fire multiple fasteners multiple times without removing the loading unit 20287 from the surgical site to reload the loading unit 20287.

[0121] The first jaw 20291 and the second jaw 20290 can be configured to clamp tissue therebetween, fire fasteners through the clamped tissue, and cut the clamped tissue. The first jaw 20291 can be configured to fire at least one fastener multiple times, or can be configured to include a replaceable multiple firing fastener bin, which includes multiple fasteners (e.g., nails, clamps, etc.) that can be fired more than once before being replaced. The second jaw 20290 may include an anvil that deforms the fastener or otherwise fixes the fastener when the fastener is ejected from the multiple firing fastener bin.

[0122] The handle 20297 may include a motor coupled to the drive shaft to affect the rotation of the drive shaft. The handle 20297 may include a control interface for selectively activating the motor. The control interface may include a button, a switch, a lever, a slider, a touch screen, and any other suitable input mechanism or user interface that can be engaged by a clinician to activate the motor.

[0123] The control interface of the handle 20297 can communicate with the controller 20298 of the handle 20297 to selectively activate the motor to affect the rotation of the drive shaft. The controller 20298 can be disposed within the handle 20297 and configured to receive input from the control interface and adapter data from the adapter 20285 or loading unit data from the loading unit 20287. The controller 20298 can analyze the input from the control interface and the data received from the adapter 20285 and / or the loading unit 20287 to selectively activate the motor. The handle 20297 can also include a display that can be viewed by a clinician during use of the handle 20297. The display can be configured to display a portion of the adapter or loading unit data before, during, or after firing the instrument 20282.

[0124] The adapter 20285 may include an adapter identification device 20284 disposed therein, and the loading unit 20287 may include a loading unit identification device 20288 disposed therein. The adapter identification device 20284 may communicate with the controller 20298, and the loading unit identification device 20288 may communicate with the controller 20298. It should be understood that the loading unit identification device 20288 may communicate with the adapter identification device 20284, which relays or transmits communications from the loading unit identification device 20288 to the controller 20298.

[0125] The adapter 20285 may also include a plurality of sensors 20286 (one is shown) disposed about it to detect various conditions of the adapter 20285 or the environment (e.g., whether the adapter 20285 is connected to the loading unit, whether the adapter 20285 is connected to the handle, whether the drive shaft is rotating, the torque of the drive shaft, the strain of the drive shaft, the temperature within the adapter 20285, the number of firings of the adapter 20285, the peak force of the adapter 20285 during firing, the total amount of force applied to the adapter 20285, the peak retraction force of the adapter 20285, the number of pauses of the adapter 20285 during firing, etc.). The plurality of sensors 20286 may provide input to the adapter identification device 20284 in the form of data signals. The data signals of the plurality of sensors 20286 may be stored in the adapter identification device 20284, or may be used to update the adapter data stored in the adapter identification device. The data signals of the plurality of sensors 20286 may be analog or digital. The plurality of sensors 20286 may include a force gauge to measure the force applied to the loading unit 20287 during firing.

[0126] The handle 20297 and the adapter 20285 can be configured to interconnect the adapter identification device 20284 and the loading unit identification device 20288 with the controller 20298 via an electrical interface. The electrical interface can be a direct electrical interface (i.e., comprising electrical contacts that engage with each other to transmit energy and signals therebetween). Additionally or alternatively, the electrical interface can be a contactless electrical interface to wirelessly transmit energy and signals therebetween (e.g., inductive transmission). It is also conceivable that the adapter identification device 20284 and the controller 20298 can communicate wirelessly with each other via a wireless connection separate from the electrical interface.

[0127] The handle 20297 may include a transceiver 20283 that is configured to transmit instrument data from the controller 20298 to other components of the system 20280 (e.g., the LAN 20292, the cloud 20293, the console 20294, or the portable device 20296). The controller 20298 may also transmit instrument data and / or measurement data associated with one or more sensors 20286 to the surgical hub. The transceiver 20283 may receive data (e.g., cartridge data, loading unit data, adapter data, or other notifications) from the surgical hub 20270. The transceiver 20283 may also receive data (e.g., cartridge data, loading unit data, or adapter data) from other components of the system 20280. For example, the controller 20298 may transmit instrument data to the console 20294, the instrument data including the serial number of the attachment adapter (e.g., adapter 20285) attached to the handle 20297, the serial number of the loading unit (e.g., loading unit 20287) attached to the adapter 20285, and the serial number of the multi-fire fastener cartridge loaded into the loading unit. Thereafter, the console 20294 may transmit data associated with the attached cartridge, loading unit, and adapter, respectively (e.g., cartridge data, loading unit data, or adapter data) back to the controller 20298. The controller 20298 may display a message on a local instrument display, or transmit the message to the console 20294 or the portable device 20296 via the transceiver 20283 to display the message on the display 20295 or the portable device screen, respectively.

[0128] Figure 7A diagram of a situational awareness surgical system 5100 is illustrated in accordance with at least one aspect of the present disclosure. Data sources 5126 may include, for example, modular devices 5102 (which may include sensors configured to detect parameters associated with a patient, HCP, and environment and / or the modular devices themselves), databases 5122 (e.g., an EMR database containing patient records), and patient monitoring devices 5124 (e.g., a blood pressure (BP) monitor and an electrocardiogram (EKG) monitor), HCP monitoring devices 35510, and / or environmental monitoring devices 35512. The surgical hub 5104 may be configured to be able to derive contextual information related to the surgical procedure from the data, for example, based on a particular combination of the received data or a particular order in which the data is received from the data sources 5126. The contextual information inferred from the received data may include, for example, the type of surgical procedure being performed, the particular step of the surgical procedure being performed by the surgeon, the type of tissue being operated on, or the body cavity being the subject of the surgery. This ability of some aspects of the surgical hub 5104 to derive or infer information related to the surgical procedure from the received data may be referred to as "situational awareness." For example, the surgical hub 5104 may incorporate a situational awareness system, which is hardware and / or programming associated with the surgical hub 5104 that derives contextual information related to the surgical procedure from data received and / or surgical planning information received from the edge computing system 35514 or the enterprise cloud server 35516.

[0129] The situational awareness system of the surgical hub 5104 may be configured to be able to derive contextual information from data received from the data source 5126 in a variety of different ways. For example, the situational awareness system may include a pattern recognition system or a machine learning system (e.g., an artificial neural network) that has been trained on training data to associate various inputs (e.g., data from the database 5122, the patient monitoring device 5124, the modular device 5102, the HCP monitoring device 35510, and / or the environmental monitoring device 35512) with corresponding contextual information about the surgical procedure. The machine learning system may be trained to accurately derive contextual information about the surgical procedure from the provided inputs. In an example, the situational awareness system may include a lookup table that stores pre-characterized environmental information about the surgical procedure associated with one or more inputs (or input ranges) corresponding to environmental information. In response to a query using one or more inputs, the lookup table may return corresponding contextual information used by the situational awareness system to control the modular device 5102. In an example, the contextual information received by the situational awareness system of the surgical hub 5104 may be associated with a specific control adjustment or a set of control adjustments for one or more modular devices 5102. In an example, the situational awareness system may include an additional machine learning system, lookup table, or other such system that generates or retrieves one or more control adjustments for one or more modular devices 5102 when contextual information is provided as input.

[0130] The surgical hub 5104 incorporating a situational awareness system can provide numerous benefits to the surgical system 5100. One benefit can include improved interpretation of sensed and collected data, which in turn will improve processing accuracy and / or use of the data during the surgical procedure. Returning to the previous example, the situational awareness surgical hub 5104 can determine the type of tissue being operated on; thus, when an unexpectedly high force is detected for the end effector of a closing surgical instrument, the situational awareness surgical hub 5104 can properly ramp up or ramp down the motor speed of the surgical instrument for the tissue type.

[0131] The type of tissue being operated on can affect the adjustment of the compression rate and load threshold of the surgical stapling and cutting instrument for a specific tissue gap measurement value. The situational awareness surgical hub 5104 can infer whether the surgical procedure being performed is a thoracic surgery or an abdominal surgery, thereby allowing the surgical hub 5104 to determine whether the tissue clamped by the end effector of the surgical stapling and cutting instrument is lung tissue (for thoracic surgery) or stomach tissue (for abdominal surgery). The surgical hub 5104 can then appropriately adjust the compression rate and load threshold of the surgical stapling and cutting instrument for the type of tissue.

[0132] The type of body cavity being operated on during an insufflation procedure can affect the function of the smoke evacuator. The situational awareness surgical hub 5104 can determine whether the surgical site is under pressure (by determining that the surgical procedure is utilizing insufflation) and determine the type of surgery. Since one type of surgery can often be performed in a particular body cavity, the surgical hub 5104 can then control the motor rate of the smoke evacuator appropriately for the body cavity in which the operation is being performed. Thus, the situational awareness surgical hub 5104 can provide consistent smoke evacuation for both thoracic and abdominal surgeries.

[0133] The type of surgery being performed can affect the optimal energy level at which an ultrasonic surgical instrument or radio frequency (RF) electrosurgical instrument operates. For example, arthroscopic surgery may require a higher energy level because the end effector of the ultrasonic surgical instrument or RF electrosurgical instrument is immersed in a fluid. The situational awareness surgical hub 5104 can determine whether the surgical procedure is an arthroscopic procedure. The surgical hub 5104 can then adjust the RF power level or ultrasonic amplitude (e.g., "energy level") of the generator to compensate for the fluid-filled environment. Relatedly, the type of tissue being operated on can affect the optimal energy level at which an ultrasonic surgical instrument or RF electrosurgical instrument operates. The situational awareness surgical hub 5104 can determine the type of surgical procedure being performed and then customize the energy level of the ultrasonic surgical instrument or RF electrosurgical instrument, respectively, according to the expected tissue profile of the surgical procedure. In addition, the situational awareness surgical hub 5104 can be configured to be able to adjust the energy level of the ultrasonic surgical instrument or RF electrosurgical instrument throughout the surgical procedure rather than just on a surgery-by-surgery basis. The situational awareness surgical hub 5104 can determine the step of a surgical procedure being performed or to be subsequently performed and then update the control algorithm of the generator and / or ultrasonic surgical instrument or RF electrosurgical instrument to set the energy level to a value appropriate for the expected tissue type based on the surgical step.

[0134] In an example, data may be extracted from additional data sources 5126 to improve conclusions extracted by the surgical hub 5104 from one data source 5126. The situational awareness surgical hub 5104 may augment the data it receives from the modular device 5102 with contextual information about the surgical procedure that has been constructed from other data sources 5126. For example, the situational awareness surgical hub 5104 may be configured to be able to determine whether hemostasis has occurred (e.g., whether bleeding at a surgical site has stopped) based on video or image data received from a medical imaging device. The surgical hub 5104 may be further configured to be able to compare physiological measurements (e.g., blood pressure sensed by a BP monitor communicatively coupled to the surgical hub 5104) with visual or image data of hemostasis (e.g., from a medical imaging device communicatively coupled to the surgical hub 5104) to determine the integrity of the staple line or tissue weld. The situational awareness system of the surgical hub 5104 may take into account the physiological measurement data to provide additional context when analyzing the visual data. Additional context can be useful when the visualization data itself may be uncertain or incomplete.

[0135] For example, if the situational awareness surgical hub 5104 determines that a subsequent step of the surgery requires the use of an RF electrosurgical instrument, it can actively activate the generator connected to the instrument. Actively activating the energy source can allow the instrument to be ready for use as soon as the previous step of the surgery is completed.

[0136] The situational awareness surgical hub 5104 may determine whether the current or subsequent steps of the surgical procedure require a different view or degree of magnification on the display based on the features that the surgeon anticipates needing to view at the surgical site. The surgical hub 5104 may proactively change the displayed view (e.g., provided by a medical imaging device for the visualization system) accordingly, such that the display is automatically adjusted throughout the surgical procedure.

[0137] The situational awareness surgical hub 5104 can determine which step of the surgical procedure is being performed or will be performed next and whether specific data or comparisons between data are required for that step of the surgical procedure. The surgical hub 5104 can be configured to automatically call up data screens based on the step of the surgical procedure being performed without having to wait for the surgeon to request that specific information.

[0138] Errors may be checked during the setup of a surgical procedure or during the course of a surgical procedure. For example, the situational awareness surgical hub 5104 may determine whether the operating room is correctly or optimally set up for the surgical procedure to be performed. The surgical hub 5104 may be configured to be able to determine the type of surgical procedure being performed, retrieve the corresponding list, product location, or setup requirements (e.g., from a memory), and then compare the current operating room layout with the standard layout determined by the surgical hub 5104 for the type of surgical procedure being performed. In some examples, the surgical hub 5104 may compare a list of items for surgery and / or a list of devices paired with the surgical hub 5104 with a suggested or expected list of items and / or devices for a given surgical procedure. If there is any discontinuity between the lists, the surgical hub 5104 may provide an alert indicating that a particular modular device 5102, a patient monitoring device 5124, an HCP monitoring device 35510, an environmental monitoring device 35512, and / or other surgical items are missing. In some examples, the surgical hub 5104 can determine the relative distance or position of the modular device 5102 and the patient monitoring device 5124, for example, via a proximity sensor. The surgical hub 5104 can compare the relative position of the devices to a suggested or expected layout for a particular surgical procedure. If there are any discontinuities between the layouts, the surgical hub 5104 can be configured to provide an alert indicating that the current layout for the surgical procedure deviates from the suggested layout.

[0139] The situational awareness surgical hub 5104 may determine if a surgeon (or other HCP) is making an error or otherwise deviating from an expected course of action during a surgical procedure. For example, the surgical hub 5104 may be configured to determine the type of surgical procedure being performed, retrieve a corresponding list of steps or order of equipment use (e.g., from memory), and then compare the steps being performed or equipment being used during the surgical procedure to the expected steps or equipment determined by the surgical hub 5104 for the type of surgical procedure being performed. The surgical hub 5104 may provide an alert indicating that an unexpected action is being performed or an unexpected device is being utilized at a particular step in the surgical procedure.

[0140] The surgical instruments (and other modular devices 5102) can be adjusted for the specific context of each surgical procedure (such as adjusting to different tissue types) and verification actions during the surgical procedure. The next steps, data, and display adjustments can be provided to the surgical instruments (and other modular devices 5102) in the surgical room according to the specific context of the surgery.

[0141] Surgical autonomous systems, apparatus, and methods may include aspects of integration with other medical devices, data sources, processes, and mechanisms. For example, surgical autonomous systems, apparatus, and methods may include aspects of integration with a computer-implemented interactive surgical system and / or one or more elements of a computer-implemented interactive surgical system. As described herein, surgical system, surgical autonomous system, and autonomous surgical system are interchangeable.

[0142] refer to Figure 8 , which may provide an overview of the surgical autonomous system 49000. Surgical instrument A 49005 and / or surgical instrument B 49035 may be used in surgical procedures as part of the surgical system 49000. The surgical hub 4900 may be configured to coordinate the flow of information to displays of surgical instruments. For example, the surgical hub may be described in U.S. Patent Application Publication No. US2019-0200844A1 (U.S. Patent Application No. 16 / 209,385), entitled “METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY,” filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. For example, exemplary surgical instruments suitable for use with surgical system 49000 are described under the heading “Surgical Instrument Hardware” in U.S. patent application publication No. US2019-0200844 A1 (U.S. patent application No. 16 / 209,385), entitled “METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY,” filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety.

[0143] Figure 8 An example of a surgical autonomous system 49000 is shown. System 49000 can be used to perform surgery on a patient lying flat on an operating table in a surgical operating room. The robotic system can be used in a surgical procedure as part of a surgical system. For example, the robotic system may be described in U.S. Patent Application Publication No. US2019-0200844 A1 (U.S. Patent Application No. 16 / 209,385), entitled “METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY,” filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. The robotic hub can be used to process images of the surgical site for subsequent display to the surgeon via the surgeon's console.

[0144] Other types of robotic systems may be readily adapted for use with the surgical system 49000. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application No. US2019-0201137A1, filed on December 4, 2018, entitled “METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL” (U.S. Patent Application No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety.

[0145] Various examples of cloud-based analytics performed by the cloud and suitable for use with the present disclosure are described in U.S. patent application publication No. US2019-0206569 A1, entitled “METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB,” filed on December 4, 2018 (U.S. patent application No. 16 / 209,403), the disclosure of which is incorporated herein by reference in its entirety.

[0146] In various aspects, an imaging device can be used in a surgical system and can include at least one image sensor and one or more optical components. Suitable image sensors can include, but are not limited to, charge coupled device (CCD) sensors and complementary metal oxide semiconductor (CMOS) sensors.

[0147] The optical components of the imaging device may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate multiple portions of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0148] One or more illumination sources may be configured to radiate electromagnetic energy in the visible spectrum as well as in the invisible spectrum. The visible spectrum (sometimes referred to as the optical spectrum or the luminescence spectrum) is that portion of the electromagnetic spectrum that is visible to (e.g., detectable by) the human eye and may be referred to as "visible light" or simply "light." A typical human eye will respond to wavelengths in the range of about 380 nm to about 750 nm in air.

[0149] The invisible spectrum (e.g., non-luminescent spectrum) is the portion of the electromagnetic spectrum below and above the visible spectrum (i.e., wavelengths below about 380 nm and above about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, and they become invisible infrared (IR), microwaves, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, and they become invisible ultraviolet, x-ray, and gamma-ray electromagnetic radiation.

[0150] In various aspects, the imaging device can be configured for use in minimally invasive surgery. Examples of imaging devices suitable for the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, choledochoscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophago-duodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngeal-renal endoscopes, sigmoidoscopes, thoracoscopes, and hysteroscopes.

[0151] The imaging device can use multispectral monitoring to distinguish the topography and the underlying structure. A multispectral image is an image that captures image data within a specific wavelength range across the electromagnetic spectrum. The wavelengths can be separated by filters or by using instruments that are sensitive to specific wavelengths, including light from frequencies outside the visible light range, such as IR and ultraviolet. Spectral imaging allows the extraction of additional information that the human eye fails to capture with its red, green, and blue receptors. The use of multispectral imaging is described in more detail under the title "Advanced Imaging Acquisition Module" in U.S. Patent Application Publication No. US2019-0200844 A1 (U.S. Patent Application No. 16 / 209,385), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY" filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. After completing the surgical task to perform one or more of the previously described tests on the treated tissue, multispectral monitoring can be a useful tool for repositioning the surgical site. It is self-evident that during any surgical operation, the operating room and surgical equipment need to be strictly sterilized. The strict sanitation and sterilization conditions required in the "surgical operating room" (i.e., operating room or treatment room) require the highest possible sterility of all medical devices and equipment. Part of this sterilization process is the need to sterilize any material that contacts the patient or penetrates the sterile field, including imaging devices and their attachments and components. It should be understood that the sterile field can be considered as a designated area that is considered to be free of microorganisms, such as in a tray or in a sterile towel, or the sterile field can be considered as the area around the patient prepared for surgery. The sterile field can include properly dressed scrubbed team members, as well as all equipment and fixtures in the area.

[0152] Surgical instrument A 49005 and / or surgical instrument B 49035 may include one or more capabilities. (e.g., surgical instrument A includes capabilities B, Z, and D, and surgical instrument includes capabilities C, F, and E). These capabilities may be associated with functions that the surgical instrument is capable of performing (e.g., A function 49010 associated with surgical instrument A 49005 and B function 49030 associated with surgical instrument B 49035). Examples of these functions may be described under the title “Surgical Instrument Hardware” in U.S. Patent Application Publication No. US 2019-0200844A1 (U.S. Patent Application No. 16 / 209,385), filed on December 4, 2018, entitled “METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY,” the disclosure of which is incorporated herein by reference in its entirety. For example, if the surgical instrument is an endoscopic cutting and closing device, one of the capabilities may be to remove tissue (e.g., tissue surrounding the colon when the surgeon is performing a proctocolectomy). These capabilities may include, for example, Fig. 9 The surgical tasks described. For example, the ability to remove tissue may include controlling energy source, cutting, suturing, knob orientation, body orientation, body positioning, anvil jaw force, reload alignment slot management, etc.

[0153] Data may be generated based on the performance of surgical instrument A 49005 and / or B 49035 (e.g., by a monitoring module located at the surgical hub 49040 or local to the surgical instrument). The data may be related to how the surgical instrument is performing. For example, the data may be associated with physical measurements, physiological measurements, etc. These measurements are described in more detail in U.S. patent application No. US17 / 156,28, filed on November 10, 2021, under the title "Monitoring Of Adjusting A Surgical Parameter Based On Biomarker Measurements," the disclosure of which is incorporated herein by reference in its entirety.

[0154] For example, an indication of the data may be transmitted to the surgical hub 49040, where the indication may be evaluated. In an example, the indication may be transmitted to a cloud service (e.g., an Amazon web service) as described herein. The data may be used as input in an analysis module.

[0155] Fig. 9 An example of an overview of an adaptive interconnected surgical system is shown. Fig. 9As shown, one or more surgical instruments may be used to perform a surgical task. For example, device A 49045 may be associated with a surgical instrument. Device B 49090 may be associated with a second surgical instrument. A surgical instrument may include one or more functions. For example, device A 49045 and / or device B 49090 may be associated with one or more functions (e.g., device A functions 1, 2, and 3 49050, and device B functions 1, 2, and 3 49085). The one or more functions may be functions that will be performed autonomously (e.g., during a surgical procedure). For example, a function of device A 49045 may be an orientation of device A 49045. If the function is enabled, as Fig. 9 As shown, the orientation of device A 49045 may be autonomously determined (eg, controlled) by the system.

[0156] If the function is not enabled, the function may be performed manually by the surgeon. A function associated with device B 49090 may be clamping. If the function is enabled, clamping of device B 49090 may be performed autonomously. If the function is not enabled, clamping may be performed by the surgeon performing the surgical task. Enabling and disabling the one or more functions associated with each of device A 49045 and device B 49090 may be dynamic. For example, when transitioning phases during the performance of a surgical task, the surgical hub 49095 may disable and / or enable one or more of the functions associated with device A 49045 and / or device B 49090 based on the introduction or removal of surgical instruments into the surgical OR or based on the introduction of a new surgical system into the surgical OR.

[0157] Enabling and / or disabling may be determined based on metrics of the surgical instrument (e.g., device A metrics 49055 and device B metrics 49080) (e.g., how device A 49045 performs its function, whether it is enabled and in autonomous mode or disabled and in manual mode). In this case, metrics may be generated based on one or more of the following sources that may be located on the surgical instrument: sensors, actuators, robotic metrics, patient biomarkers, forces, surgeon biomarkers, surgical context, number of personnel, changes in number of personnel, etc. Patient and / or surgeon biomarkers may be associated with measurements taken from a wearable device that the patient and / or surgeon may wear during the performance of a surgical task. These actuators may include one or more servomotors located on the surgical instrument. Metric data associated with these categories may be transmitted to the surgical hub 49095 via a message (e.g., a JSON message). The message may be delivered via a surgical application program interface. The surgical hub 49095 may receive the data and may process the data via the surgical hub processor 49110.

[0158] The one or more functions may be enabled and / or based on available resources associated with the system. For example, the available resources may be stored in a resource management subsystem 49115 of the surgical hub. Device A 49045 and device B 49090 may use resources from the resource management subsystem 49115 to (e.g., autonomously) perform one or more of their functions. In an example, the surgical hub 49095 may determine which functions to enable for the surgical instrument based on the available resources. The surgical hub 49095 may query the resource management subsystem 49115 to determine the available resources (e.g., the total resources available), which may be relative to the surgical instrument. Figure 5 and Figure 6 Determining which features to enable may involve the use of a rules engine 49105. The rules engine 49105 may be located within the surgical hub 49095. In an example, the rules engine 49105 may be located on a third-party service. These features may be collected by the surgical hub 49095 and used to generate a matrix (e.g., using a matrix generator 49100), such as with respect to Fig.11 described.

[0159] For example, function 1 of device A 49045 may be associated with a certain amount of bandwidth and a certain amount of power (e.g., the amount of bandwidth and the amount of power that can be used). The surgical hub 49095 may use this information in order to generate a matrix, which may be, for example, a data structure that links the functions of each surgical device to the resources (e.g., resource ranges) required to perform the function. The rules engine 49105 may be responsible for evaluating different variations of allocating resources to functions, and may optimize which functions to enable based on the evaluation at the surgical hub 49095. The functions determined to be enabled may be sent to the surgical instrument via a message. In an example, the resources and / or functions of the surgical instrument may be manually entered by medical personnel 49120.

[0160] Fig.10 An example of the relationship between a function set 49125, a surgical hub 49150, and resource management is shown. Function set 49125 may be included in the system. Function set 49125 may be manually input by a user 49160 (such as a surgeon or medical staff). In an example, function set 49125 may be a learning set determined using a machine learning algorithm.

[0161] One or more functions may be associated with a surgical instrument. Figure 2 Described, this one or more functions can be the function that surgical instrument can be able to perform autonomously.If this function is enabled, then this can indicate that surgical instrument performs this function autonomously, for example clamping.

[0162] The one or more functions may be associated with one or more resources 49145. For example, function A 49135 may be associated with resources 49145A, B, and C. In an example, a function may be associated with a resource range 49140. Resource range 49140 may be a range within which a surgical instrument can autonomously perform a function (e.g., the range may include a value indicating the expected performance of the function, which may be based on historical data and surgical context). For example, there may be a minimum resource and a maximum resource. In an example, there may be an optimal resource.

[0163] The one or more functions linked to the resources 49145 and / or resource scopes 49140 may be transmitted to the surgical hub 49150. The surgical hub 49150 may store and analyze such values ​​in a resource management subsystem 49155, which may include one or more different data structures for saving and querying functions, resources 49145, and resource scopes 49140. The enabled functions may be adjusted based on the introduction or removal of one or more surgical instruments or surgical systems (e.g., by changing the autonomy level associated with the function). A higher autonomy level may be associated with more tasks of the function being performed autonomously, and a lower autonomy level may be associated with fewer tasks of the function being performed autonomously.

[0164] In an example, a second surgical hub may be introduced into a surgical OR. The second surgical hub may provide more allowable resources 49145 for use by, for example, surgical instruments already present in the surgical OR. In this case, due to the increase in allowable resources 49145, more functionality may be enabled, as determined by the rules engine of the surgical hub 49150.

[0165] Making a determination whether to enable and / or disable one or more features may be performed by a rules engine, such as with respect to Figure 6 When an instrument (e.g., a new instrument) or system (e.g., a new system) is introduced, the instrument or system may register with the surgical hub 49150. The surgical hub 49150 may configure the system or surgical instrument and may notify other surgical instruments and / or systems already present in the surgical OR. Surgical instruments and surgical systems may have an identification associated therewith, which may be assigned by the surgical hub 49150 upon registration.

[0166] A device (e.g., a new device) and / or a system may request (e.g., send a request message) to the surgical hub 49150 to request permission to connect. The request message may indicate which resources 49145 are available for use by the system (e.g., the new system). A surgical instrument (e.g., a new surgical instrument) may send a request message to the surgical hub 49150 to request a connection. The surgical instrument may indicate in the request message the functions it can autonomously perform and the number of resources 49145 used for those functions.

[0167] The hub 49150 can determine whether to allow the surgical system and / or new surgical instrument to connect based on the current settings. After connecting, the surgical hub 49150 can determine whether to adjust which functions are enabled and / or disabled for the current system, and can send such a determination to the surgical instrument via a message. Based on receiving the message, the surgical instrument can follow and enable and / or disable the functions indicated by the surgical hub 49150.

[0168] Enabling and / or disabling functionality may take into account performance metrics, such as latency.The surgical hub 49150 may include an optimization module that may run a cost analysis by enabling and / or disabling variations of functionality in determining the optimal set of functionality to use for the surgical task at hand.

[0169] Fig.11 An example of matrix generator 49170 and function determination is shown. Fig.11 As shown, functional resource 49165 can be associated with a surgical instrument (e.g., device A). Functional resource 49165 can be associated with a surgical instrument (e.g., device A). Fig. 9 For example, functional resource 49165 may show a generated matrix showing the links between the functions and resources of device A, such as Fig.11 When a device is connected to a surgical hub, the device may send the function along with the resources associated with the device to the surgical hub, such as with respect to Fig.10 The resource may be a resource used to autonomously perform the function of the surgical instrument. Fig.11 As shown, function one of device A may be associated with bandwidth X. Matrix generator 49170 may combine the resources of auxiliary devices brought into the surgical hub and run optimization techniques to figure out which functions should be enabled in order to best perform the surgical task at hand.

[0170] For example, device A function one may be associated with a bandwidth resource (e.g., associated with bandwidth, power, payload work, data packet size, transmission frequency, etc.) X, and the matrix generator 49170 may input the bandwidth Z of device B, and the rule engine (e.g., which obtains the combination from the matrix generator) may determine the cost of enabling function one with the combination. In a second variation, the matrix generator 49170 may use the bandwidth X of function one with another device having a bandwidth T, and the rule engine may determine the cost associated with the combination. In an example, the matrix generator 49170 may use the bandwidth X of function one to combine with a new device having a bandwidth L introduced into the OR, and the rule engine may determine the cost of the combination. After evaluating these different variations, the surgical hub may determine (e.g., via the rule engine) whether to enable function one with a particular tested resource combination based on a cost analysis, which may be included in the analysis module. If a surgical system is introduced into an OR, the bandwidth X of function one may be used in the matrix generator 49170 along with the bandwidth of the surgical system (eg, total bandwidth), which may be used to determine whether function one may be enabled.

[0171] like Fig.11 As shown, another resource that can be used to determine whether to enable function one, function two or function three can be power. Function one of device A can remain constant, and the device associated with power resources introduced can be used in matrix generator 49170, and the cost associated with each combination can be determined and the cost can be used to determine whether to enable function one. This technology can be used for functions two and three. For example, in order to autonomously perform function two for device A, it can use bandwidth N. Matrix generator 49170 can keep bandwidth constant, while combining different variations of bandwidth of other devices to determine whether to enable function two. After the function is determined by the surgical hub or a third-party service, the surgical hub can send a message about which functions to enable to each surgical instrument in the surgical instrument, and can send a notification to the user (e.g., a surgeon). In an example, the surgical hub can send a message to the user based on the introduction or removal of a surgical instrument or surgical system, informing them to manually disable one or more functions.

[0172] The cost analysis performed by the rule engine may include a measure of the performance of the surgical task. For example, the surgical hub may run a simulation (e.g., locally or remotely via a cloud service) utilizing each of the resource combinations generated by the matrix. The simulation framework may be described in more detail in "Method for Surgical Simulation" in U.S. patent application No. US17 / 332,593, filed on May 27, 2021, the disclosure of which is incorporated herein by reference in its entirety. Performance metrics may be collected based on the simulation. For example, CPI as well as latency and throughput may be determined. A total score may be generated for each combination in the combination, and the surgical hub may determine the maximum score and use the combination of functions that results in the maximum score for the surgical task simulation. In an example, determining which combination should be used may involve the rule engine assigning weights to performance metrics and other considerations associated with the surgical task. Determining a rule may be based on one or more of the following: surgical context, function type, historical data, or resource type, such as relative to the surgical task. Fig.12 For example, situational awareness can provide the system with additional context that the device (e.g., each device) cannot obtain, which can allow the system to adapt optimally. Knowing the surgical context can provide the system with the advantage of appropriate adjustment. Historical data can show that the operation is more risky for a particular patient. Historical data can include global information and / or patient-specific information.

[0173] Fig.12 A flow diagram of an adaptively interconnected surgical system is shown. A surgical hub may be used to autonomously determine surgical functions in a surgical environment. At 49175, the surgical hub may detect a first surgical device in a surgical environment (e.g., including a processor configured to enable the first surgical device in the surgical environment). The first surgical device may include a first plurality of functions. The surgical hub may include a plurality of resources.

[0174] At 49180, the surgical hub can detect a second surgical device in the surgical environment. The second surgical device can include a second plurality of functions.

[0175] At 49185, one or more functions of the first plurality of functions for the first surgical device to perform can be determined based on the plurality of resources. One or more functions of the second plurality of functions for the second surgical device to perform can be determined based on the plurality of resources.

[0176] In an example, a surgical hub may include a plurality of ports. A first surgical device may be connected to the surgical hub via a first port of the plurality of ports. A second surgical device may be connected to the surgical hub via a second port of the plurality of ports.

[0177] Determining the one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform may involve using an optimization associated with one or more rules (e.g., Newton's method). The optimization associated with the one or more rules may be based on one or more of the following: surgical context, type of the first plurality of functions, type of the second plurality of functions, historical data, or the plurality of resources.

[0178] In an example, the surgical hub may determine a surgical context associated with the surgical environment. The surgical hub may determine the one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform based on the plurality of resources and the surgical context.

[0179] The surgical hub may send an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device. The surgical hub may detect a third surgical device in the surgical environment. The third surgical device may include a third plurality of functions. The surgical hub may adjust the one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform based on the plurality of resources and the third plurality of functions. One or more functions of the third plurality of functions for the third surgical device to perform may be determined based on the plurality of resources.

[0180] The surgical hub may determine one or more optional functions from the first plurality of functions for the first surgical device to perform and one or more optional functions from the second plurality of functions for the second surgical device to perform based on the plurality of resources. The optional functions may be selected by a user of the first surgical instrument and a user of the second surgical instrument. The surgical hub may determine one or more resources from the plurality of resources for the first surgical device and one or more resources from the plurality of resources for the second surgical device. The determined one or more resources may be used to perform the determined one or more functions.

[0181] Fig.13 An example of an overview of the rules engine 49195 is shown. In an example, the rules engine 49195 can be included in a third-party system that the surgical hub can access (e.g., using a security key). The rules engine 49195 can be connected to a third-party system such as the one shown in FIG. Fig. 9 The rule engine may assign weights based on the importance of the autonomously executable function 49200 of the surgical instrument and the resources associated with the surgical instrument. For example, when compared to function two, such as relative to Fig.11 The function A can be given a higher weight. Fig.11Function two may be given a higher weight than function three. The weights may be determined based on surgical context and / or historical data. The weights may be determined based on results collected from surgical simulations.

[0182] Weights may be assigned to performance metrics. Weights may be assigned to resources used for each function in function 49205. For example, for function one, Fig.11 As described above, bandwidth can be assigned a higher weight when compared to power. Fig.11 As described above, the power used for function two of device A may be assigned a higher weight than the bandwidth. This may depend on the type of autonomously performed function 49200. For example, the power used to autonomously perform orientation may be assigned a higher weight when compared to the power used to autonomously perform clamping.

[0183] Resources associated with each of the functions 49200 may be assigned a priority 49210 based on a weight. A high and / or low priority 49210 may be determined. Functions 49200 associated with a high priority 49210 may be selected to be enabled before functions 49200 associated with a low priority 49210. The weights may be dynamic based on the surgical context as the surgical task 49220 is performed. The rules engine 49195 may determine an updated weight for each of the functions 49200 for the surgical instrument. The weights may be updated based on a new surgical instrument or a new system introduced into the surgical OR.

[0184] Autonomous system interactions may be provided. Adjustment of autonomous options and / or processes may be based on automatic identification and / or connection between the system and an auxiliary system. Adjustment of automated functional options may be based on the interconnection of the systems. For example, a surgical hub may be connected to one or more auxiliary systems, and when the connection is established, operations that may be performed autonomously may be adjusted. Adjustments may include settings, updating operating systems, specifying communication priorities or processor load priorities, and / or establishing communications directly between auxiliary systems (e.g., two auxiliary systems).

[0185] Autonomous communication establishment and / or registration may be provided. The surgical hub and / or instrument may be associated with settings and / or configurations when starting or accessing the intelligent network. The settings and / or configurations of the surgical hub and / or instrument when starting or accessing the intelligent network may be described herein. In an example, if a device requests and / or establishes a connection to a hub-defined network, the hub may define the communication type, frequency, system security level (e.g., so that the device adjusts its security level), tagging technology, software revision, etc. for the device. During the definition of the communication exchange, the hub may adjust the expected level of mutual communication based on the situation it monitors, based on the surgery, network backlog / delay, capacity level, and / or surgery. The hub may define multiple systems within its network to communicate directly (e.g., using network bandwidth) and / or communicate through the hub (e.g., allowing it to record and adjust communication based on additional data), which may depend on the hub's capabilities and capacity at the time and for a specific surgery. For example, the hub may determine the communication mode (e.g., the best mode) between devices (e.g., two devices) and may tell the devices how to communicate with each other. The hub may send tag information to the device to, for example, re-contact the hub based on a specific change. The hub may request the device to adjust the frequency of its operation, the amount of system communication (e.g., in dB), and / or the protocol it uses based on the amount of noise and / or data detected by the hub within the OR. The hub may adjust numerous system communication instructions based on the introduction of the device, the priority of its data, or the risk of its function relative to other systems within the network to provide the highest priority or highest risk to data that interacts using the maximum capacity. The hub may define communication interactions between systems (e.g., two systems) within its system to adjust their communications. For example, the hub may detect that the signal of the device is decreasing, or that the next step in the operation uses a higher bandwidth to transmit an appropriate amount of data. The hub may enable an antenna (e.g., an additional antenna) to try and increase the signal strength, or it may tell the device to increase the signal output strength.

[0186] Connection to available networks and / or request to register as a participant in the hub control matrix may be provided. Automated identification (ID) and / or registration may be based on detected aspects of the network and / or device. Unconnected and connectable surgical devices may know the networks around them. This perception may allow selective inquiry of possible connection options. These options may be compared with a pre-existing table or a hierarchical order of networks that may have connection preferences. The device may announce its connection intention, and the network may provide information related to which networks expect it to come and / or which networks do not want it to interfere or attempt to connect. This may be based on surgical steps, surgical progress, instruments in communication (e.g., currently in communication), the arrival of duplicate devices or incompatible devices, and / or another situation. Automated adjustment of system functions may be based on parameters established by communication between systems. The parameter may be a step in surgery. For example, when establishing a connection, if the connection occurs within a certain step in surgery (e.g., only at this time), the device may adjust the system.

[0187] Automatically changing and / or updating software and / or functional testing of the assembled configuration (e.g., after the first assembly) can be used to confirm the adjustment. The software can update the configuration settings. Confirmation of the update can be used. The functional check of the configuration may include one or more of the following. The functional check may include verification of the transducer function. The functional check may include verification of the torque. The functional check may include a maximum value of the joint motion angle. The functional check may include reestablishing the device zero or home position. Shipping issues may cause the device to be out of calibration. For example, the energy device may be announced and / or detected by the hub. The hub may know the next step and / or use case of the announced device in the operation (e.g., based on factory set configuration, surgeon selection / modification / machine, or clinical database learning). The hub may automatically send configuration settings to the device for its activation.

[0188] Selective automatic communication establishment between systems within the same operating room (OR) network can be provided, which may include distributed analysis, communication and data verification performed by a commonly networked OR device. For example, the interactive communication of operating parameters between a device and a hub controlling an operation plan and / or a surgical procedure may include one or more of the following. A surgical suturing device may be preset at a cutting speed of 15 mm / sec by default at the factory. The suturing device may be powered on in the OR and may establish communication with the hub. The hub may identify the device. The hub may analyze a database and determine that the optimal speed for the operation is 5 mm / sec and a 20 second pre-compression delay. The hub may communicate with the device and send updates of the cutting speed and compression time. The device may use these parameters and check whether these values ​​are within the operating boundaries of the device system. The device may transmit back to the hub: 5 mm / sec is the lower limit of the system motor performance, which may potentially result in insufficient delivery of the sealing staple line and / or device performance degradation. The hub may obtain updated information and determine the risk profile at a higher cutting speed. An acceptable increase may be determined and transmitted back to the device. The device can verify the parameters and confirm to the hub that it is ready. One or more of the techniques described herein can be completed autonomously. If an acceptable solution cannot be achieved, the hub can alert the OR personnel. Based on the risk level of the patient, device, and surgery, the system can set an overall surgical risk level. Based on the calculated risk level, the system can complete the task autonomously, or if the risk is considered too high, the system can wait for the surgeon to confirm before completing the surgical steps. The mobile flexible endoscopic robot can be rolled into the OR for transbronchial imaging and dissection and / or a portion of the mobile operation. If the system is aware that it is operating in close proximity to the patient of other smart or digital ecosystem devices (e.g., above-the-knee retractors, staples, energy devices, etc.), the hub can establish communication with the device when the hub control system is close to the other systems to help avoid collisions or unintentional tissue tension situations.

[0189] Autonomous identification of trackable and / or identifiable elements may be system-wide. Automatic identification of wearable systems may include one or more of the following. The automatic identification may include automatic determination of active users of wearable devices, which may involve automated verification and / or confirmation of identity using previous data sets to determine users and / or (e.g., using previous biomarkers—fingerprints to verify users). The automatic identification may include classification of user characteristics (such as work (e.g., skills, competencies, trained equipment, etc.)), determination of the use of monitored data, and / or prioritization of monitored data relative to other data sources processed simultaneously by the system. Automatic definition of reporting frequency, registration, and / or data collection parameters. Instrument, tool, medical inventory identification may be provided, which may include detection of brand, model, and / or serial number. Automatic determination of system limits or ranges may include physical and / or functional boundaries of which systems the hub is allowed to control, acceptable ranges of communication establishment, and / or tracking capabilities.

[0190] The following is a list of numbered embodiments that may or may not be claimed:

[0191] 1. A surgical hub for autonomously determining surgical device functions to enable for a surgical procedure, the surgical hub comprising:

[0192] A processor configured to:

[0193] detecting a first surgical device in a surgical environment, wherein the first surgical device includes a first plurality of functions, and wherein the surgical hub includes a plurality of resources;

[0194] detecting a second surgical device in the surgical environment, wherein the second surgical device includes a second plurality of functions;

[0195] receiving or deriving instructions for the surgical procedure to be performed, the surgical procedure comprising a plurality of surgical tasks; and

[0196] Determining one or more functions of the first plurality of functions of the first surgical device to be enabled to perform a surgical task of the plurality of surgical tasks, and determining one or more functions of the second plurality of functions of the second surgical device to be enabled to perform a surgical task of the plurality of surgical tasks, wherein the determination of which functions to enable is based on the plurality of resources.

[0197] The plurality of resources of the hub are limited, and the enabled instrument functions require the use of the hub's resources in order to perform one or more surgical tasks. The hub advantageously autonomously coordinates the allocation of limited hub resources to the instrument functions required to perform one or more tasks of the surgical procedure to enable those instrument functions.

[0198] The plurality of resources may include computer resources, which may include power and bandwidth. Functionality may be associated with a resource range (e.g., a power and / or bandwidth range) under which the surgical instrument is able to autonomously perform the surgical task. In other words, a resource may be a computer processing requirement for a function to autonomously perform the surgical task.

[0199] 2. The surgical hub of embodiment 1, wherein the processor is further configured to:

[0200] sending an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device; or

[0201] A signal is transmitted to the first device and / or the second device to enable the determined one or more functions.

[0202] Advantageously, in addition to determining the functions to be enabled, the hub can also transmit signals to the first device and the second device to enable the determined functions, thereby preparing the devices and their functions for the surgical procedure or tasks of the surgical procedure (since the enabling of functions can be done dynamically based on the stage in the surgery). Alternatively, the hub can transmit messages to users of the devices indicating which functions to enable.

[0203] 3. A surgical hub according to embodiment 1 or 2, wherein the enabled function is a function for autonomously performing the surgical task.

[0204] The surgical hub is configured to be able to determine which tasks will be performed autonomously by instrument functions and which tasks will be performed manually by the surgeon, wherein the allocation of resources is a factor used to determine which tasks should be performed autonomously or manually and therefore which functions should be enabled or disabled.

[0205] 4. A surgical hub according to any of the preceding embodiments, wherein these functions are functions that the device is capable of performing.

[0206] 5. A surgical hub according to embodiment 4, wherein the first device and / or the second device is an endoscopic cutting and closing device, and the first plurality of functions and / or the second plurality of functions include one or more of the following: controlling energy source, cutting, suturing, knob orientation, body orientation, body positioning, clamping, anvil jaw force, and reloading alignment slot management.

[0207] 6. A surgical hub according to any of the preceding embodiments, wherein the processor is further configured to be capable of receiving or determining indications of the first plurality of functions and the second plurality of functions.

[0208] The user may manually input the functionality of the device. Alternatively or in addition, the hub may be able to determine the functionality by knowing the device. For example, the device may be coupled to the hub and share an identifier that identifies the device, and from the identifier, the hub may be able to determine the functionality of the device (e.g., via a lookup table stored in memory or via an external memory / server).

[0209] 7. A surgical hub according to any of the preceding embodiments, wherein the processor is configured to detect the first device and the second device when the first device and the second device send a request to couple to the hub, or when the first device and the second device are coupled to the hub.

[0210] 8. A surgical hub according to any of the preceding embodiments, wherein the surgical hub comprises a plurality of ports, and wherein the first surgical device is connected to the surgical hub via a first port among the plurality of ports, and wherein the second surgical device is connected to the surgical hub via a second port among the plurality of ports.

[0211] 9. A surgical hub according to any of the preceding embodiments, wherein determining which functions should be enabled includes using optimizations associated with one or more rules.

[0212] 10. A surgical hub according to embodiment 9, wherein the optimization associated with one or more rules is based on at least one of: the surgical context, the type of the first plurality of functions, the type of the second plurality of functions, historical data, or the plurality of resources.

[0213] Some functions may be given a higher priority, for example based on the complexity of the tasks performed by those functions or based on the surgical results of the surgical task when performed manually and when performed automatically (which may be based on historical data from past surgeries that included that task performed by that function). For example, historical data may show that the surgical results of a surgical task are improved when a function performs the task autonomously, and therefore, the optimization may prioritize enabling that function to autonomously perform the surgical task. By optimizing in this manner, the results of the surgical procedure may generally be improved.

[0214] For example, the outcome of tissue resection may correspond to the integrity of the seal line, wherein less seal leakage equates to a more positive outcome than more seal leakage. The outcome of the surgery may also correspond to the number of difficulties that occurred during the surgical procedure, which may include, for example, instrument failure, seal line leakage, staple line misfiring, etc.

[0215] 11. The surgical hub according to any preceding embodiment, wherein the processor is further configured to:

[0216] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0217] The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are adjusted based on the plurality of resources and the third plurality of functions.

[0218] Advantageously, the surgical hub can dynamically determine which functions of a device should be enabled or disabled while a surgical procedure is being performed and / or when a device is brought into the surgical environment and / or connected to the hub.

[0219] 12. A surgical hub according to embodiment 11, wherein the adjustment includes disabling a previously enabled function.

[0220] By dynamically enabling and disabling functionality, it is possible to reallocate resources when a new device is detected or while a surgical procedure is ongoing.

[0221] 13. The surgical hub according to any preceding embodiment, wherein the processor is further configured to:

[0222] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0223] One or more functions of the third plurality of functions for the third surgical device to perform are determined based on the plurality of resources.

[0224] 14. The surgical hub of any preceding embodiment 1, wherein the processor is further configured to:

[0225] One or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

[0226] 15. A surgical hub according to embodiment 14, wherein these optional functions are selected by the user of the first surgical instrument and the user of the second surgical instrument.

[0227] 16. A method for autonomously determining a surgical device function to enable for a surgical procedure, the method comprising:

[0228] detecting a first surgical device in a surgical environment, wherein the first surgical device includes a first plurality of functions, and wherein the surgical hub includes a plurality of resources;

[0229] detecting a second surgical device in the surgical environment, wherein the second surgical device includes a second plurality of functions;

[0230] receiving or deriving instructions for the surgical procedure to be performed, the surgical procedure comprising a plurality of surgical tasks; and

[0231] determining one or more functions of the first plurality of functions of the first surgical device to be enabled to perform a surgical task of the plurality of surgical tasks, and determining one or more functions of the second plurality of functions of the second surgical device to be enabled to perform a surgical task of the plurality of surgical tasks, wherein the determination is based on the plurality of resources.

[0232] The plurality of resources of the hub are limited, and the enabled instrument functions require use of the resources of the hub in order to perform one or more surgical tasks. The method advantageously autonomously coordinates the allocation of limited hub resources to instrument functions required to perform one or more tasks of the surgical procedure to enable those instrument functions.

[0233] The plurality of resources may include computer resources, which may include power and bandwidth. Functionality may be associated with a resource range (e.g., a power and / or bandwidth range) under which the surgical instrument is able to autonomously perform the surgical task. In other words, a resource may be a computer processing requirement for a function to autonomously perform the surgical task.

[0234] 17. The method according to embodiment 16, further comprising:

[0235] sending an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device; or

[0236] A signal is transmitted to the first device and / or the second device to enable the determined one or more functions.

[0237] Advantageously, in addition to determining the functions to be enabled, the method also includes transmitting a signal to the first device and the second device to enable the determined functions, thereby preparing the devices and their functions for the surgical procedure or the tasks of the surgical procedure (as the enabling of functions may be done dynamically based on the stage in the surgery). Alternatively, the method includes transmitting a message to a user of the device indicating which functions are to be enabled.

[0238] 18. A method according to embodiment 16 or 17, wherein the enabled function is a function for autonomously performing the surgical task.

[0239] The method is configured to be able to determine which tasks are to be performed autonomously by instrument functions and which tasks are to be performed manually by the surgeon, wherein the allocation of resources is a factor used to determine which tasks should be performed autonomously or manually and therefore which functions should be enabled or disabled.

[0240] 19. A method according to any one of embodiments 16 to 18, wherein these functions are functions that the device is capable of performing.

[0241] 20. A method according to embodiment 19, wherein the first device and / or the second device is an endoscopic cutting and closing device, and the multiple first functions and / or the multiple second functions include one or more of the following: controlling energy source, cutting, suturing, knob orientation, body orientation, body positioning, clamping, anvil jaw force, and reloading alignment slot management.

[0242] 21. The method according to any one of embodiments 16 to 20 also includes receiving or determining indications of the first plurality of functions and the second plurality of functions.

[0243] 22. A method according to any one of embodiments 16 to 21, wherein detecting the first device and the second device includes detecting when the first device and the second device send a request to couple to the hub, or detecting when the first device and the second device are already coupled to the hub.

[0244] 23. A method according to any one of embodiments 16 to 22, wherein the surgical hub comprises a plurality of ports, and wherein the first surgical device is connected to the surgical hub via a first port of the plurality of ports, and wherein the second surgical device is connected to the surgical hub via a second port of the plurality of ports.

[0245] 24. A method according to any one of embodiments 16 to 23, wherein determining which functions should be enabled includes using optimizations associated with one or more rules.

[0246] 25. A method according to embodiment 24, wherein the optimization associated with one or more rules is based on at least one of: the surgical context, the type of the first plurality of functions, the type of the second plurality of functions, historical data, or the plurality of resources.

[0247] Some functions may be given a higher priority, for example based on the complexity of the tasks performed by those functions or based on the surgical results of the surgical task when performed manually and when performed automatically (which may be based on historical data from past surgeries that included that task performed by that function). For example, historical data may show that the surgical results of a surgical task are improved when a function performs the task autonomously, and therefore, the optimization may prioritize enabling that function to autonomously perform the surgical task. By optimizing in this manner, the results of the surgical procedure may generally be improved.

[0248] For example, the outcome of tissue resection may correspond to the integrity of the seal line, wherein less seal leakage equates to a more positive outcome than more seal leakage. The outcome of the surgery may also correspond to the number of difficulties that occurred during the surgical procedure, which may include, for example, instrument failure, seal line leakage, staple line misfiring, etc.

[0249] 26. The method according to any one of embodiments 16 to 25, further comprising:

[0250] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0251] The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are adjusted based on the plurality of resources and the third plurality of functions.

[0252] Advantageously, the method includes dynamically determining which functions of the device should be enabled or disabled while a surgical procedure is being performed and / or when the device is brought into a surgical environment and / or connected to a hub.

[0253] 27. A method according to embodiment 26, wherein the adjustment includes disabling a previously enabled function.

[0254] By dynamically enabling and disabling functionality, it is possible to reallocate resources when a new device is detected or while a surgical procedure is ongoing.

[0255] 28. The method according to any one of embodiments 16 to 27, further comprising:

[0256] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0257] One or more functions of the third plurality of functions for the third surgical device to perform are determined based on the plurality of resources.

[0258] 29. The method according to any preceding embodiment, further comprising:

[0259] One or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

[0260] 30. A method according to embodiment 29, wherein these optional functions are selected by the user of the first surgical instrument and the user of the second surgical instrument.

[0261] 31. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of embodiments 16 to 30.

[0262] 32. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform a method according to any one of embodiments 16 to 30.

[0263] The following are numbered aspects of the disclosure that may or may not be claimed:

[0264] 1. A surgical hub for autonomously determining surgical functions in a surgical environment, the surgical hub comprising:

[0265] A processor configured to:

[0266] detecting a first surgical device in a surgical environment, wherein the first surgical device includes a first plurality of functions, and wherein the surgical hub includes a plurality of resources;

[0267] detecting a second surgical device in the surgical environment, wherein the second surgical device includes a second plurality of functions; and

[0268] One or more functions of the first plurality of functions for the first surgical device to perform and one or more functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

[0269] 2. A surgical hub according to aspect 1, wherein the surgical hub comprises a plurality of ports, and wherein the first surgical device is connected to the surgical hub via a first port among the plurality of ports, and wherein the second surgical device is connected to the surgical hub via a second port among the plurality of ports.

[0270] 3. A surgical hub according to aspect 1, wherein determining the one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform includes using optimization associated with one or more rules.

[0271] 4. A surgical hub according to aspect 3, wherein the optimization associated with one or more rules is based on at least one of: the surgical context, the type of the first plurality of functions, the type of the second plurality of functions, historical data, or the plurality of resources.

[0272] 5. The surgical hub of aspect 1, wherein the processor is further configured to:

[0273] determining the surgical context associated with the surgical setting; and

[0274] The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources and the surgical context.

[0275] 6. The surgical hub of aspect 1, wherein the processor is further configured to:

[0276] An indication of the determined one or more functions is sent to a user of the first surgical device and a user of the second surgical device.

[0277] 7. The surgical hub of aspect 1, wherein the processor is further configured to:

[0278] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0279] The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are adjusted based on the plurality of resources and the third plurality of functions.

[0280] 8. The surgical hub of aspect 2, wherein the processor is further configured to:

[0281] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0282] One or more functions of the third plurality of functions for the third surgical device to perform are determined based on the plurality of resources.

[0283] 9. The surgical hub of aspect 1, wherein the processor is further configured to:

[0284] One or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

[0285] 10. The surgical hub of aspect 9, wherein the optional functions are selected by a user of the first surgical instrument and a user of the second surgical instrument.

[0286] 11. The surgical hub of aspect 1, wherein the processor is further configured to:

[0287] One or more resources of the plurality of resources for the first surgical device and one or more resources of the plurality of resources for the second surgical device are determined, wherein the determined one or more resources are used to perform the determined one or more functions.

[0288] 12. A method for autonomously determining a surgical function in a surgical environment, the method comprising:

[0289] detecting a first surgical device in a surgical environment, wherein the first surgical device includes a first plurality of functions, and wherein the surgical hub includes a plurality of resources;

[0290] detecting a second surgical device in the surgical environment, wherein the second surgical device includes a second plurality of functions; and

[0291] One or more functions of the first plurality of functions for the first surgical device to perform and one or more functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

[0292] 13. A method according to aspect 12, wherein the surgical hub includes a plurality of ports, and wherein the first surgical device is connected to the surgical hub via a first port of the plurality of ports, and wherein the second surgical device is connected to the surgical hub via a second port of the plurality of ports.

[0293] 14. A method according to aspect 12, wherein determining the one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform includes using optimization associated with one or more rules.

[0294] 15. The method of aspect 14, wherein the optimization associated with one or more rules is based on at least one of: surgical context, type of the first plurality of functions, type of the second plurality of functions, historical data, or the plurality of resources.

[0295] 16. The method according to aspect 12, further comprising:

[0296] determining the surgical context associated with the surgical setting; and

[0297] The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources and the surgical context.

[0298] 17. The method according to aspect 12, further comprising:

[0299] An indication of the determined one or more functions is sent to a user of the first surgical device and a user of the second surgical device.

[0300] 18. The method according to aspect 12, further comprising:

[0301] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0302] The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are adjusted based on the plurality of resources and the third plurality of functions.

[0303] 19. The method according to aspect 13, further comprising:

[0304] detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and

[0305] One or more functions of the third plurality of functions for the third surgical device to perform are determined based on the plurality of resources.

[0306] 20. The method according to aspect 12, further comprising:

[0307] One or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

[0308] 21. The method of aspect 20, wherein the optional functions are selected by a user of the first surgical instrument and a user of the second surgical instrument.

Claims

1. A surgical hub for autonomously determining surgical device functions to enable for a surgical procedure, the surgical hub comprising: A processor configured to: detecting a first surgical device in a surgical environment, wherein the first surgical device comprises a first plurality of functions, and wherein the surgical hub comprises a plurality of resources; detecting a second surgical device in the surgical environment, wherein the second surgical device includes a second plurality of functions; receiving or deriving instructions for the surgical procedure to be performed, the surgical procedure comprising a plurality of surgical tasks; and Determine one or more functions of the first plurality of functions of the first surgical device to be enabled to perform a surgical task among the plurality of surgical tasks, and determine one or more functions of the second plurality of functions of the second surgical device to be enabled to perform a surgical task among the plurality of surgical tasks, wherein the determination of which functions to enable is based on the plurality of resources.

2. The surgical hub according to claim 1, wherein: The processor is further configured to: sending an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device; or A signal is transmitted to the first device and / or the second device to enable the determined one or more functions.

3. The surgical hub according to claim 1 or 2, wherein: The enabled function is the function for autonomously performing the surgical task.

4. A surgical hub according to any preceding claim, wherein: The functions are functions that the device is capable of performing.

5. The surgical hub of claim 4, wherein: The first device and / or the second device is an endoscopic cutting and closing device, and the first plurality of functions and / or the second plurality of functions include one or more of the following: controlling energy source, cutting, suturing, knob orientation, body orientation, body positioning, clamping, anvil jaw force, and reloading alignment slot management.

6. A surgical hub according to any preceding claim, wherein: The processor is further configured to be capable of receiving or determining an indication of the first plurality of functions and the second plurality of functions.

7. A surgical hub according to any preceding claim, wherein: The processor is configured to be capable of detecting the first device and the second device when the first device and the second device send a request to couple to the hub, or when the first device and the second device are coupled to the hub.

8. A surgical hub according to any preceding claim, wherein: The surgical hub comprises a plurality of ports, and wherein the first surgical device is connected to the surgical hub via a first port of the plurality of ports, and wherein the second surgical device is connected to the surgical hub via a second port of the plurality of ports.

9. A surgical hub according to any preceding claim, wherein: Determining which features should be enabled includes using optimizations associated with one or more rules.

10. The surgical hub of claim 9, wherein: The optimization associated with one or more rules is based on at least one of: a surgical context, a type of the first plurality of functions, a type of the second plurality of functions, historical data, or the plurality of resources.

11. A surgical hub according to any preceding claim, wherein: The processor is further configured to: detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are adjusted based on the plurality of resources and the third plurality of functions.

12. The surgical hub of claim 11, wherein: The adjusting includes disabling a previously enabled function.

13. A surgical hub according to any preceding claim, wherein: The processor is further configured to: detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and One or more functions of the third plurality of functions for the third surgical device to perform are determined based on the plurality of resources.

14. The surgical hub of any preceding claim 1, wherein: The processor is further configured to: One or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

15. The surgical hub of claim 14, wherein: The optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument.

16. A method for autonomously determining a surgical device function to enable for a surgical procedure, the method comprising: detecting a first surgical device in a surgical environment, wherein the first surgical device includes a first plurality of functions, and wherein the surgical hub includes a plurality of resources; detecting a second surgical device in the surgical environment, wherein the second surgical device includes a second plurality of functions; receiving or deriving instructions for the surgical procedure to be performed, the surgical procedure comprising a plurality of surgical tasks; and and determining one or more functions of the first plurality of functions of the first surgical device to be enabled to perform a surgical task of the plurality of surgical tasks, and determining one or more functions of the second plurality of functions of the second surgical device to be enabled to perform a surgical task of the plurality of surgical tasks, wherein the determination is based on the plurality of resources.

17. The method according to claim 16, further comprising: sending an indication of the determined one or more functions to a user of the first surgical device and a user of the second surgical device; or A signal is transmitted to the first device and / or the second device to enable the determined one or more functions.

18. The method according to claim 16 or 17, wherein: The enabled function is the function for autonomously performing the surgical task.

19. The method according to any one of claims 16 to 18, wherein: The functions are functions that the device is capable of performing.

20. The method according to claim 19, wherein: The first device and / or the second device is an endoscopic cutting and closing device, and the plurality of first functions and / or the plurality of second functions include one or more of: controlling energy source, cutting, suturing, knob orientation, body orientation, body positioning, clamping, anvil jaw force, and reloading alignment slot management.

21. The method of any one of claims 16 to 20, further comprising receiving or determining an indication of the first plurality of functions and the second plurality of functions.

22. The method according to any one of claims 16 to 21, wherein: Detecting the first device and the second device includes detecting when the first device and the second device send a request to couple to the hub, or detecting when the first device and the second device are already coupled to the hub.

23. The method according to any one of claims 16 to 22, wherein: The surgical hub comprises a plurality of ports, and wherein the first surgical device is connected to the surgical hub via a first port of the plurality of ports, and wherein the second surgical device is connected to the surgical hub via a second port of the plurality of ports.

24. The method according to any one of claims 16 to 23, wherein: Determining which features should be enabled includes using optimizations associated with one or more rules.

25. The method according to claim 24, wherein: The optimization associated with one or more rules is based on at least one of: a surgical context, a type of the first plurality of functions, a type of the second plurality of functions, historical data, or the plurality of resources.

26. The method according to any one of claims 16 to 25, further comprising: detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and The one or more functions of the first plurality of functions for the first surgical device to perform and the one or more functions of the second plurality of functions for the second surgical device to perform are adjusted based on the plurality of resources and the third plurality of functions.

27. The method according to claim 26, wherein: The adjusting includes disabling a previously enabled function.

28. The method according to any one of claims 16 to 27, further comprising: detecting a third surgical device in the surgical environment, wherein the third surgical device includes a third plurality of functions; and One or more functions of the third plurality of functions for the third surgical device to perform are determined based on the plurality of resources.

29. The method of any preceding claim, further comprising: One or more optional functions of the first plurality of functions for the first surgical device to perform and one or more optional functions of the second plurality of functions for the second surgical device to perform are determined based on the plurality of resources.

30. The method of claim 29, wherein: The optional function is selected by a user of the first surgical instrument and a user of the second surgical instrument.

31. A computer program comprising instructions which, when a computer executes the program, cause the computer to perform the method according to any one of claims 16 to 30.

32. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 16 to 30.

Images