Adaptable operating range for surgical devices

Abstract

A device, such as a computing device or a surgical device, may receive surgical operation data associated with a surgical operation. Based on the surgical procedure data, the device may identify a surgical device to be used for the surgical procedure and / or one or more surgical steps associated with the surgical procedure. Based on the identified surgical device, the one or more surgical steps, and / or the surgical operational data, the device may determine an allowable operational range for controlling the surgical device for the surgical procedure. The device may receive an adjustment input configuration to control the surgical device to perform a surgical step. The device may determine whether the adjustment input configuration is outside the determined allowable operating range. Based on a determination that the adjustment input configuration is outside the determined allowable operating range, the device may prevent the adjustment input configuration from controlling the surgical device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is related to the following concurrently filed patent applications, the contents of each of which are incorporated herein by reference:

[0003] ●U.S. patent application with attorney case number END9438USNP1, entitled “A Method for Advanced Algorithm Support.” Background Art

[0004] Patient care is often improved when tailored to the individual. Every person has different needs, so surgical and interventional solutions centered on each patient's unique journey can represent an effective, groundbreaking path to healing. At the same time, the high cost of patient care, particularly surgical procedures, often drives a focus on conservative, repeatable activities.

[0005] Innovative medical technologies, such as advanced surgical support computing systems and intelligent surgical instruments, can improve patient care methods and address the specific needs of healthcare providers.

[0006] The increasing availability of data and computing resources has made non-traditional algorithms, such as machine learning algorithms, a particular technological opportunity in healthcare systems. However, incorporating such non-traditional algorithms into any medical technology presents many challenges.

[0007] A computing system, which may include a surgical device and / or a surgical hub, may be configured with machine learning to train data and provide allowable control input ranges for controlling one or more surgical devices. The allowable control input ranges may provide recommended input ranges for controlling one or more surgical devices for a healthcare provider (HCP) during a surgical procedure. Machine learning may be used to improve data, such as allowable control input ranges for surgical devices. However, training machine learning may be time-consuming and inconvenient. Summary of the Invention

[0008] According to one embodiment of the present invention, a method performed by a processor is provided, the method comprising: receiving surgical operation data associated with a surgical operation, wherein the surgical operation data includes data associated with at least one of a patient, an operator, or a surgical device to be used for the surgical operation; identifying a surgical step associated with the surgical operation based on the surgical operation data; determining an allowable operating range associated with the surgical device based on the surgical step and the surgical operation data, wherein the allowable operating range is an operating range for controlling the surgical device to perform the surgical step; receiving an adjustment input configuration, wherein the adjustment input configuration is configured to be able to control the surgical device to perform the surgical step; determining whether the adjustment input configuration is within the determined allowable operating range; and preventing the adjustment input configuration from controlling the surgical device based on a determination that the adjustment input configuration is not within the allowable operating range.

[0009] In this way, rather than relying solely on the operator's past experience with surgical procedures (and specifically, surgical steps), surgical procedure data can be utilized to determine the permissible operating range of the surgical device. By preventing adjustment input configurations outside the permissible operating range, it is possible to prevent the use of inappropriate or unsafe operating inputs on the surgical device to perform surgical steps.

[0010] In an embodiment, the data associated with the patient may include body mass index (BMI), height, weight, and / or medical history. In an embodiment, the data associated with the operator may include the number of surgical procedures and / or surgical steps performed, the success rate and / or failure rate for performing the surgical procedures and / or surgical steps, the preferred input configuration for performing the surgical procedures and / or surgical steps using the surgical device, the possibility of adjusting the input configuration for performing the surgical procedures and / or surgical steps, and / or data associated with other operators who have performed the same surgical procedures and / or surgical steps.

[0011] In an embodiment, the surgical operation data is received from a surgical device. In an embodiment, the surgical operation data is received from a local database, an edge and / or fog network, and / or a cloud network. The surgical device, the local database, the edge and / or fog network, and / or the cloud network may collect the surgical operation data.

[0012] In an embodiment, surgical procedure data (i.e., data associated with at least one of a patient, an operator, or a surgical device to be used in a surgical procedure) may be used to identify a surgical device to be used in a surgical procedure and surgical steps associated with the surgical procedure by searching a database of planned surgical procedures using the surgical procedure data. In an embodiment, the identified surgical device may be a particular class of surgical device, a particular model of surgical device, or may be a particular surgical device.

[0013] In an embodiment, preventing adjustment of the input configuration includes not allowing the input configuration to fall outside an allowable operating range.

[0014] Determining the allowable operating range associated with the surgical device may include inputting surgical operation data into a machine learning (ML) model, wherein the ML model is trained to determine the allowable operating range of the surgical device for performing the surgical step using historical surgical operation data.

[0015] In this way, the permissible operating range is not only set by the operator, but is also set using historical surgical data (including historical surgical data from many different operators). Furthermore, the ML model may be able to use the success of surgical steps as part of the training data (i.e., surgical outcomes) and thus generalize previously used operating ranges to establish permissible operating ranges that improve surgical outcomes.

[0016] In an embodiment, a supervised machine learning algorithm may be used to determine the permissible operating range. This type of algorithm is trained on labeled data, where the correct output is provided for each input. By learning the relationship between the input data and the desired outcome, the algorithm can make predictions about the permissible operating range of the device in different surgical scenarios.

[0017] Determining the allowable operating range associated with the surgical device may include retrieving the allowable operating range of the surgical device for performing the surgical step from historical surgical operation data.

[0018] In this manner, it is possible to allow the operating range to be set not only by the operator, but also using historical surgical data (including historical surgical data from many different operators).

[0019] The method may further include identifying a surgical device to be used for the surgical procedure based on the surgical procedure data, wherein determining the allowable operating range associated with the surgical device is further based on the surgical device.

[0020] In this way, any unique characteristics of the surgical device can be taken into account within the allowable operating range. For example, if the surgical device is not calibrated correctly, the allowable operating range can compensate for this.

[0021] In an embodiment, the identified surgical device can be a specific class of surgical devices, a specific model of surgical devices, or can be a specific surgical device. In an embodiment, the allowable range of operation can depend on the specific class of surgical devices, the specific model of surgical devices, and / or the specific surgical device.

[0022] The method may further include sending an alert to an operator, wherein the alert indicates that the adjustment input configuration is not within an allowable operating range, and wherein the alert further indicates a surgical risk associated with adjusting the input to the adjustment input configuration that is not within the allowable operating range.

[0023] In this way, the operator is informed of the surgical risks of proceeding with the adjustment input configuration.This means that the operator cannot accidentally use an adjustment input configuration that has a surgical risk associated with it.

[0024] The adjustment input configuration may be received from an operator or generated by an ML model, wherein the ML model is trained using historical surgical operation data to determine the adjustment input configuration as the input to control the surgical device to perform the surgical step.

[0025] The method may further include determining whether the adjustment input configuration is from the operator; and based on the determination that the adjustment input configuration is from the operator, sending a message to the operator to confirm whether to adjust the input to control the surgical device using the adjustment input configuration that is not within the allowable operating range.

[0026] In this way, the operator is provided with a check as to whether the adjustment input configuration is intended.Thus, the chance of starting a surgical procedure with an unintended adjustment input configuration is significantly reduced.

[0027] In an embodiment, the surgical device has a display configured to display a message to the operator. In an embodiment, the message may request the operator to confirm whether to adjust the input by activating a button on the surgical device. In an embodiment, the display may be a touch screen, and the message may request the operator to confirm whether to adjust the input using the touch screen.

[0028] The method may further include receiving a feedback message from the operator confirming use of the adjustment input configuration as the input to control the surgical device; and allowing the adjustment input configuration to be used as the input to control the surgical device to perform the surgical step.

[0029] In an embodiment, the operator can send a feedback message by activating a button on the surgical device.In an embodiment, the operator can send a feedback message using the touch screen of the device.

[0030] In an embodiment, allowing adjustment of the input configuration as input to control the surgical device to perform the surgical procedure includes changing the input configuration outside of an allowable operating range.

[0031] The method may further include sending a request message to an operator to confirm whether to revise the allowable operating range based on the adjustment input configuration.

[0032] In this way, feedback from the operator can be used to improve the allowable operating range.

[0033] In an embodiment, once the operator confirms the revised allowable operating range, the revised allowable operating range is stored in a database of surgical operation data for use in future surgical operations. In an embodiment, the revised allowable operating range can be provided to an ML model that determines the allowable operating range to further train the ML model.

[0034] The method may further include determining whether the adjustment input configuration is generated by a machine learning (ML) model, and based on the determination that the adjustment input configuration is generated by the ML model, sending ML model metadata to an operator.

[0035] In this manner, the ML model metadata can be used by an operator to determine whether the adjusted input configuration generated by the ML model is reasonable. For example, the operator may be able to use ML model metadata, such as information about how often other operators have used the adjusted input configuration for a surgical procedure, or the surgical success rate of surgical procedures using the adjusted input configuration, to determine whether the adjusted input configuration generated by the ML model should be used.

[0036] In an embodiment, if an operator determines an adjustment, the input configuration generated by the ML model may be provided to the ML model that determines the allowable operating range for further training.

[0037] The ML model metadata may include at least one of: frequency information of other operators using the adjusted input configuration for the surgical step, or a surgical success rate of surgical operations using the adjusted input configuration.

[0038] The method may further include adjusting an input to control the surgical device to perform the surgical step using the adjustment input configuration based on a determination that the adjustment input configuration is within the allowable operating range.

[0039] According to another embodiment of the present invention, a surgical device is provided, comprising: a processor configured to perform any of the methods described above.

[0040] According to another embodiment of the present invention, there is provided a device (eg, a non-surgical device) comprising a processor configured to perform any of the methods described above.

[0041] According to another embodiment of the present invention, there is provided a computer program which, when executed by a processor, causes the processor to perform any one of the methods described above.

[0042] Disclosed herein are methods, systems, and apparatus for a device, such as a computing device or a surgical device, to determine an allowable operating range for controlling an input associated with a surgical device. The device may use data from a machine learning (ML) model to determine an allowable operating range associated with a surgical device. The device may utilize data from the ML model to improve an artificial intelligence algorithm, which may reduce iterations for training the artificial intelligence algorithm and / or may make training the machine learning algorithm less time-consuming. An adaptive learning algorithm may be used to aggregate one or more data streams. An adaptive learning algorithm may be used to generate and / or determine metadata from a data set. Adaptive learning may be used to determine one or more improvements from a previous machine learning analysis. Improvements in the collection and / or processing of data feeds may be used to determine an allowable operating range, for example, to control one or more surgical devices.

[0043] The device may receive surgical procedure data associated with a surgical procedure. The surgical procedure data may be or may include information associated with the surgical procedure. For example, the surgical procedure data may be or may include at least one of patient information, healthcare professional (HCP) information, or information associated with a surgical device to be used in the surgical procedure.

[0044] Based on the surgical operation data, the device may identify a surgical device to be used for the surgical operation and / or one or more surgical steps associated with the surgical operation. Based on the identified surgical device, the identified surgical steps and / or the received surgical operation data, the device may determine an allowable operating range associated with controlling the surgical device. For example, the device may feed the identified surgical device, one or more surgical steps and / or surgical operation data to an ML model and train the model. The device may use data from the ML trained model and determine an allowable operating range. The allowable operating range may be a range of control inputs used to control the surgical device to perform the surgical step. For example, the allowable operating range may have an upper operating range and a lower operating range, where the surgical device is safely (e.g., recommended) operated, for example, based on an ML trained model using an ML process and / or ML algorithm.

[0045] The device may receive an adjustment input configuration. The adjustment input configuration may provide input to control the surgical device. The device may determine whether the adjustment input configuration is within or outside an allowable operating range. Based on a determination that the adjustment input configuration is outside the determined allowable operating range, the device may prevent the adjustment input configuration from controlling the surgical device (e.g., to prevent inadvertent input and / or avoid catastrophic consequences). The device may send an alert (e.g., an alert message) to the HCP notifying them that the adjustment input configuration has been received and is outside the determined allowable operating range.

[0046] This Summary is provided in a simplified form to introduce some concepts that are further described in the Detailed Description herein. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Other features are described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a block diagram of a computer-implemented surgical system.

[0048] Figure 2 An example surgical system is shown in a surgical operating room.

[0049] Figure 3 Example surgical hubs paired with various systems are illustrated.

[0050] Figure 4 A surgical data network is illustrated having a set of communicating surgical hubs configured to be able to connect with a set of sensing systems, an environmental sensing system, a set of devices, etc.

[0051] Figure 5 A logic diagram of a control system for a surgical instrument is illustrated.

[0052] Figure 6 An example surgical system is shown that includes a handle having a controller and a motor, an adapter releasably coupled to the handle, and a loading unit releasably coupled to the adapter.

[0053] 7A to 7D An example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and an illustration of surgical information in the context of surgery are shown, respectively.

[0054] Figure 8A and Figure 8B An example supervised learning framework and an example unsupervised learning framework are shown respectively.

[0055] Figure 9 A flow chart illustrating a process by which a device, such as a computing device, determines an allowable operating range to control a surgical device.

[0056] Figure 10 A flow chart illustrating a process for a device, such as a surgical device, to determine an allowable operating range for controlling the surgical device.

[0057] Figure 11 A computing device that determines an allowable operating range associated with a surgical device is illustrated.

[0058] Figure 12A computing device configured to adjust an allowable operating range associated with a surgical device based on adjustment input from a healthcare professional is illustrated.

[0059] Figure 13 A computing device is illustrated that receives an adjustment input configuration outside of an allowable operating range, wherein the adjustment input configuration is from machine learning (ML) training data and / or an ML algorithm. DETAILED DESCRIPTION

[0060] Figure 1 1 is a block diagram of a computer-implemented surgical system 100. An example surgical system, such as surgical system 100, may include one or more surgical systems (e.g., surgical subsystems) 102, 103, 104. For example, surgical system 102 may include a computer-implemented interactive surgical system. For example, surgical systems 102, 103, 104 may include a surgical computing system, such as a surgical hub 106 and / or a computing device 116, that communicates with a cloud computing system 108. Cloud computing system 108 may include a cloud server 109 and a cloud storage unit 110.

[0061] The surgical systems 102, 103, 104 can each be a computer-enabled surgical instrument and a surgical device. For example, the surgical systems 102, 103, 104 can include a wearable sensing system 111, a human-machine interface system 112, a robotic system 113, one or more intelligent devices 114, an environmental sensing system 115, and / or the like. The wearable sensing system 111 can include one or more devices for sensing various aspects of the status and activity of individuals within the surgical environment. For example, the wearable sensing system 111 can include a healthcare provider sensing system and / or a patient sensing system.

[0062] The human-machine interface system 112 may include devices that enable an individual to interact with the surgical systems 102, 103, 104 and / or the cloud computing system 108. The human-machine interface system 112 may include human-machine interface devices.

[0063] The robotic system 113 may include a surgical robotic device, such as a surgical robot. The robotic system 113 may perform robotic surgery. The robotic system 113 may receive information, settings, programming, control, etc. from the surgical hub 106. For example, the robotic system 113 may send data, such as sensor data, feedback information, video information, operation logs, and the like, to the surgical hub 106.

[0064] The environment sensing system 115 may include, for example, one or more devices for measuring one or more environmental attributes, such as Figure 2 As further described. The robotic system 113 may include a plurality of devices for performing surgical procedures, such as Figure 2 as further described.

[0065] The surgical system 102 can communicate with a remote server 109, which can be part of a cloud computing system 108. In one example, the surgical system 102 can communicate with the remote server 109 via a networking connection, such as an Internet connection (e.g., a commercial Internet service, T3, cable / FIOS networking node, and the like). The surgical system 102 and / or components therein can communicate with the remote server 109 via a cellular transmission / reception point (TRP) or base station using one or more of the following cellular protocols: GSM / GPRS / EDGE (2G), UMTS / HSPA (3G), Long Term Evolution (LTE) or 4G, LTE-Advanced (LTE-A), New Radio (NR), or 5G.

[0066] In one example, the surgical hub 106 can facilitate the display of images from a surgical imaging device (e.g., a laparoscope). The surgical hub 106 interacts collaboratively with other local systems to facilitate the display of information related to those local systems. The surgical hub 106 can interact with one or more sensing systems 111, 115, one or more smart devices 114, and / or multiple displays. For example, the surgical hub 106 can be configured to collect measurement data from one or more sensing systems 111, 115 and send notification or control messages to the one or more sensing systems 111, 115. The surgical hub 106 can send information including notification information to the human-machine interface system 112 and / or receive information including notification information from the human-machine interface system. The human-machine interface system 112 may include one or more human-machine interface devices (HIDs). The surgical hub 106 can send and / or receive notification information or control information to convert into audio, display, and / or control information to various devices communicating with the surgical hub.

[0067] For example, the sensing systems 111, 115 may include a wearable sensing system 111 (the wearable sensing system may include one or more HCP sensing systems and one or more patient sensing systems) and an environmental sensing system 115. The one or more sensing systems 111, 115 may measure data related to various biomarkers. The one or more sensing systems 111, 115 may use one or more sensors, such as optical sensors (e.g., photodiodes, photoresistors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, pyroelectric sensors, infrared sensors, etc. to measure biomarkers. The one or more sensors may use one or more of the following sensing technologies to measure biomarkers as described herein: photoplethysmography, electrocardiography, electroencephalography, colorimetry, impedance spectroscopy, potentiometry, amperometry, etc.

[0068] Biomarkers measured by the one or more sensing systems 111, 115 may include, but are not limited to, sleep, core body temperature, VO2 max, physical activity, alcohol consumption, respiratory rate, oxygen saturation, blood pressure, blood glucose, heart rate variability, blood pH, hydration status, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, coughing and sneezing, gastrointestinal motility, gastrointestinal imaging, respiratory bacteria, edema, psychiatric factors, sweat, circulating tumor cells, autonomic nervous system tone, circadian rhythm, and / or menstrual cycle.

[0069] Biomarkers may be related to physiological systems, which may include, but are not limited to, behavior and psychology, cardiovascular system, renal system, skin system, nervous system, gastrointestinal system, respiratory system, endocrine system, immune system, tumor, musculoskeletal system and / or reproductive system. For example, information from biomarkers can be determined and / or used by a computer-implemented patient and surgical system 100. For example, the information from biomarkers can be determined and / or used by a computer-implemented patient and surgical system 100 to improve the system and / or improve patient outcomes. One or more sensing systems 111, 115, biomarkers and physiological systems are described in more detail in U.S. application No. 17 / 156,287 (Agent Docket No. END9290USNP1), filed January 22, 2021, entitled “METHODOF ADJUSTING A SURGICAL PARAMETER BASED ON BIOMARKER MEASUREMENTS,” the disclosure of which is incorporated herein by reference in its entirety.

[0070] Figure 2 An example of a surgical system 202 in a surgical operating room is shown. Figure 2 As illustrated, a patient undergoes surgery by one or more healthcare professionals (HCPs). The HCP is monitored by one or more HCP sensing systems 220 worn by the HCP. The HCP and the environment surrounding the HCP may also be monitored by one or more environmental sensing systems, including, for example, a set of cameras 221, a set of microphones 222, and other sensors that may be deployed in an operating room. The HCP sensing system 220 and the environmental sensing system may communicate with a surgical hub 206, which in turn may communicate with one or more cloud servers 209 of a cloud computing system 208, such as Figure 1 The environmental sensing system can be used to measure one or more environmental attributes, such as the position of the HCP in the surgical room, HCP movement, ambient noise in the surgical room, temperature / humidity in the surgical room, etc.

[0071] like Figure 2As illustrated, a primary display 223 and one or more audio output devices (e.g., speakers 219) are positioned within the sterile field so as to be visible to an operator at an operating table 224. Additionally, a visualization / notification tower 226 is positioned outside the sterile field. The visualization / notification tower 226 may include a first non-sterile human-machine interface device (HID) 227 and a second non-sterile HID 229 facing away from each other. The HIDs may be displays or displays with touchscreens that allow a person to interface directly with the HIDs. The human-machine interface system, guided by the surgical hub 206, may be configured to utilize the HIDs 227, 229, and 223 to coordinate the flow of information to operators inside and outside the sterile field. In one example, the surgical hub 206 may cause an HID (e.g., primary HID 223) to display notifications and / or information about a patient and / or surgical procedure. In one example, the surgical hub 206 may prompt and / or receive input from personnel in the sterile or non-sterile field. In one example, the surgical hub 206 can cause the HID to display a snapshot of the surgical site recorded by the imaging device 230 on the non-sterile HID 227 or 229 while maintaining real-time feedback of the surgical site on the primary HID 223. For example, the snapshot on the non-sterile display 227 or 229 can allow a non-sterile operator to perform diagnostic steps related to the surgical procedure.

[0072] In one aspect, the surgical hub 206 can be configured to route diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 to the primary display 223 within the sterile field, where the sterile operator at the operating table can view the diagnostic input or feedback. In one example, the input can be a modified form of a snapshot displayed on the non-sterile display 227 or 229, which can be routed to the primary display 223 via the surgical hub 206.

[0073] refer to Figure 2, the surgical instrument 231 is used in a surgical procedure as part of the surgical system 202. The hub 206 can be configured to coordinate the flow of information to the display of the surgical instrument 231. For example, described in U.S. Patent Application Publication No. US2019-0200844 A1 (U.S. Patent Application No. 16 / 209,385), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. Diagnostic input or diagnostic feedback entered by a non-sterile operator at the visualization tower 226 can be routed by the hub 206 to the surgical instrument display within the sterile field, where the operator of the surgical instrument 231 can view the diagnostic input or diagnostic feedback. For example, example surgical instruments suitable for use with the surgical system 202 are described under the heading “Surgical Instrument Hardware” in U.S. patent application publication No. US2019-0200844A1, entitled “METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY,” filed on December 4, 2018 (U.S. patent application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety.

[0074] Figure 2 An example of a surgical system 202 is illustrated for performing a surgical procedure on a patient lying flat on an operating table 224 in a surgical operating room 235. A robotic system 234 can be used as part of the surgical system 202 during the surgical procedure. The robotic system 234 can include a surgeon's console 236, a patient-side cart 232 (surgical robot), and a surgical robotic hub 233. While the surgeon views the surgical site through the surgeon's console 236, the patient-side cart 232 can manipulate at least one removably coupled surgical tool 237 through a minimally invasive incision in the patient's body. Images of the surgical site can be obtained through a medical imaging device 230, which can be manipulated by the patient-side cart 232 to orient the imaging device 230. The robotic hub 233 can be used to process the images of the surgical site for subsequent display to the surgeon through the surgeon's console 236.

[0075] Other types of robotic systems may be readily adapted for use with the surgical system 202. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. patent application Ser. No. US2019-0201137 A1, filed Dec. 4, 2018, entitled “METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL” (U.S. Patent Application Ser. No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety.

[0076] Various examples of cloud-based analytics performed by the cloud computing system 208 and suitable for use with the present disclosure are described in U.S. patent application publication No. US2019-0206569 A1, entitled “METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB,” filed on December 4, 2018 (U.S. patent application Ser. No. 16 / 209,403), the disclosure of which is incorporated herein by reference in its entirety.

[0077] In various aspects, imaging device 230 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, charge coupled device (CCD) sensors and complementary metal oxide semiconductor (CMOS) sensors.

[0078] The optical components of imaging device 230 may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate multiple portions of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0079] One or more illumination sources can be configured to radiate electromagnetic energy in the visible spectrum as well as in the invisible spectrum. The visible spectrum (sometimes referred to as the optical spectrum or the luminous spectrum) is that portion of the electromagnetic spectrum that is visible to the human eye (i.e., detectable by the human eye) and can be referred to as visible light or simply light. The typical human eye responds to wavelengths in the range of about 380 nm to about 750 nm in air.

[0080] The invisible spectrum (e.g., the non-luminescent spectrum) is the portion of the electromagnetic spectrum below and above the visible spectrum (i.e., wavelengths below about 380 nm and above about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, and they become invisible infrared (IR), microwaves, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, and they become invisible ultraviolet, x-ray, and gamma-ray electromagnetic radiation.

[0081] In various aspects, the imaging device 230 is configured for use in minimally invasive surgery. Examples of imaging devices suitable for use in the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, choledochoscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophagoduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngeal-renal endoscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0082] Imaging devices can use multispectral monitoring to distinguish topography and underlying structures. Multispectral images are images that capture image data within a specific wavelength range across the electromagnetic spectrum. The wavelengths can be separated by filters or by using instruments sensitive to specific wavelengths, including light from frequencies outside the visible light range, such as IR and ultraviolet. Spectral imaging can allow the extraction of additional information that the human eye fails to capture with its red, green, and blue receptors. The use of multispectral imaging is described in more detail under the title "Advanced Imaging Acquisition Module" in U.S. Patent Application Publication No. US2019-0200844 A1 (U.S. Patent Application No. 16 / 209,385), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed on December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety. After completing the surgical task to perform one or more of the previously described tests on the treated tissue, multispectral monitoring can be a useful tool for repositioning the surgical field. It is self-evident that during any surgical procedure, the operating room and surgical equipment need to be strictly sterilized. The strict hygienic and sterile conditions required in a "surgical room" (i.e., an operating room or treatment room) require the highest possible sterility of all medical devices and equipment. Part of this sterilization process is the need to sterilize any material that contacts the patient or penetrates the sterile field, including the imaging device 230 and its attachments and components. It should be understood that the sterile field can be considered to be a designated area that is considered to be free of microorganisms, such as in a tray or in a sterile towel, or the sterile field can be considered to be the area around the patient that has been prepared for surgery. The sterile field can include properly dressed scrubbed team members, as well as all equipment and fixtures in the area.

[0083] Figure 1 The illustrated wearable sensing system 211 may include one or more sensing systems, such as Figure 2HCP sensing system 220 is shown. HCP sensing system 220 may include a sensing system for monitoring and detecting a set of physical and / or physiological states of a healthcare professional (HCP). The HCP may typically be a surgeon or one or more healthcare professionals or other healthcare providers assisting the surgeon. In one example, sensing system 220 may measure a set of biomarkers to monitor the HCP's heart rate. In one example, sensing system 220 worn on the surgeon's wrist (e.g., a watch or wristband) may use an accelerometer to detect hand movement and / or tremors and determine the amplitude and frequency of the tremors. Sensing system 220 may send measurement data associated with the biomarker set and data associated with the surgeon's physical state to surgical hub 206 for further processing. One or more environmental sensing devices may send environmental information to surgical hub 206. For example, the environmental sensing device may include a camera 221 for detecting the hand / body position of the HCP. The environmental sensing device may include a microphone 222 for measuring ambient noise in the surgical room. Other environmental sensing devices may include, for example, a thermometer for measuring temperature and a hygrometer for measuring humidity in the surgical room. The surgical hub 206, either alone or in communication with a cloud computing system, can use surgeon biomarker measurement data and / or environmental sensing information to modify the control algorithms of the handheld instruments or the average latency of the robotic interface, for example, to minimize tremors. In one example, the HCP sensing system 220 can measure one or more surgeon biomarkers associated with the HCP and send measurement data associated with the surgeon biomarkers to the surgical hub 206. The HCP sensing system 220 can communicate with the surgical hub 20006 using one or more of the following RF protocols: Bluetooth, Bluetooth Low Energy (BLE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low Power Wireless Personal Area Network (6LoWPAN), Wi-Fi. The surgeon biomarkers may include one or more of the following: stress, heart rate, etc. Environmental measurements from the surgical room may include ambient noise levels associated with the surgeon or patient, surgeon and / or personnel movement, surgeon and / or personnel attention levels, etc.

[0084] The surgical hub 206 can use the surgeon's biomarker measurement data associated with the HCP to adaptively control one or more surgical instruments 231. For example, the surgical hub 206 can send a control program to the surgical instrument 231 to control its actuator to limit or compensate for fatigue and the use of fine motor skills. The surgical hub 206 can send the control program based on situational awareness and / or context regarding the importance or criticality of the task. When control is needed, the control program can instruct the instrument to change its operation to provide more control.

[0085] Figure 3An example surgical system 302 is shown having a surgical hub 306. The surgical hub 306 can be paired with a wearable sensing system 311, an environmental sensing system 315, a human-machine interface system 312, a robotic system 313, and smart instruments 314 via modular controls. The hub 306 includes a display 348, an imaging module 349, a generator module 350, a communication module 356, a processor module 357, a storage array 358, and an operating room mapping module 359. In certain aspects, as Figure 3 As shown, hub 306 also includes a smoke evacuation module 354 and / or a suction / irrigation module 355. Various modules and systems can be connected to the modular control directly via a router or via a communication module 356. Operating room devices can be coupled to cloud computing resources and data storage devices via the modular control. Human-machine interface system 312 can include a display subsystem and a notification subsystem.

[0086] The modular control can be coupled to a non-contact sensor module. The non-contact sensor module can use ultrasonic, laser-based, and / or similar non-contact measurement devices to measure the size of the operating room and generate a map of the surgical room. Other distance sensors can be used to determine the boundaries of the operating room. The ultrasound-based non-contact sensor module can scan the operating room by transmitting a burst of ultrasound and receiving the echo when it bounces off the walls of the operating room, as described in U.S. Provisional Patent Application Serial No. 62 / 611,341, entitled “INTERACTIVE SURGICAL PLATFORM,” filed on December 28, 2017, under the title “Surgical Hub Spatial Awareness Within an Operating Room,” which is incorporated herein by reference in its entirety. The sensor module can be configured to determine the size of the operating room and adjust the Bluetooth pairing distance limit. The laser-based non-contact sensor module can scan the operating room by transmitting laser pulses, receiving laser pulses that bounce off the walls of the operating room, and comparing the phase of the transmitted pulses with the received pulses to determine the size of the operating room and adjust the Bluetooth pairing distance limit.

[0087] During surgical procedures, the application of energy to tissue for sealing and / or cutting is often associated with smoke evacuation, suctioning of excess fluid, and / or flushing of tissue. Fluid lines, power lines, and / or data lines from different sources often become tangled during surgical procedures. Resolving this problem during surgical procedures wastes valuable time. Disconnecting a line may require disconnecting the line from its corresponding module, which may require resetting the module. The hub modular housing 360 provides a unified environment for managing power lines, data lines, and fluid lines, which reduces the frequency of tangles between such lines. Aspects of the present disclosure provide a surgical hub 306 for use in surgical procedures that involve applying energy to tissue at a surgical site.

[0088] The surgical hub 306 includes a hub housing 360 and a combination generator module slidably housed in a docking station of the hub housing 360. The docking station includes data contacts and power contacts. The combination generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component housed in a single unit. In one aspect, the combination generator module further includes a smoke evacuation component for connecting the combination generator module to at least one energy delivery cable of a surgical instrument, at least one smoke evacuation component configured to evacuate smoke, fluid, and / or particles generated by applying therapeutic energy to tissue, and a fluid line extending from a remote surgical site to the smoke evacuation component. In one aspect, the fluid line can be a first fluid line, and a second fluid line can extend from the remote surgical site to a suction and irrigation module 355 slidably housed in the hub housing 360. In one aspect, the hub housing 360 can include a fluid interface.

[0089] Some surgical procedures may require the application of more than one energy type to tissue. One energy type may be more advantageous for cutting tissue, while a different energy type may be more advantageous for sealing tissue. For example, a bipolar generator may be used to seal tissue, while an ultrasonic generator may be used to cut sealed tissue. Aspects of the present disclosure present a solution in which a hub modular housing 360 is configured to accommodate different generators and facilitate interactive communication between them. The hub modular housing 360 can enable rapid removal and / or replacement of various modules. Aspects of the present disclosure provide a modular surgical housing for use in surgical procedures involving the application of energy to tissue. The modular surgical housing includes a first energy generator module configured to generate a first energy for application to tissue, and a first docking station including a first docking port, the first docking port including first data and power contacts, wherein the first energy generator module is slidably movable into electrical engagement with the power and data contacts, and wherein the first energy generator module is slidably movable out of electrical engagement with the first power and data contacts. Further describing the above, the modular surgical housing also includes a second energy generator module, the second energy generator module being configured to generate a second energy different from the first energy for application to tissue, and a second docking base, the second docking base including a second docking port, the second docking port including second data and power contacts, wherein the second energy generator module is slidably movable into electrical engagement with the power and data contacts, and wherein the second energy generator is slidably movable into no longer electrical engagement with the second power and data contacts. In addition, the modular surgical housing also includes a communication bus between the first docking port and the second docking port, which is configured to facilitate communication between the first energy generator module and the second energy generator module. Reference Figure 3 Aspects of the present disclosure are presented as a hub modular housing 360 that allows for modular integration of a generator module 350, a smoke evacuation module 354, and a suction / irrigation module 355. The hub modular housing 360 also facilitates interactive communication between the modules 359, 354, and 355. The generator module 350 may have integrated monopolar, bipolar, and ultrasonic components supported in a single housing unit that is slidably inserted into the hub modular housing 360. The generator module 350 may be configured to be connectable to a monopolar device 351, a bipolar device 352, and an ultrasonic device 353. Alternatively, the generator module 350 may include a series of monopolar, bipolar, and / or ultrasonic generator modules that interact through the hub modular housing 360. The hub modular housing 360 may be configured to facilitate the insertion of multiple generators and interactive communication between the generators docked into the hub modular housing 360, so that the generators will function as a single generator.

[0090] Figure 4 A surgical data network having a communication hub set according to at least one aspect of the present disclosure is illustrated, the communication hub set being configured to connect a set of sensing systems, environmental sensing systems, and other modular devices located in one or more surgical rooms of a healthcare facility, a patient recovery room, or a room in a healthcare facility specifically equipped for surgical procedures to a cloud.

[0091] like Figure 4 As illustrated, the surgical hub system 460 can include a modular communication hub 465 configured to connect modular devices located in a healthcare facility to a cloud-based system (e.g., a cloud computing system 464, which may include a remote server 467 coupled to a remote storage device 468). The modular communication hub 465 and the devices can be connected in a room in the healthcare facility specifically equipped for surgical procedures. In one aspect, the modular communication hub 465 can include a network hub 461 and / or a network switch 462 in communication with a network router 466. The modular communication hub 465 can be coupled to a local computer system 463 to provide local computer processing and data manipulation.

[0092] The computer system 463 may include a processor and a network interface. The processor may be coupled to a communication module, storage device, memory, non-volatile memory, and input / output (I / O) interfaces via a system bus. The system bus may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures, including but not limited to a 9-bit bus, Industry Standard Architecture (ISA), Micro Charmel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), USB, Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), Small Computer System Interface (SCSI), or any other peripheral bus.

[0093] The controller can be any single-core or multi-core processor, such as those offered by Texas Instruments under the trade name ARM Cortex. In one aspect, the processor can be, for example, the LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments, which includes 256KB of single-cycle flash or other non-volatile memory (up to 40MHz) on-chip memory, a prefetch buffer for improved execution above 40MHz, 32KB of single-cycle sequential random access memory (SRAM), a Internal read-only memory (ROM) for software, 2KB electrically erasable programmable read-only memory (EEPROM), and / or one or more pulse-width modulation (PWM) modules, one or more quadrature encoder input (QEI) simulations, one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, details of which can be found in the product datasheet.

[0094] In one example, the processor may include a safety controller that includes two controller-based families (such as TMS570 and RM4x), also known as Hercules ARMCortex R4, produced by Texas Instruments. The safety controller can be configured to be specifically designed for IEC 61508 and ISO 26262 safety-critical applications, etc., to provide advanced integrated safety features while delivering scalable execution, connectivity, and memory options.

[0095] It should be understood that the computer system 463 may include software that acts as an intermediary between a user and the basic computer resources described in a suitable operating environment. Such software may include an operating system. The operating system, which may be stored on a disk storage device, may be used to control and allocate the resources of the computer system. System applications may utilize the operating system to manage resources through program modules and program data stored in system memory or disk storage devices. It should be understood that the various components described herein may be implemented using various operating systems or combinations of operating systems.

[0096] A user can enter commands or information into the computer system 463 via input devices coupled to the I / O interface. Input devices may include, but are not limited to, pointing devices such as a mouse, touchball, stylus, touchpad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, webcam, and the like. These and other input devices are connected to the processor via the system bus via one or more interface ports. Interface ports include, for example, serial ports, parallel ports, game ports, and USB. Output devices use the same type of port as input devices. Thus, for example, a USB port can be used to provide input to the computer system 463 and output information from the computer system 463 to an output device. Output adapters are provided to illustrate that among other output devices, some may require special adapters, such as monitors, displays, speakers, and printers. Output adapters may include, by way of example, but are not limited to, video and sound cards that provide a connection between an output device and the system bus. It should be noted that other devices and / or systems of devices, such as a system of remote computers, may provide both input and output capabilities.

[0097] The computer system 463 can operate in a networked environment using logical connections to one or more remote computers (such as cloud computers) or local computers. The remote cloud computer can be a personal computer, server, router, network PC, workstation, microprocessor-based appliance, peer device or other public network node and the like, and typically includes many or all of the elements described relative to the computer system. For simplicity, only a memory storage device with a remote computer is illustrated. The remote computer can be logically connected to the computer system via a network interface and then physically connected via a communication connection. The network interface can cover communication networks such as local area networks (LANs) and wide area networks (WANs). LAN technologies may include fiber distributed data interface (FDDI), copper distributed data interface (CDDI), Ethernet / IEEE 802.3, token ring / IEEE 802.5 and the like. WAN technologies may include, but are not limited to, point-to-point links, circuit switching networks such as integrated services digital networks (ISDN) and their variants, packet switching networks and digital subscriber lines (DSL).

[0098] In various examples, computer system 463 may include an image processor, an image processing engine, a media processor, or any dedicated digital signal processor (DSP) for processing digital images. The image processor may employ parallel computing with single instruction, multiple data (SIMD) or multiple instruction, multiple data (MIMD) techniques to increase speed and efficiency. The digital image processing engine may perform a range of tasks. The image processor may be a system on a chip with a multi-core processor architecture.

[0099] The communication connection may refer to the hardware / software used to connect the network interface to the bus. Although the communication connection is shown for illustrative purposes within the computer system 463, it may also be located outside the computer system 463. The hardware / software necessary to connect to the network interface may include internal and external technologies, such as modems, including conventional telephone-grade modems, cable modems, fiber optic modems, and DSL modems, ISDN adapters, and Ethernet cards, for illustrative purposes only. In some examples, an RF interface may also be used to provide the network interface.

[0100] The surgical data network associated with the surgical hub system 460 can be configured to be passive, intelligent, or switched. A passive surgical data network acts as a conduit for data, enabling it to be transferred from one device (or segment) to another device (or segment) and to cloud computing resources. An intelligent surgical data network includes additional features to enable monitoring of traffic passing through the surgical data network and configuration of each port in the network hub 461 or network switch 462. An intelligent surgical data network can be referred to as a managed hub or switch. The switching hub reads the destination address of each packet and then forwards the packet to the correct port.

[0101] Modular devices 1a-1n located in an operating room can be coupled to a modular communication hub 465. A network hub 461 and / or a network switch 462 can be coupled to a network router 466 to connect devices 1a-1n to a cloud computing system 464 or a local computer system 463. Data associated with devices 1a-1n can be transmitted via the router to a cloud-based computer for remote data processing and manipulation. Data associated with devices 1a-1n can also be transmitted to a local computer system 463 for local data processing and manipulation. Modular devices 2a-2m located in the same operating room can also be coupled to a network switch 462. Network switch 462 can be coupled to a network hub 461 and / or a network router 466 to connect devices 2a-2m to the cloud 464. Data associated with devices 2a-2m can be transmitted via the network router 466 to a cloud computing system 464 for data processing and manipulation. Data associated with devices 2a-2m can also be transmitted to a local computer system 463 for local data processing and manipulation.

[0102] like Figure 4 As illustrated, a computing system (such as a surgical hub system 460) can include a modular communication hub 465 configured to connect modular devices (e.g., surgical devices) located in a healthcare facility to a cloud-based system (e.g., a cloud computing system 464, which can include a remote server 467 coupled to a remote storage device 468). The modular communication hub 465 and the devices can be connected in a room in the healthcare facility that is specifically equipped for surgical procedures. In one aspect, the modular communication hub 465 can include a network hub 461 and / or a network switch 462 that communicates with a network router 466. The modular communication hub 465 can be coupled to a local computer system (e.g., a computing device) to provide local computer processing and data manipulation.

[0103] Figure 5A logic diagram of a control system 520 for a surgical instrument or surgical tool according to one or more aspects of the present disclosure is illustrated. The surgical instrument or surgical tool may be configurable. The surgical instrument may include a surgical fixation device specific to the procedure being performed, such as an imaging device, a surgical stapler, an energy device, an endoscopic cutting and sealing device, or the like. For example, the surgical instrument may include any of a powered stapler, a powered stapler generator, an energy device, a pre-energy device, a pre-energy jaw device, an endoscopic cutting and sealing clamp, an energy device generator, an operating room imaging system, a smoke evacuator, a suction-irrigation device, an insufflation system, and the like. System 520 may include control circuitry. The control circuitry may include a microcontroller 521 including a processor 522 and a memory 523. For example, one or more of sensors 525, 526, and 527 provide real-time feedback to processor 522. A motor 530 driven by a motor driver 529 is operably coupled to a longitudinally movable displacement member to drive the I-beam blade element. Tracking system 528 can be configured to determine the position of the longitudinally movable displacement member. Position information can be provided to processor 522, which can be programmed or configured to determine the position of the longitudinally movable drive member and the position of the firing member, firing rod and I-beam blade element. Additional motors can be provided at the tool driver interface to control I-beam firing, closure tube advancement, shaft rotation and joint motion. Display 524 can display a variety of operating conditions of the instrument and can include touch screen functionality for data input. The information displayed on display 524 can be superimposed with images obtained via an endoscopic imaging module.

[0104] The microcontroller 521 can be any single-core or multi-core processor, such as those offered by Texas Instruments under the trade name ARM Cortex. In one aspect, the main microcontroller 521 can be, for example, the LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments, which includes on-chip memory of 256KB single-cycle flash memory or other non-volatile memory (up to 40MHz), a prefetch buffer for improving performance above 40MHz, 32KB single-cycle SRAM, a CPU loaded with Internal ROM for software, 2KB EEPROM, one or more PWM modules, one or more QEI analogs and / or one or more 12-bit ADCs with 12 analog input channels, details of which can be found in the product datasheet.

[0105] The microcontroller 521 may include a safety controller including two controller-based families (such as TMS570 and RM4x), also known from Texas Instruments under the trade name Hercules ARM Cortex R4. The safety controller can be configured to be specifically designed for IEC 61508 and ISO 26262 safety-critical applications, among others, to provide advanced integrated safety features while delivering scalable execution, connectivity, and memory options.

[0106] The microcontroller 521 can be programmed to perform various functions, such as precise control of the speed and position of the knife and articulation system. In one aspect, the microcontroller 521 can include a processor 522 and a memory 523. The electric motor 530 can be a brushed direct current (DC) motor having a gearbox and a mechanical link to the articulation or knife system. In one aspect, the motor driver 529 can be an A3941 available from Allegro Microsystems, Inc. Other motor drivers can be easily substituted for use in the tracking system 528 including the absolute positioning system. A detailed description of the absolute positioning system is described in U.S. Patent Application Publication No. 2017 / 0296213, entitled “SYSTEMS AND METHODS FOR CONTROLLING ASURGICAL STAPLING AND CUTTING INSTRUMENT,” published on October 19, 2017, which is incorporated herein by reference in its entirety.

[0107] The microcontroller 521 can be programmed to provide precise control of the velocity and position of the displacement member and articulation system. The microcontroller 521 can be configured to calculate the response in the microcontroller's 521 software. The calculated response can be compared with the measured response of the actual system to obtain an "observed" response, which is used in actual feedback decisions. The observed response can be a favorable tuning value that balances the smooth, continuous nature of the simulated response with the measured response, which can detect external influences on the system.

[0108] The motor 530 can be controlled by a motor driver 529 and can be used by the firing system of the surgical instrument or tool. In various forms, the motor 530 can be a brushed DC drive motor with a maximum rotational speed of approximately 25,000 RPM. In some examples, the motor 530 may include a brushless motor, a cordless motor, a synchronous motor, a stepper motor, or any other suitable electric motor. The motor driver 529 may include, for example, an H-bridge driver including a field effect transistor (FET). The motor 530 can be powered by a power assembly that can be releasably mounted to the handle assembly or tool housing for supplying control power to the surgical instrument or tool. The power assembly may include a battery that may include a plurality of battery cells connected in series that can be used as a power source to provide power to the surgical instrument or tool. In some cases, the battery cells of the power assembly may be replaceable and / or rechargeable. In at least one example, the battery cells may be lithium-ion batteries that can be coupled to and separated from the power assembly.

[0109] The motor driver 529 may be the A3941, available from Allegro Microsystems, Inc. The A3941 is a full-bridge controller designed for use with external N-channel power metal-oxide semiconductor field-effect transistors (MOSFETs) specifically designed for inductive loads, such as brushed DC motors. The driver 529 may include a unique charge pump regulator that provides full (>10V) gate drive for battery voltages as low as 7V and allows the A3941 to operate with reduced gate drive down to 5.5V. A bootstrap capacitor may be used to provide the battery supply voltage required by the N-channel MOSFETs. The internal charge pump of the high-side driver allows for DC (100% duty cycle) operation. The full bridge may be driven in either fast or slow decay mode using diodes or synchronous rectification. In slow decay mode, current recirculation may be through either the high-side FET or the low-side FET. A resistor-adjustable dead time may be used to protect the power FETs from breakdown. Comprehensive diagnostics provide indications of undervoltage, overtemperature, and power bridge faults, and can be configured to protect the power MOSFETs under most short circuit conditions.Other motor drivers can be easily substituted for use in tracking system 528 including an absolute positioning system.

[0110] Tracking system 528 may include a controlled motor drive circuit arrangement including a position sensor 525 according to one aspect of the present disclosure. Position sensor 525 for an absolute positioning system can provide a unique position signal corresponding to the position of a displacement member. In some examples, the displacement member may represent a longitudinally movable drive member including a rack of drive teeth for meshing engagement with a corresponding drive gear of a gear reducer assembly. In some examples, the displacement member may represent a firing member, which may be adapted and configured to include a rack of drive teeth. In some examples, the displacement member may represent a firing rod or an I-beam, each of which may be adapted and configured to include a rack of drive teeth. Therefore, as used herein, the term displacement member may be generally used to refer to any movable component of a surgical instrument or tool, such as a drive member, firing member, firing rod, I-beam, or any element capable of displacement. In one aspect, the longitudinally movable drive member may be coupled to the firing member, firing rod, and I-beam. Thus, the absolute positioning system can, in effect, track the linear displacement of the I-beam by tracking the linear displacement of the longitudinally movable drive member. In various aspects, the displacement member can be coupled to any position sensor 525 suitable for measuring linear displacement. Thus, a longitudinally movable drive member, firing member, firing rod, or I-beam, or a combination thereof, can be coupled to any suitable linear displacement sensor. The linear displacement sensor can include a contact displacement sensor or a non-contact displacement sensor. The linear displacement sensor can include a linear variable differential transformer (LVDT), a differential variable reluctance transducer (DVRT), a slide potentiometer, a magnetic sensing system including a movable magnet and a series of linearly arranged Hall effect sensors, a magnetic sensing system including a fixed magnet and a series of movable linearly arranged Hall effect sensors, an optical sensing system including a movable light source and a series of linearly arranged photodiodes or photodetectors, an optical sensing system including a fixed light source and a series of movable linearly arranged photodiodes or photodetectors, or any combination thereof.

[0111] The electric motor 530 may include a rotatable shaft operably interfaced with a gear assembly mounted on the displacement member in meshing engagement with a set of drive teeth or a rack of drive teeth. The sensor element may be operably coupled to the gear assembly such that a single rotation of the position sensor 525 element corresponds to a certain linear longitudinal translation of the displacement member. The transmission and sensor arrangement may be connected to a linear actuator via a rack and pinion arrangement, or to a rotary actuator via spur gears or other connections. A power source may supply power to the absolute positioning system, and an output indicator may display the output of the absolute positioning system. The displacement member may represent a longitudinally movable drive member including a rack of drive teeth formed thereon for meshing engagement with a corresponding drive gear of the gear reducer assembly. The displacement member may represent a longitudinally movable firing member, a firing rod, an I-beam, or a combination thereof.

[0112] A single rotation of the sensor element associated with position sensor 525 can be equivalent to a longitudinal linear displacement d1 of the displacement member, where d1 is the longitudinal linear distance the displacement member moves from point "a" to point "b" following a single rotation of the sensor element coupled to the displacement member. The sensor arrangement can be connected via a gear reduction that enables position sensor 525 to complete one or more rotations for a full stroke of the displacement member. Position sensor 525 can complete multiple rotations for a full stroke of the displacement member.

[0113] A series of switches (where n is an integer greater than one) can be employed, alone or in conjunction with gear reduction, to provide unique position signals for more than one rotation of the position sensor 525. The state of the switches can be fed back to the microcontroller 521, which applies logic to determine a unique position signal corresponding to the longitudinal linear displacement d1+d2+...dn of the displacement member. The output of the position sensor 525 is provided to the microcontroller 521. The position sensor 525 of this sensor arrangement can include a magnetic sensor, an analog rotation sensor (such as a potentiometer), or an array of analog Hall effect elements, which output a unique combination of position signals or values.

[0114] The position sensor 525 may include any number of magnetic sensing elements, such as, for example, magnetic sensors classified according to whether they measure the total magnetic field or the vector components of the magnetic field. The technologies used to produce the above two types of magnetic sensors can cover many aspects of physics and electronics. Technologies for magnetic field sensing may include probe coils, fluxgates, optical pumps, nuclear spins, superconducting quantum interference devices (SQUIDs), Hall effects, anisotropic magnetoresistors, giant magnetoresistors, magnetic tunnel junctions, giant magnetoimpedance, magnetostrictive / piezoelectric composites, magnetodiodes, magnetotransistors, optical fibers, magneto-optics, and micro-electromechanical system-based magnetic sensors, among others.

[0115] The position sensor 525 for the tracking system 528, which includes an absolute positioning system, may include a magnetic rotary absolute positioning system. The position sensor 525 may be implemented as the AS5055EQFT monolithic magnetic rotary position sensor available from Austria Microsystems, AG. The position sensor 525 interfaces with the microcontroller 521 to provide the absolute positioning system. The position sensor 525 may be a low-voltage and low-power component and may include four Hall-effect elements in an area of the position sensor 525 that may be located above the magnet. A high-resolution ADC and an intelligent power management controller may also be provided on-chip. A coordinate rotation digital computer (CORDIC) processor (also known as the bitwise method and Volder algorithm) may be provided to execute a simple, efficient algorithm for calculating hyperbolic and trigonometric functions, requiring only addition, subtraction, bit shifts, and table lookup operations. Angular position, alarm bits, and magnetic field information may be transmitted to the microcontroller 521 via a standard serial communication interface, such as a serial peripheral interface (SPI) interface. The position sensor 525 may provide 12 or 14-bit resolution. The position sensor 525 may be an AS5055 chip provided in a small QFN 16-pin 4x4x0.85mm package.

[0116] Tracking system 528, including an absolute positioning system, may include and / or be programmed to implement feedback controllers such as PID, state feedback, and adaptive controllers. A power source converts the signal from the feedback controller into a physical input to the system: in this case, voltage. Other examples include PWM of voltage, current, and force. In addition to the position measured by position sensor 525, other sensors may be provided to measure physical parameters of the physical system. In some aspects, one or more other sensors may include sensor arrangements such as those described in U.S. Patent No. 9,345,481, entitled “STAPLE CARTRIDGETISSUE THICKNESS SENSOR SYSTEM,” issued on May 24, 2016, which is incorporated herein by reference in its entirety; U.S. Patent Application Publication No. 2014 / 0263552, entitled “STAPLE CARTRIDGE TISSUETHICKNESS SENSOR SYSTEM,” published on September 18, 2014, which is incorporated herein by reference in its entirety; and U.S. Patent Application Serial No. 15 / 628,175, entitled “TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT,” filed on June 20, 2017, which is incorporated herein by reference in its entirety. In a digital signal processing system, the absolute positioning system is coupled to a digital data acquisition system, where the output of the absolute positioning system will have a finite resolution and sampling frequency. The absolute positioning system may include comparison and combination circuits to combine the calculated response with the measured response using algorithms (such as weighted averages and theoretical control loops) that drive the calculated response toward the measured response. The calculated response of the physical system may take into account characteristics such as mass, inertia, viscous friction, inductance, and resistance to predict the state and output of the physical system from known inputs.

[0117] Thus, the absolute positioning system can provide the absolute position of the displacement member when the instrument is powered on, and does not retract or advance the displacement member to a reset (zero or home) position as may be required with conventional rotary encoders, which simply count the number of forward or backward steps taken by the motor 530 to infer the position of the device actuator, drive rod, knife, and the like.

[0118] The sensor 526 (such as, for example, a strain gauge or a micro strain gauge) can be configured to measure one or more parameters of the end effector, such as, for example, the amplitude of the strain applied to the anvil during the clamping operation, which can indicate the closing force applied to the anvil. The measured strain can be converted into a digital signal and provided to the processor 522. Alternatively or in addition to the sensor 526, a sensor 527 (such as, for example, a load cell) can measure the closing force applied to the anvil by the closed drive system. The sensor 527 (such as, for example, a load cell) can measure the firing force applied to the I-beam during the firing stroke of the surgical instrument or tool. The I-beam is configured to engage a wedge-shaped sliding member that is configured to enable the nail driver to cam upward to push the nail out to deform and contact the anvil. The I-beam can also include a sharp cutting edge that can be used to cut tissue when the I-beam is pushed distally by the firing rod. Alternatively, a current sensor 531 can be used to measure the current drawn by the motor 530. For example, the force required to advance the firing member may correspond to the current drawn by the motor 530. The measured force may be converted into a digital signal and provided to the processor 522.

[0119] For example, a strain gauge sensor 526 can be used to measure the force applied to tissue by the end effector. A strain gauge can be coupled to the end effector to measure the force on the tissue being processed by the end effector. A system for measuring the force applied to tissue grasped by the end effector can include a strain gauge sensor 526, such as, for example, a micro strain gauge, which can be configured to measure, for example, one or more parameters of the end effector. In one aspect, the strain gauge sensor 526 can measure the amplitude or magnitude of the strain applied to the jaw member of the end effector during the clamping operation, which can indicate tissue compression. The measured strain can be converted into a digital signal and provided to the processor 522 of the microcontroller 521. The load sensor 527 can measure the force used to operate the knife element, for example, to cut tissue captured between the anvil and the staple cartridge. A magnetic field sensor can be used to measure the thickness of the captured tissue. The measurements of the magnetic field sensor can also be converted into a digital signal and provided to the processor 522.

[0120] The microcontroller 521 can characterize the corresponding values of the selected position of the firing member and / or the velocity of the firing member using the measurements of tissue compression, tissue thickness, and / or the force required to close the end effector on the tissue, respectively, as measured by the sensors 526, 527. In one embodiment, the memory 523 can store techniques, equations, and / or lookup tables that can be used by the microcontroller 521 in making the evaluation.

[0121] The control system 520 of the surgical instrument or tool may also include wired or wireless communication circuitry to communicate with a surgical hub, such as, for example, the surgical hub 460, as shown. Figure 4 shown.

[0122] Figure 6 An example surgical system 680 according to the present disclosure is illustrated and may include a surgical instrument 682 that can communicate with a console 694 or a portable device 696 via a wired and / or wireless connection through a local area network 692 and / or a cloud network 693. The console 694 and the portable device 696 may be any suitable computing device. The surgical instrument 682 may include a handle 697, an adapter 685, and a loading unit 687. The adapter 685 is releasably coupled to the handle 697, and the loading unit 687 is releasably coupled to the adapter 685 such that the adapter 685 transmits force from a drive shaft to the loading unit 687. The adapter 685 or the loading unit 687 may include a force gauge (not explicitly shown) disposed therein to measure the force applied to the loading unit 687. The loading unit 687 may include an end effector 689 having a first jaw 691 and a second jaw 690. The loading unit 687 may be an in-situ loading or multi-fire loading unit (MFLU) that allows the clinician to fire multiple fasteners multiple times without having to remove the loading unit 687 from the surgical site to reload the loading unit 687 .

[0123] The first jaw 691 and the second jaw 690 can be configured to clamp tissue therebetween, fire a fastener through the clamped tissue, and sever the clamped tissue. The first jaw 691 can be configured to fire at least one fastener multiple times, or can be configured to include a replaceable multiple-fire fastener cartridge that includes multiple fasteners (e.g., nails, clamps, etc.) that can be fired more than once before being replaced. The second jaw 690 can include an anvil that deforms or otherwise secures the fastener as it is ejected from the multiple-fire fastener cartridge.

[0124] The handle 697 may include a motor coupled to the drive shaft to affect the rotation of the drive shaft. The handle 697 may include a control interface for selectively activating the motor. The control interface may include a button, a switch, a lever, a slider, a touch screen, and any other suitable input mechanism or user interface that can be engaged by a clinician to activate the motor.

[0125] The control interface of the handle 697 can communicate with the controller 698 of the handle 697 to selectively activate the motor to affect the rotation of the drive shaft. The controller 698 can be disposed within the handle 697 and configured to receive input from the control interface and adapter data from the adapter 685 or loading unit data from the loading unit 687. The controller 698 can analyze the input from the control interface and the data received from the adapter 685 and / or the loading unit 687 to selectively activate the motor. The handle 697 can also include a display that the clinician can view during use of the handle 697. The display can be configured to display portions of the adapter or loading unit data before, during, or after firing the instrument 682.

[0126] The adapter 685 may include an adapter identification device 684 disposed therein, and the loading unit 687 may include a loading unit identification device 688 disposed therein. The adapter identification device 684 may communicate with the controller 698, and the loading unit identification device 688 may communicate with the controller 698. It should be understood that the loading unit identification device 688 may communicate with the adapter identification device 684, which relays or transmits communications from the loading unit identification device 688 to the controller 698.

[0127] The adapter 685 may also include a plurality of sensors 686 (one shown) disposed about the adapter 685 to detect various conditions of the adapter 685 or the environment (e.g., whether the adapter 685 is connected to the loading unit, whether the adapter 685 is connected to the handle, whether the drive shaft is rotating, the torque of the drive shaft, the strain of the drive shaft, the temperature within the adapter 685, the number of adapter 685 firings, the peak force of the adapter 685 during firings, the total amount of force applied to the adapter 685, the peak retraction force of the adapter 685, the number of pauses of the adapter 685 during firings, etc.). The plurality of sensors 686 may provide input to the adapter identification device 684 in the form of data signals. The data signals of the plurality of sensors 686 may be stored in the adapter identification device 684 or may be used to update adapter data stored in the adapter identification device. The data signals of the plurality of sensors 686 may be analog or digital. The plurality of sensors 686 may include a force gauge to measure the force applied to the loading unit 687 during firing.

[0128] The handle 697 and the adapter 685 can be configured to interconnect the adapter identification device 684 and the loading unit identification device 688 with the controller 698 via an electrical interface. The electrical interface can be a direct electrical interface (i.e., comprising electrical contacts that engage with each other to transmit energy and signals therebetween). Additionally or alternatively, the electrical interface can be a contactless electrical interface to wirelessly transmit energy and signals therebetween (e.g., inductive transmission). It is also contemplated that the adapter identification device 684 and the controller 698 can communicate with each other wirelessly via a wireless connection separate from the electrical interface.

[0129] The handle 697 may include a transceiver 683 configured to transmit instrument data from the controller 698 to other components of the system 680 (e.g., the LAN 20292, the cloud 693, the console 694, or the portable device 696). The controller 698 may also transmit instrument data and / or measurement data associated with one or more sensors 686 to the surgical hub. The transceiver 683 may receive data (e.g., cartridge data, loading unit data, adapter data, or other notifications) from the surgical hub 670. The transceiver 683 may also receive data (e.g., cartridge data, loading unit data, or adapter data) from other components of the system 680. For example, the controller 698 may transmit instrument data to the console 694, including the serial number of the attachment adapter (e.g., adapter 685) attached to the handle 697, the serial number of the loading unit (e.g., loading unit 687) attached to the adapter 685, and the serial number of the multi-shot fastener cartridge loaded into the loading unit. Thereafter, the console 694 may transmit data associated with the attached cartridge, loading unit, and adapter, respectively (e.g., cartridge data, loading unit data, or adapter data) back to the controller 698. The controller 698 may display the message on a local instrument display or transmit the message to the console 694 or portable device 696 via the transceiver 683 for display on the display 695 or portable device screen, respectively.

[0130] Figure 7A A surgical system 700 is illustrated that may include a surgical information matrix. This surgical information may include any discrete information atoms related to a surgical procedure. Generally described, such surgical information may include information related to the context and scope of the surgical procedure itself (e.g., healthcare information 728). For example, such information may include data such as surgical data and patient record data. The surgical data and / or patient record data may be associated with an associated healthcare data system 716 in communication with the surgical computing device 704.

[0131] Surgical data may include information related to instruments and / or replaceable instrument parts to be used in a given surgery, such as, for example, a master list. The surgical computing device 704 may record (e.g., capture barcode scans) the instruments and / or replaceable instrument parts used in the surgery. Such surgical information can be used to algorithmically confirm that the appropriate configuration of surgical instruments and / or replaceable parts are being used. See U.S. Patent Application Publication No. US2020-0405296A1 (U.S. Patent Application No. 16 / 458,103), entitled "PACKAGING FOR AREPLACEABLE COMPONENT OF ASURGICAL STAPLING SYSTEM," filed on June 30, 2019, the contents of which are hereby incorporated by reference herein in their entirety.

[0132] For example, patient record data may be applicable to changing the configuration of certain surgical devices. For example, patient data may be used to understand and improve surgical device algorithm behavior. In one example, a surgical stapler may adjust operating parameters related to compression, operating rate, usage position, and feedback based on information in a patient record (e.g., information indicating the tissue and / or tissue characteristics of a particular patient). See U.S. Patent Application Publication No. US 2019-0200981A1 (U.S. Patent Application No. 16 / 209,423), filed on December 4, 2018, entitled “METHOD OF COMPRESSING TISSUE WITHIN STAPLING DEVICE AND SIMULTANEOUSLY DISPLAYING THE LOCATION OF THE TISSUEWITHIN THE JAWS,” the contents of which are hereby incorporated by reference in their entirety.

[0133] The surgical information may include information related to the configuration and / or control of a device used in a surgical procedure (e.g., device operation information 729). Such device operation information 729 may include information regarding the initial settings of the surgical device. The device operation information 729 may include information regarding changes to the settings of the surgical device. The device operation information 729 may include information regarding controls sent from the surgical computing device 704 to the device and information flows related to such controls.

[0134] Surgical information may include information generated during the surgical procedure itself (e.g., surgical information 727). Such surgical information 727 may include any information generated by a surgical data source 726. Data source 726 may include any device in a surgical setting that can generate useful surgical information 727. This surgical information 727 may manifest itself as observable qualities of the data source 726. Observable qualities may include static qualities, such as the model number, serial number, and the like of the device. Observable qualities may include dynamic qualities, such as the status of a configurable setting of the device. For example, surgical information 727 may manifest itself as the result of sensor observations. Sensor observations may include those from specific sensors within the surgical room, sensors used to monitor conditions (such as the patient's condition), sensors embedded in surgical devices, and the like. Sensor observations may include information used during the surgical procedure, such as video, audio, and the like. Surgical information 727 may manifest itself as device event data. Surgical devices may generate notifications and / or may record events, and such events may be included in the surgical information 727 for transmission to the surgical computing device 704. For example, surgical information 727 may be presented as a result of manual recording. A healthcare professional may take notes during a surgical procedure, such as by taking notes, capturing still images from a display, and the like.

[0135] The surgical data sources 726 may include modular devices (e.g., modular devices may include sensors configured to detect parameters associated with the patient, HCP, and environment and / or the modular devices themselves), local databases (e.g., a local EMR database containing patient records), patient monitoring devices (e.g., blood pressure (BP) monitors and electrocardiogram (EKG) monitors), HCP monitoring devices, environmental sensing devices, surgical instruments, surgical support equipment, and the like.

[0136] Smart surgical instruments can sense and measure certain operating parameters during their operation. For example, smart surgical instruments (such as surgical robots, digital laparoscopic devices, and the like) can use such measurements to improve operation, such as limiting excessive compression, reducing collateral damage, minimizing tissue tension, optimizing the position of use, and the like. See U.S. Patent Application Publication No. US2018-0049822 A1, filed on August 16, 2016, entitled “CONTROL OF ADVANCEMENT RATE AND APPLICATION FORCE BASED ON MEASURED FORCES” (U.S. Patent Application No. 15 / 237,753), the contents of which are hereby incorporated by reference in their entirety. Such surgical information can be transmitted to the surgical computing device 704.

[0137] The surgical computing device 704 may be configured to be capable of deriving contextual information related to the surgical procedure from the data, for example, based on a particular combination of the received data or a particular order in which the data is received from the data source 726. The contextual information inferred from the received data may include, for example, the type of surgical procedure being performed, the particular step of the surgical procedure being performed by the surgeon, the type of tissue being operated on, or the body cavity being operated on. Some aspects of the surgical computing device 704's ability to derive or infer information related to the surgical procedure from the received data may be referred to as "situational awareness." For example, the surgical computing device 704 may incorporate a situational awareness system, which is hardware and / or programming associated with the surgical computing device 704 that derives contextual information related to the surgical procedure from the received data and / or surgical planning information received from the edge computing system 714 or healthcare data system 716 (e.g., an enterprise cloud server). Such situational awareness capabilities may be used to generate surgical information (such as control and / or configuration information) based on sensed situation and / or usage. See U.S. patent application publication No. US2019-0104919 A1 (U.S. patent application Ser. No. 16 / 209,478), filed on December 4, 2018, entitled “METHOD FOR SITUATIONAL AWARENESS FOR SURGICAL NETWORK OR SURGICAL NETWORK CONNECTED DEVICE CAPABLE OF ADJUSTING FUNCTION BASED ON A SENSED SITUATION ORUSAGE,” the contents of which are hereby incorporated by reference in their entirety.

[0138] In operation, the surgical information matrix may exist as one or more information flows. For example, surgical information may flow from a surgical data source 726 to a surgical computing device 704. Surgical information may flow from the surgical computing device 704 to the surgical data source 726 (e.g., a surgical device). Surgical information may flow between the surgical computing device 704 and one or more healthcare data systems 716. Surgical information may flow between the surgical computing device 704 and one or more edge computing devices 714. Various aspects of the information flow may be managed relative to the surgical system 700 (e.g., relative to a healthcare facility), including, for example, information flow endpoints, information storage, data interpretation, and the like. See U.S. Patent Application Publication No. US2019-0206564 A1 (U.S. Patent Application No. 16 / 209,490), filed on December 4, 2018, entitled “METHOD FOR FACILITYDATA COLLECTION AND INTERPRETATION,” the contents of which are hereby incorporated by reference in their entirety.

[0139] The surgical information, as presented in one or more information streams thereof, may be used in conjunction with one or more artificial intelligence (AI) systems to further enhance the operation of the surgical system 700. For example, a machine learning system, such as the machine learning systems described herein, may operate on one or more information streams to further enhance the operation of the surgical system 700.

[0140] Figure 7B An example computer-implemented surgical system 730 is shown having multiple information streams 732. The surgical computing device 704 can communicate with and / or integrate with one or more surgical data sources. For example, the imaging module 733 (and the endoscope) can exchange surgical information with the surgical computing device 704. Such information can include information from the imaging module 733 (and the endoscope), such as video information, current settings, system status information, and the like. The imaging module 733 can receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software / firmware), and the like.

[0141] For example, the generator module 734 (and corresponding energy device) can exchange surgical information with the surgical computing device 704. Such information can include information from the generator module 734 (and corresponding energy device), such as electrical information (e.g., current, voltage, impedance, frequency, wattage), activity state information, sensor information (such as temperature, current setting, system events, activity duration, and activation timestamp), and the like. The generator module 734 can receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to healthcare professionals (e.g., changing the pitch, duration, and melody of an audible tone), electrical application profiles and / or application logic that can instruct the generator module to provide energy with a defined characteristic curve within an application time, operational updates (such as software / firmware), and the like.

[0142] For example, the smoke evacuator 735 can exchange surgical information with the surgical computing device 704. Such information can include information from the smoke evacuator 735, such as operational information (e.g., revolutions per minute), activity status information, sensor information (such as air temperature), current settings, system events, activity duration, activation timestamps, and the like. The smoke evacuator 735 can receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software / firmware), and the like.

[0143] For example, the aspiration / irrigation module 736 can exchange surgical information with the surgical computing device 704. Such information can include information from the aspiration / irrigation module 736, such as operational information (e.g., liters / minute), activity status information, internal sensor information, current settings, system events, activity duration and activation timestamps, and the like. The aspiration / irrigation module 736 can receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software / firmware), and the like.

[0144] For example, the communication module 739, the processor module 737, and / or the storage array 738 can exchange surgical information with the surgical computing device 704. In one example, the communication module 739, the processor module 737, and / or the storage array 738 can constitute all or part of a computing platform on which the surgical computing device 704 operates. In one example, the communication module 739, the processor module 737, and / or the storage array 738 can provide local computing resources to other devices in the surgical system 730. Information from the communication module 739, the processor module 737, and / or the storage array 738 to the surgical computing device 704 can include logical computing-related reports such as processing load, processing power, process identification, CPU%, CPU time, threads, GPU%, GPU time, memory utilization, memory threads, memory ports, energy usage, bandwidth-related information, packets input, packets output, data rate, channel utilization, buffer status, packet loss information, system events, other status information, and the like. The communication module 739, the processor module 737, and / or the storage array 738 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software / firmware), and the like. The communication module 739, the processor module 737, and / or the storage array 738 may also receive information generated by another element or device of the surgical system 730 from the surgical computing device 704. For example, data source information may be sent to and stored in the storage array. For example, the data source information may be processed by the processor module 737.

[0145] For example, the smart instrument 740 (with or without a corresponding display) can exchange surgical information with the surgical computing device 704. Such information can include information from the smart instrument 740 related to the operation of the instrument, such as device electrical and / or mechanical information (e.g., current, voltage, impedance, frequency, wattage, torque, force, pressure, etc.), load state information (e.g., information about the identity, type, and / or state of reusable parts such as staple cartridges), internal sensor information (such as clamping force, tissue compression pressure and / or time), system events, activity duration and activation timestamps, and the like. The smart instrument 740 can receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to healthcare professionals (e.g., changes in the pitch, duration, and melody of an audible tone), mechanical application profiles and / or application logic that can instruct the mechanical components of the instrument to operate with defined characteristics (e.g., blade / anvil advancement speed, mechanical advantage, firing time, etc.), operational updates (such as software / firmware), and the like.

[0146] For example, in surgical stapling and cutting instruments, control and configuration information can be used to modify operating parameters, such as motor speed. The data set of surgical information can be used to define the power, force and / or other functional operations and / or behaviors of intelligent surgical stapling and cutting instruments. See U.S. Patent No. 10,881,399B2 (U.S. Patent Application No. 15 / 628,175), entitled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," filed on June 20, 2017, the contents of which are hereby incorporated by reference in their entirety.

[0147] For example, in an energy device, control and configuration information (e.g., control and configuration information based on the situational awareness of the surgical computing device 704) can be used to adapt functionality and / or behavior to obtain improved results. See U.S. Patent Application Publication No. US2019-0201047 A1, entitled "METHOD FOR SMART ENERGY DEVICE INFRASTRUCTURE," filed on December 4, 2018 (U.S. Patent Application No. 16 / 209,458), the contents of which are hereby incorporated by reference in their entirety. Similarly, in a combination energy device (e.g., a device that can use more than one energy modality), such control and / or configuration information can be used to select an appropriate operating mode. For example, the surgical computing device 704 can use surgical information including information received from patient monitoring to send control and / or configuration information to the combination energy device. See U.S. Patent Application Publication No. US2017-0202605 A1, filed on December 16, 2016, entitled “MODULAR BATTERY POWERED HANDHELD SURGICAL INSTRUMENT AND METHODS THEREFOR” (U.S. Patent Application No. 15 / 382,515), the contents of which are hereby incorporated by reference in their entirety.

[0148] For example, the sensor module 741 can exchange surgical information with the surgical computing device 704. Such information can include information from the sensor module 741 related to its sensor functionality, such as sensor results themselves, observation frequency and / or resolution, observation type, device alarms (such as alarms for sensor failure), observations exceeding a defined range, observations exceeding an observable range, and the like. The sensor module 741 can receive information from the surgical computing device 704, such as control information, configuration information, changes in observation properties (e.g., frequency, resolution, observation type, etc.), triggers defining specific events for observation, on-control, off-control, data buffering, data pre-processing algorithms, operational updates (such as software / firmware), and the like.

[0149] For example, the visualization system 742 can exchange surgical information with the surgical computing device 704. Such information can include information from the visualization system 742, such as the visualization data itself (e.g., still images, video, advanced spectral visualization, etc.), visualization metadata (e.g., visualization type, resolution, frame rate, encoding, bandwidth, etc.). The visualization system 742 can receive information from the surgical computing device 704, such as control information, configuration information, changes to video settings (e.g., visualization type, resolution, frame rate, encoding, etc.), visual display overlay data, data buffer size, data pre-processing algorithms, operational updates (such as software / firmware), and the like.

[0150] Surgical information can be exchanged and / or used with advanced imaging systems. For example, surgical information can be exchanged and / or used to provide context for imaging data streams. For example, surgical information can be exchanged and / or used to expand the conditional understanding of such imaging data streams. See U.S. patent application No. 17 / 493,904, entitled “SURGICAL METHODS USING MULTI-SOURCE IMAGING,” filed on October 5, 2021, the contents of which are hereby incorporated by reference herein in their entirety. See U.S. patent application No. 17 / 493,913, entitled “SURGICAL METHODS USING FIDUCIAL IDENTIFICATION AND TRACKING,” filed on October 5, 2021, the contents of which are hereby incorporated by reference herein in their entirety.

[0151] For example, the surgical robot 743 may exchange surgical information with the surgical computing device 704. In one example, the surgical information may include information related to the collaborative registration and interaction of the surgical robotic system. See U.S. patent application No. 17 / 449,765, entitled "COOPERATIVE ACCESS HYBRID PROCEDURES," filed on October 1, 2021, the contents of which are hereby incorporated by reference in their entirety. Information from the surgical robot 743 may include any of the aforementioned information applied to robotic instruments, sensors, and devices. Information from the surgical robot 743 may also include information related to the robotic operation or control of such instruments, such as electrical / mechanical feedback of the robotic articulator, system events, system settings, mechanical resolution, control operation logs, articulator path information, and the like. The surgical robot 743 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software / firmware), and the like.

[0152] Surgical devices communicating with the surgical computing device 704 can exchange surgical information to facilitate collaborative operation between the devices. For example, the surgical robot 743 and the energy generator 734 can exchange surgical information with each other and / or with the surgical computing device 704 for collaborative operation. The collaborative operation between the collaborative surgical robot 743 and the energy generator 734 can be used to minimize unwanted side effects, such as tissue adhesions. The collaborative operation between the collaborative surgical robot 743 and the energy generator 734 can be used to improve tissue welding. See U.S. Patent Application Publication No. US2019-0059929 A1, entitled “METHODS, SYSTEMS, AND DEVICES FOR CONTROLLING ELECTROSURGICAL TOOLS,” filed on August 29, 2017 (U.S. Patent Application No. 15 / 689,072), the contents of which are hereby incorporated by reference herein in their entirety. Surgical information can be generated by the collaborative devices and / or the surgical computing device 704 in conjunction with their collaborative operation.

[0153] The surgical computing system 704 can record, analyze, and / or act on surgical information streams, such as those disclosed above. The surgical computing system 704 can aggregate such data for analysis. For example, the surgical computing system 704 can perform operations such as defining device relationships, establishing device collaborative behavior, monitoring and / or storing surgical details, and the like. Surgical information related to such operations can be further analyzed to refine algorithms, identify trends, and / or adapt surgical procedures. For example, surgical information can be further analyzed based on such operations compared to patient outcomes. See U.S. Patent Application Publication No. US2019-0206562 A1, entitled “METHOD OF HUB COMMUNICATION, PROCESSING, DISPLAY, AND CLOUD ANALYTICS,” filed on December 4, 2018 (U.S. Patent Application No. 16 / 209,416), the contents of which are hereby incorporated by reference in their entirety.

[0154] Figure 7CAn example information flow associated with multiple surgical computing systems 704a, 704b in a common environment is illustrated. When the overall configuration of a computer-implemented surgical system (e.g., computer-implemented surgical system 750) changes (e.g., when data sources are added and / or removed from the surgical computing system), additional surgical information can be generated to reflect these changes. In this example, a second surgical computing system 704b (e.g., a surgical hub) can be added (along with a corresponding surgical robot) to the surgical system 750 with the existing surgical computing system 704a. The message passing flow described herein represents an additional surgical information flow 755 to be employed as disclosed herein (e.g., further merged, analyzed, and / or processed according to an algorithm (such as a machine learning algorithm)).

[0155] Here, two surgical computing systems 704a, 704b request permission from the surgical operator for the second surgical computing system 704b (with the corresponding surgical robot 756) to take control of the operating room from the existing surgical computing system 704a. The second surgical computing system 704b is present in the operating room under the control of the corresponding surgical robot 756, the robotic visualization tower 758, the single-cap tool 759, and the robotic stapler 749. Permission may be requested through the surgeon interface or console 751. Once permission is granted, the second surgical computing system 704b sends a message to the existing surgical computing system 704a requesting a transfer of control of the operating room.

[0156] In one example, surgical computing systems 704a and 704b can negotiate the nature of their interaction based on previously acquired data without external input. For example, surgical computing systems 704a and 704b can jointly determine that the next surgical task requires the use of a robotic system. Such a determination can cause the existing surgical computing system 704a to autonomously hand over control of the operating room to the second surgical computing system 704b. Upon completion of the surgical task, the second surgical computing system 704b can then autonomously return control of the operating room to the existing surgical computing system 704a.

[0157] like Figure 7C As illustrated, the existing surgical computing system 704a has transferred control to the second surgical computing system 704b, which also controls the surgeon interface 751 and the auxiliary display 752. The second surgical computing system 704b assigns a new identification number to the newly transferred devices. The existing surgical computing system 704a maintains control of the handheld stapler 753, the handheld electric dissector 754, and the visualization tower 757. In addition, the existing surgical computing system 704a can perform a support role, where the processing and storage capabilities of the existing surgical computing system 704a are now available to the second surgical computing system 704b.

[0158] Figure 7DExample surgical information flows in the context of surgical procedures and corresponding example uses of surgical information for predictive modeling are illustrated. The surgical information disclosed herein may provide data regarding one or more surgical procedures, including surgical tasks, instruments, instrument settings, procedure information, surgical variations, and corresponding desired metrics, such as improved patient outcomes, lower costs (e.g., utilizing fewer resources, less surgical time, etc.). In the context of one or more surgical systems and devices disclosed herein, the surgical information disclosed herein (e.g., regarding 7A to 7C The disclosed surgical information) provides a platform on which the specific machine learning algorithms and techniques disclosed herein can be used.

[0159] Surgical information 762 (eg, a subset of surgical information from each procedure) may be collected from multiple surgical procedures 764. For example, surgical information 762 may be collected from multiple surgical procedures 764 by collecting data represented by one or more information flows disclosed herein.

[0160] For illustration, an example instance of surgical information 766 may be generated from an example surgery 768 (e.g., a segmentectomy as shown on timeline 769). Surgical information 766 may be generated during preoperative planning and may include patient record information. Surgical information 766 may be generated from data sources (e.g., data source 726) during the course of the surgical procedure, including data generated each time medical personnel utilize a modular device paired with a surgical computing system (e.g., surgical computing system 704). The surgical computing system may receive this data from the paired modular devices and other data sources. The surgical computing system itself may generate surgical information as part of its operations during the surgery. For example, the surgical computing system may record information related to configuration and control operations. The surgical computing system may record information related to situational awareness activities. For example, the surgical computing system may record recommendations, prompts, and / or other information provided to the healthcare team (e.g., via a display screen) that may be relevant to the next surgical step. For example, the surgical computing system may record configuration and control changes (e.g., adjusting modular devices based on context). Such configuration and control changes may include activating a monitor, adjusting the field of view (FOV) of a medical imaging device, changing the energy level of an ultrasonic surgical instrument or an RF electrosurgical instrument, or the like.

[0161] At 770, hospital staff retrieves the patient's EMR from the hospital's EMR database. Based on the patient data selected in the EMR, the surgical computing system determines that the procedure to be performed is thoracic surgery.

[0162] At 771, the staff scans the incoming medical supplies for the procedure. The surgical computing system may cross-reference the scanned supplies with a list of supplies utilized in various types of surgery. The surgical computing system may confirm that the mix of supplies corresponds to a thoracic procedure. Furthermore, the surgical computing system may determine that the procedure is not a wedge procedure (because the incoming supplies lack certain supplies required for a thoracic wedge procedure or otherwise do not correspond to a thoracic wedge procedure). The medical staff may also scan the patient belt via a scanner communicatively connected to the surgical computing system. The surgical computing system may confirm the patient's identity based on the scanned data.

[0163] At 774, the medical staff activates the auxiliary devices. The auxiliary devices utilized may vary depending on the type of surgical procedure and the techniques the surgeon intends to use. In this example, the auxiliary devices may include a smoke evacuator, an insufflator, and a medical imaging device. When activated, the auxiliary devices may pair with the surgical computing system. The surgical computing system may derive contextual information about the surgical procedure based on the type of pairing. In this example, the surgical computing system determines that the surgical procedure is a VATS procedure based on this particular combination of paired devices. The contextual information about the surgical procedure may be confirmed by the surgical computing system via information from the patient's EMR.

[0164] The surgical computing system can retrieve the steps of the surgery to be performed. For example, the steps can be associated with a surgical plan (e.g., a surgical plan specific to the surgical procedure for the patient, a surgical plan associated with a specific surgeon, a surgical plan template commonly used for surgery, and the like).

[0165] At 775, staff attach EKG electrodes and other patient monitoring devices to the patient. The EKG electrodes and other patient monitoring devices are paired with the surgical computing system. The surgical computing system can receive data from the patient monitoring devices.

[0166] At 776, the medical personnel induce anesthesia in the patient. The surgical computing system may record information related to the surgical step, such as data from the modular device and / or patient monitoring device, including, for example, EKG data, blood pressure data, ventilator data, or a combination thereof.

[0167] At 777, the lung of the patient undergoing surgery is collapsed (ventilation may be switched to the contralateral lung). The surgical computing system may determine that the surgical step has begun and may accordingly collect surgical information including, for example, ventilator data, one or more time stamps, and the like.

[0168] At 778, a medical imaging device (e.g., an endoscope) is inserted and video from the medical imaging device is started. The surgical computing system may receive medical imaging device data (i.e., video or image data) through its connection to the medical imaging device. The data from the medical imaging device may include imaging data and / or imaging metadata, such as the angle at which the medical imaging device is oriented relative to visualization of the patient's anatomy, the number of medical imaging devices currently active, and the like. The surgical computing system may record positioning information for the medical imaging device. For example, one technique for performing a VATS lobectomy places a camera in the inferior anterior corner of the patient's thorax above the diaphragm. Another technique for performing a VATS segmentectomy places the camera in an anterior intercostal position relative to the segmental fissure.

[0169] For example, using pattern recognition or machine learning techniques, a surgical computing system can be trained to identify the positioning of a medical imaging device based on visualization of a patient's anatomy. For example, one technique for performing a VATS lobectomy utilizes a single medical imaging device. Another technique for performing a VATS segmentectomy may utilize multiple cameras. Yet another technique for performing a VATS segmentectomy utilizes an infrared light source (which may be communicatively coupled to the surgical computing system as part of the visualization system).

[0170] At 779, the surgical team begins the dissection step of the procedure. The surgical computing system may collect data from the RF or ultrasonic generator indicating that the energy instrument is being fired. The surgical computing system may cross-reference the received data with the retrieved steps of the surgical procedure to determine that the energy instrument fired at that point in the procedure (i.e., after the previously discussed steps of the procedure are completed) corresponds to the dissection step. In one example, the energy instrument may be an energy tool mounted to a robotic arm of a robotic surgical system.

[0171] At 780, the surgical team proceeds to the ligation step of the procedure. The surgical computing system may collect surgical information 766 about the surgeon's ligation of the artery and vein based on data received from the surgical stapling and cutting instruments indicating that such instruments are being fired. Next, the segmentectomy portion of the procedure is performed. The surgical computing system may collect information related to the surgeon's transecting of the parenchyma. For example, the surgical computing system may receive surgical information 766 from the surgical stapling and cutting instruments, including data about their cartridges, settings, firing details, and the like.

[0172] At 782, a node dissection step is performed. The surgical computing system may collect surgical information 766 from the surgical team regarding the dissection node and the leak test. For example, the surgical computing system may collect data received from the generator indicating that the RF or ultrasonic instrument is being fired and including electrical information and status information associated with the firing. The surgeon regularly switches back and forth between the surgical stapling / cutting instrument and the surgical energy (i.e., RF or ultrasonic) instrument according to specific steps in the operation. The surgical computing system may collect surgical information 766 based on a specific order of using the stapling / cutting instrument and the surgical energy instrument. In one example, a robotic tool may be used for one or more steps in the surgical procedure. The surgeon may, for example, alternate between the robotic tool and the handheld surgical instrument and / or may use the devices simultaneously.

[0173] The incision is then closed and the postoperative portion of the surgery begins. At 784, the patient's anesthesia is reversed. The surgical computing system can collect surgical information that the patient is waking up from anesthesia based on, for example, ventilator data (e.g., the patient's respiratory rate begins to increase).

[0174] At 785, the medical staff removes the various patient monitoring devices from the patient. The surgical computing system may collect information regarding the end of the surgery. For example, the surgical computing system may collect information related to the loss of EKG, BP, and other data from the patient monitoring devices.

[0175] The surgical information 762 (including the surgical information 766) can be structured and / or labeled. The surgical computing system can inherently provide such structure and / or labeling in the data collection. For example, the surgical information 762 can be labeled according to specific characteristics, desired results (e.g., efficiency, patient outcomes, cost, and / or combinations thereof or the like), certain surgical techniques, aspects of instrument use (e.g., selection, timing, and activation of surgical instruments, instrument settings, nature of instrument use, etc.), the identity of the healthcare professionals involved, specific patient characteristics, and the like, each of which can be present in the data set.

[0176] Surgical information (e.g., surgical information 762 collected across procedures 764) can be used in conjunction with one or more artificial intelligence (AI) systems. AI can be used to perform computer cognitive tasks. For example, AI can be used to perform complex tasks based on observations of data. AI can be used to enable computing systems to perform cognitive tasks and solve complex tasks. AI can include the use of machine learning and machine learning techniques. ML techniques can include, for example, performing complex tasks without being programmed (e.g., explicitly programmed). For example, ML techniques can improve over time based on completing tasks with different inputs. ML processes can, for example, use input data and / or learning data sets to train themselves.

[0177] Machine learning (ML) techniques can be employed, for example, in the medical field. For example, ML can be applied to a dataset (e.g., a surgical dataset) to produce an output (e.g., reduced surgical data, processed surgical data). In an example, the output of the ML process can include identified trends or relationships in the data input for processing. The output can include results and / or conclusions that validate the data associated with the input. In an example, the input to the ML process can include medical data, such as surgical images and patient scans. The ML process can output a determined medical condition based on the input surgical images and patient scans. The ML process can be used, for example, to diagnose a medical condition based on a surgical scan.

[0178] The ML process can improve itself, for example, using historical data and / or input data used to train the ML process. Thus, the ML process may continuously improve with additional input and processing. The ML process can be updated based on the input data. For example, over time, an ML process that generates medical conclusions based on medical data may improve and become more accurate and consistent in its medical diagnosis.

[0179] ML processes can be used to solve various complex tasks (e.g., medical tasks). For example, ML processes can be used for data reduction, data preparation, data processing, trend identification, conclusion determination, medical diagnosis, and / or the like. For example, an ML process can take surgical data as input and process the data for medical analysis. The processed data can be used to determine a medical diagnosis. Ultimately, the ML process can take raw surgical data and generate useful medical information associated with the raw surgical data (e.g., medical trends and / or diagnoses).

[0180] ML processes can be combined to perform different discrete tasks on an input data set. For example, an ML process can include testing different combinations of ML sub-processes that perform discrete tasks to determine which combination of ML sub-processes performs best (e.g., competitive use and training of different process / algorithm types to determine the best combination for a data set). For example, an ML process can include sub-process (e.g., algorithm) control and monitoring to improve and / or validate results and / or conclusions (e.g., error bounds).

[0181] An ML process may be initialized and / or configured to perform a task. For example, the ML process may be initialized based on initialization configuration information. The initialized ML process may be an untrained and / or basic ML process for performing a task. An untrained ML process may not accurately perform a specified task. As the ML process is trained, the task can be performed more accurately.

[0182] Initialization configuration information for an ML process may include initial settings and / or parameters. For example, the initial settings and / or parameters may include defined ranges for the ML process to employ. Ranges may include ranges for manually entered and / or received data. Ranges may include default ranges and / or randomized ranges for variables not received, which may be used, for example, to complete the dataset for processing. For example, if a dataset is missing a data range, a default data range may be used as an alternative for executing the ML process.

[0183] Initialization configuration information for an ML process may include data storage locations. For example, it may include a location or data storage device and / or database associated with a data interaction. The database associated with the data interaction may be used to identify trends in the data set. The database associated with the data interaction may include a mapping of data to medical conditions. For example, the database associated with the data interaction may include a mapping of heart rate data to medical conditions (such as, for example, arrhythmia and / or the like).

[0184] Initialization configuration information may include parameters associated with defining the system. Initialization configuration information may include instructions (e.g., methods) associated with displaying, confirming, and / or providing information to a user. For example, the initialization configuration may include instructions for an ML process to output data in a specific format for user visualization.

[0185] ML techniques can be used, for example, to perform data reduction. ML techniques for data reduction can include using a variety of different data reduction techniques. For example, ML techniques for data reduction can include using one or more of: CUR matrix decomposition; decision trees; expectation maximization (EM) procedures (e.g., algorithms); explicit semantic analysis (ESA); exponential smoothing forecasting; generalized linear models; k-means clustering (e.g., nearest neighbor); naive Bayes; neural network procedures; multivariate analysis; o-clustering; singular value decomposition; Q-learning; temporal difference (TD); deep adversarial networks; support vector machines (SVMs); linear regression; dimensionality reduction; linear discriminant analysis (LDA); adaptive boosting (e.g., AdaBoost); gradient descent (e.g., stochastic gradient descent (SGD)); outlier detection; and / or the like.

[0186] ML techniques can be used to perform data reduction, for example, using CUR matrix decomposition. CUR matrix decomposition may include using a matrix decomposition model (e.g., process, algorithm), such as a low-rank matrix decomposition model. For example, CUR matrix decomposition may include a low-rank matrix decomposition process that expresses (e.g., explicitly expresses) multiple (e.g., a small amount) columns and / or rows of a data matrix (e.g., CUR matrix decomposition may be interpretable). CUR matrix decomposition may include selecting columns and / or rows associated with statistical leverage and / or large influences in the data matrix. Using CUR matrix decomposition can achieve identification of attributes and / or rows in the data matrix. Simplification of larger data sets (e.g., using CUR matrix decomposition) can achieve user review and interaction (e.g., interaction with data). CUR matrix decomposition can promote regression, classification, clustering and / or the like.

[0187] ML techniques can be used, for example, to perform data reduction using decision trees (e.g., decision tree models). Decision trees can be used, for example, as a framework for quantifying the value of an outcome and / or the probability of an outcome occurring. Decision trees can be used, for example, to calculate the value of an uncertain outcome node (e.g., in a decision tree). Decision trees can be used, for example, to calculate the value of a decision node (e.g., in a decision tree). Decision trees can be models that implement classification and / or regression (e.g., can be adapted to classification and / or regression problems). Decision trees can be used to analyze numerical (e.g., continuous values) and / or categorical data. Decision trees may be more successful and / or more effective for large data sets (e.g., compared to other data reduction techniques).

[0188] Decision trees can be used in conjunction with other decision trees. For example, a random forest can refer to a collection of decision trees (e.g., an ensemble of decision trees). A random forest can include a collection of decision trees, the results of which can be aggregated into a result. A random forest can be a supervised learning algorithm. A random forest can be trained, for example, using a bagging training procedure.

[0189] Random decision forests (e.g., random forests) can add randomness (e.g., additive randomness) to a model, for example, when growing trees. For example, random forests can be used to search for the best feature among a random subset of features, rather than searching for the most important feature (e.g., when splitting a node). Searching for the best feature among a random subset of features can generate a wide variety of results, which can result in a better (e.g., more efficient and / or more accurate) model.

[0190] Random forests can include the use of parallel ensembles. Parallel ensembles can include, for example, fitting (e.g., several) decision tree classifiers in parallel on different subsamples of a dataset. Parallel ensembles can include using majority voting or averaging for the results or final results. Parallel ensembles can be used to minimize overfitting and / or improve prediction accuracy and control. A random forest with multiple decision trees can (e.g., typically) be more accurate than a single decision tree-based model. For example, by combining bootstrap aggregation (e.g., bagging) and random feature selection, a series of decision trees with controlled variation can be constructed.

[0191] ML techniques can be used, for example, to perform data reduction using an expectation-maximization (EM) model (e.g., a process, an algorithm). For example, an EM model can be used to find likelihood (e.g., local maximum likelihood) parameters of a statistical model. An EM model can be used in situations where a direct solution to an equation may not be possible. An EM model can account for latent variables and / or unknown parameters as well as known data observations. For example, an EM model can determine the presence of missing values in a dataset. The EM model receives configuration information indicating the presence of missing (e.g., unobserved) data points in a hypothetical dataset.

[0192] The EM model can use component clustering. For example, component clustering can enable EM components to be grouped into high-level clusters. For example, if component clustering is disabled (e.g., in the EM model), then the components can be considered clustered.

[0193] ML techniques can be used, for example, to perform data reduction using explicit semantic analysis (ESA). ESA can be used at the semantic (e.g., meaning) level rather than at the vocabulary level of words or documents (e.g., surface-form vocabulary). ESA can focus on the meaning of a collection of texts, for example, as a combination of concepts found in the texts. ESA can be used for document classification. ESA can be used to calculate semantic relevance (e.g., how similar words or text fragments are to each other in meaning). ESA can be used for information retrieval.

[0194] For example, ESA can be used for document classification. Document classification can include tagging documents for management and classification. Tagging documents (e.g., with keywords) can allow for easier searching. Keyword tagging (e.g., using only keyword tags) can limit the accuracy and / or efficiency of document classification. For example, using keyword tags can reveal (e.g., only reveal) documents with keywords, but not documents with words that have similar meanings to the keywords. Semantically classifying text (e.g., using ESA) can improve the model's understanding of the text. Semantically classifying text can include representing documents as concepts and reducing reliance on specific keywords.

[0195] ML techniques can be used to perform data reduction, for example, using exponential smoothing forecasting models. Exponential smoothing can be used, for example, to smooth time series data using an exponential window function. For example, in a moving average, past observations may be weighted equally, but an exponential function can be used to assign weights that decrease exponentially over time.

[0196] ML techniques can be used to perform data reduction, for example, using linear regression. Linear regression can be used to predict continuous outcomes. For example, linear regression can be used to predict the value of a variable (e.g., a dependent variable) based on the values of different variables (e.g., independent variables). Linear regression can apply a linear approach to model the relationship between a scalar response and one or more explanatory variables (e.g., dependent and / or independent variables). Simple linear regression can refer to a linear regression use case associated with one explanatory variable. Multiple linear regression can refer to a linear regression use case associated with more than one explanatory variable. Linear regression can model relationships, for example, using a linear predictor function. The linear predictor function can estimate unknown model parameters from a data set.

[0197] For example, linear regression can be used to identify patterns within a training dataset. The identified patterns can involve groupings of values and / or labels. The model can learn the relationship between (e.g., each) label and the expected outcome. After training, the model can be applied to raw data outside the training dataset (e.g., data without a mapping and / or known output). The trained model using linear regression can determine predictions for calculations associated with the raw data, such as identifying seasonal changes in sales data, for example.

[0198] ML techniques can be used to perform data reduction, for example, generalized linear models (GLMs). GLMs can be used as a flexible generalization of linear regression. GLMs can generalize linear regression, for example, by enabling the linear model to be related to the response variable.

[0199] ML techniques can be used to perform data reduction, for example, using k-means clustering (e.g., a nearest neighbor model). K-means clustering can be used for vector quantization. K-means clustering can be used for signal processing. K-means clustering can aim to partition n observations into k clusters, for example, where each observation is classified into the cluster with the closest mean.

[0200] K-means clustering may include K-nearest neighbor (KNN) learning. KNN may be instance-based learning (e.g., non-generalized learning, lazy learning). KNN may avoid building a general internal model. KNN may include storing instances corresponding to training data in an n-dimensional space. KNN may use the data and classify data points, for example, based on a similarity measure (e.g., a Euclidean distance function). For example, the classification may be calculated based on a majority vote of the k nearest neighbors of (e.g., each) point. KNN may be robust to noisy training data. Accuracy may depend on data quality (e.g., for KNN). KNN may include selecting the number of neighbors to consider (e.g., an optimal number of neighbors to consider). KNN may be used for classification and / or regression.

[0201] ML techniques can be used, for example, to perform data reduction using a Naive Bayesian model (e.g., a process). For example, a Naive Bayesian model can be used to build a classifier. The Naive Bayesian model can be used to assign a class label (e.g., a vector represented as a feature value) to a problem instance. The class label can be extracted from a set (e.g., a finite set). Different processes (e.g., algorithms) can be used to train the classifier. A series of processes (e.g., a series of algorithms) can be used. The series of processes can be based on the principle that a Naive Bayesian classifier (e.g., all Naive Bayesian classifiers) assumes that the value of a feature is independent of the value of a different feature (e.g., a given class variable).

[0202] ML techniques can be used, for example, to perform data reduction using neural networks. A neural network can learn (e.g., be trained) by processing examples, for example, to perform other tasks (e.g., similar tasks). A processed example can include an input and an outcome (e.g., an input mapped to an outcome). The neural network can learn by forming probabilistic weighted associations between the input and the outcome. The probabilistic weighted associations can be stored within a data structure of the neural network. Training of the neural network from a given example can be performed by determining the difference between the processed output (e.g., a prediction) of the network and the target output. The difference can be an error. The neural network can adjust the weighted associations (e.g., stored weighted associations), for example, based on a learning rule and an error value.

[0203] ML techniques can be used to perform data reduction, for example, using multivariate analysis. The multivariate analysis can include performing multivariate state estimation and / or non-negative matrix factorization.

[0204] ML techniques can be used, for example, to perform data reduction using support vector machines (SVMs). SVMs can be used in multidimensional spaces (e.g., high-dimensional spaces, infinite-dimensional spaces). SVCs can be used to construct hyperplanes (e.g., a set of hyperplanes). The hyperplane that has the largest distance (e.g., compared to other constructed hyperplanes) to the nearest training data point in a class (e.g., any class) can achieve strong separation (e.g., generally, the larger the margin, the lower the generalization error of the classifier). SVMs may be effective in high-dimensional spaces. SVMs may behave differently, for example, based on different mathematical functions (e.g., kernels, kernel functions). For example, kernel functions may include one or more of the following: linear, polynomial, radial basis function (RBF), sigmoid, etc. Kernel functions can be used as SVM classifiers. SVMs may be limited in use cases, for example, when the dataset contains a lot of noise (e.g., overlapping target classes).

[0205] ML techniques can be used to perform data reduction, such as dimensionality reduction. Reducing the dimensionality of data samples (e.g., unlabeled data) can help refine groups and / or clusters. Reducing the number of variables in a model can simplify data trends. Simplified data trends can enable more efficient processing. For example, dimensionality reduction can be used if many (e.g., too many) dimensions obscure (e.g., negatively impact) insights, trends, patterns, conclusions, and / or the like.

[0206] Dimensionality reduction can include using principal component analysis (PCA). PCA can be used to establish principal components that govern relationships between data points. PCA can focus on simplifying (e.g., simplifying only) the principal components. Dimensionality reduction (e.g., PCA) can be used to maintain the diversity of data groupings in a dataset, but reduce the number of independent groups.

[0207] ML techniques can be used to perform data reduction, for example, linear discriminant analysis (LDA). LDA can refer to a linear decision boundary classifier, for example, which can be created by fitting a class-conditional density to the data (e.g., and applying Bayes' rule). LDA can include a generalization of Fisher's linear discriminant (e.g., projecting a given data set into a lower dimensional space, for example, to reduce the dimensionality and minimize the complexity of the model and reduce computational cost). An LDA model (e.g., a standard LDA model) can be suitable for classes with Gaussian density. An LDA model can assume that classes (e.g., all classes) share a covariance matrix. LDA can be similar to an analysis of variance (ANOVA) process and / or regression analysis. For example, LDA can be used to express a dependent variable as a linear combination of other features and / or measurements.

[0208] ML techniques can be used to perform data reduction, for example, such as adaptive boosting (e.g., AdaBoost). Adaptive boosting can include creating a classifier (e.g., a powerful classifier). Adaptive boosting can include creating a classifier by combining multiple classifiers (e.g., poorly performing classifiers), for example, to obtain a resulting classifier with high accuracy. AdaBoost can be an adaptive classifier that improves the efficiency of the classifier. AdaBoost can trigger overfitting. AdaBoost can be used (e.g., optimally used) to improve the performance of decision trees, base estimators, binary classification problems, and / or the like. AdaBoost may be sensitive to noisy data and / or outliers.

[0209] ML techniques can be used to perform data reduction, such as stochastic gradient descent (SGD). SGD can include an iterative process for optimizing a function (e.g., an objective function). SGD can be used to optimize an objective function, for example, one with certain smoothness properties. Stochastic can refer to random probability. SGD can be used to reduce computational burden, for example, in high-dimensional optimization problems. SGD can be used to achieve faster iterations, for example, while trading off a lower convergence rate. Gradient can refer to the slope of a function, for example, a function that calculates the degree of change in a variable in response to a change in another variable. Gradient descent can refer to a convex function that outputs partial derivatives with respect to its set of input parameters. For example, α can be the learning rate, and Ji can be the cost of training examples at iteration i. This equation can represent the stochastic gradient descent weight update method for iteration j. In large-scale ML and sparse ML, SGD can be applied to problems in text classification and / or natural language processing (NLP). SGD can be sensitive to feature scaling (e.g., a range of hyperparameters, such as regularization parameters and number of iterations, may need to be used).

[0210] ML techniques can be used, for example, to perform data reduction, such as using outlier detection. An outlier can be a data point that contains information (e.g., useful information) about unusual behavior of the system described by the data. The outlier detection process can include univariate and multivariate processes.

[0211] For example, the ML process can be trained using one or more training methods. For example, the ML process can be trained using one or more of the following training techniques: supervised learning; unsupervised learning; semi-supervised learning; reinforcement learning; and / or the like.

[0212] Machine learning can be supervised (e.g., supervised learning). A supervised learning algorithm can create a mathematical model using a training data set (e.g., training data). Figure 8A An example supervised learning framework 800 is illustrated. Training data (e.g., training examples 802, e.g., Figure 8A) can be composed of a set of training examples (e.g., input data mapped to labeled outputs, such as Figure 8A ). The training examples 802 may include one or more inputs and one or more labeled outputs. The labeled outputs may be used as supervised feedback. In mathematical models, the training examples 802 may be represented by arrays or vectors (sometimes called eigenvectors). The training data may be represented by the rows of the eigenvectors that make up the matrix. Through iterative optimization of an objective function (e.g., a cost function), a supervised learning algorithm may learn a function (e.g., a prediction function) that can be used to predict outputs associated with one or more new inputs. A suitably trained prediction function (e.g., a trained ML model 808) may determine outputs 804 (e.g., labeled outputs) for one or more inputs 806 that may not be part of the training data (e.g., input data that does not have a mapped labeled output, such as Figure 8A ). Example algorithms may include linear regression, logistic regression, neural networks, nearest neighbor, naive Bayes, decision trees, SVMs, and / or the like. Example problems that may be solved by supervised learning algorithms may include classification, regression problems, and the like.

[0213] Machine learning can be unsupervised (e.g., unsupervised learning). Figure 8B An example unsupervised learning framework 810 is illustrated. An unsupervised learning algorithm 814 can be trained on a dataset that may include input 811 and can find structure 812 in the data (e.g., pattern detection and / or descriptive modeling). The structure 812 in the data may be similar to grouping or clustering of data points. Thus, the algorithm 814 can learn from training data that may not yet be labeled. Instead of responding to supervised feedback, the unsupervised learning algorithm can identify commonalities in the training data and can react based on the presence or absence of such commonalities in each training data. For example, training can include operating on the training input data to generate a model and / or output with a specific energy (e.g., such as a cost function), wherein such energy can be used to further refine the model (e.g., to define a model that minimizes the cost function given the training input data). Example algorithms may include a priori algorithms, K-means, K-nearest neighbors (KNN), K-median, and the like. Example problems that can be solved by unsupervised learning algorithms may include clustering problems, anomaly / outlier detection problems, and the like.

[0214] Machine learning can be semi-supervised (e.g., semi-supervised learning). Semi-supervised learning algorithms can be used in scenarios where labeling data is expensive (e.g., because it requires skilled experts to label the data) and labels for the data are limited. Semi-supervised learning models can exploit the idea that although group membership is unknown for unlabeled data, the data still carries important information about group parameters.

[0215] Machine learning may include reinforcement learning, which may be a field of machine learning that deals with the concept of how a software agent can take actions in an environment to maximize a cumulative reward. Reinforcement learning algorithms may not assume knowledge of an exact mathematical model of the environment (e.g., represented by a Markov decision process (MDP)) and may be used when an exact model is not feasible. Reinforcement learning algorithms may be used for self-driving vehicles or for learning to play games against human opponents. Example algorithms may include Q-learning, temporal difference (TD), deep adversarial networks, and / or the like.

[0216] Reinforcement learning can include an algorithm (e.g., an agent) that continuously learns from an environment in an iterative manner. During training, the agent can learn from the environment's experience until the agent explores the entire range of states (e.g., possible states). Reinforcement learning can be defined by the type of problem. Solutions to reinforcement learning can be categorized as reinforcement learning algorithms. In a problem, an agent can decide which action to choose (e.g., the best action) based on the agent's current state. If the steps are repeated, the problem can be referred to as an MDP.

[0217] For example, reinforcement learning may include an operation step. The operation step in reinforcement learning may include an agent observing an input state. The operation step in reinforcement learning may include using a decision function to cause the agent to perform an action. The operation step may include (e.g., after performing the action) the agent receiving a reward and / or reinforcement from the environment. The operation step in reinforcement learning may include storing state-action pair information about the reward.

[0218] Machine learning can be part of a technology platform known as cognitive computing (CC), which encompasses various disciplines such as computer science and cognitive science. CC systems can learn at scale, reason purposefully, and interact naturally with humans. Through self-learning algorithms that can use data mining, visual recognition, and / or natural language processing, CC systems can solve problems and optimize manual processes.

[0219] The output of the training process of machine learning can be a model for predicting the results of a new data set. For example, a linear regression learning algorithm can be a cost function that minimizes the prediction error of the linear prediction function during the training process by adjusting the coefficients and constants of the linear prediction function. When a minimum value can be reached, the linear prediction function with the adjusted coefficients can be considered to have been trained and constitutes a model for which a training process has been generated. For example, a neural network (NN) algorithm for classification (e.g., a multilayer perceptron (MLP)) may include a hypothesis function represented by a network of node layers, which are assigned biases and interconnected with weight connections. The hypothesis function can be a nonlinear function (e.g., a highly nonlinear function), which may include nested linear functions and logistic functions, wherein the outermost layer consists of one or more logistic functions. The NN algorithm may include a cost function to minimize the classification error by adjusting the biases and weights through a process of feedforward propagation and backpropagation. When a global minimum value can be reached, the optimized hypothesis function of the layer with its adjusted biases and weights can be considered to have been trained and constitutes a model for which a training process has been generated.

[0220] As the first stage of the machine learning lifecycle, data collection can be performed for machine learning. Data collection can include steps such as identifying various data sources, collecting data from the data sources, integrating the data, and the like. For example, in order to train a machine learning model for predicting surgical complications and / or postoperative recovery rates, a data source containing preoperative data (such as the patient's medical conditions and biomarker measurement data) can be identified. Such data sources can be the patient's electronic medical record (EMR), a computing system that stores the patient's preoperative biomarker measurement data, and / or other similar data stores. Data from such data sources can be retrieved and stored in a central location for further processing in the machine learning lifecycle. Data from such data sources can be linked (e.g., logically linked) and can be accessed as if they were centrally stored. Surgical data and / or postoperative data can be similarly identified and collected. In addition, the collected data can be integrated. In an example, the patient's preoperative medical record data, preoperative biomarker measurement data, preoperative data, surgical data, and / or postoperative data can be combined into the patient's record. The patient's record can be an EMR.

[0221] As another stage in the machine learning lifecycle, data preparation can be performed for machine learning. Data preparation can include data preprocessing steps such as data formatting, data cleaning, and data sampling. For example, the collected data may not be in a data format suitable for training the model. Such data records can be converted into a flat file format for model training. Such data can be mapped to numerical values for model training. Such identifying data can be deleted before model training. For example, identifying data can be deleted for privacy reasons. As another example, data may be deleted because there may be more data available than for model training. In this case, a subset of the available data can be randomly sampled and selected for model training, and the remaining data can be discarded.

[0222] Data preparation can include data transformation procedures (e.g., after preprocessing), such as scaling and aggregation. For example, preprocessed data can include data values at various scales. These values can be scaled up or down (e.g., between 0 and 1) for model training. For example, preprocessed data can include data values that carry more meaning when aggregated.

[0223] Model training can be another aspect of the machine learning life cycle. The model training process as described herein may depend on the machine learning algorithm used. After a model has been trained, cross-validated, and tested, it can be considered to be properly trained. Therefore, the dataset from the data preparation phase (e.g., the input dataset) can be divided into a training dataset (e.g., 60% of the input dataset), a validation dataset (e.g., 20% of the input dataset), and a test dataset (e.g., 20% of the input dataset). After the model has been trained on the training dataset, the model can be run against the validation dataset to reduce overfitting. If the accuracy of the model continues to improve, but when the model is run against the validation dataset, the accuracy of the model decreases, this may indicate that there is an overfitting problem. The test dataset can be used to test the accuracy of the final model to determine whether it is ready for deployment or may require more training.

[0224] Model deployment can be another aspect of the machine learning lifecycle. The model can be deployed as part of a standalone computer program. It can also be deployed as part of a larger computing system. Model performance parameters can be used to deploy the model. These performance parameters can monitor model accuracy when the model is used to make predictions on production datasets. For example, these parameters can track the true and false positives of a classification model. These parameters can also store the true and false positives for further processing to improve the model's accuracy.

[0225] Post-deployment model updates can be another aspect of the machine learning cycle. For example, as true positives and / or false positives are predicted on production data, the deployed model can be updated. In one example, for a deployed MLP model for classification, when a false positive occurs, the deployed MLP model can be updated to increase the likely cutoff for predicting a true positive, thereby reducing the false positives. In one example, for a deployed MLP model for classification, when a false negative occurs, the deployed MLP model can be updated to increase the likely cutoff for predicting a true positive, thereby reducing the false negatives. In one example, for a deployed MLP model for surgical complication classification, when both false positives and false negatives occur, the deployed MLP model can be updated to decrease the likely cutoff for predicting a true positive, thereby reducing the false negatives, because predicting a true positive may be less critical than predicting a negative.

[0226] For example, the deployed model may be updated as more real-time production data becomes available as training data. In this case, the deployed model may be further trained, validated, and tested using this additional real-time production data. In one example, the updated biases and weights of the further trained MLP model may update the biases and weights of the deployed MLP model. Those skilled in the art will recognize that post-deployment model updates may not occur all at once and may occur at a frequency appropriate to improve the accuracy of the deployed model.

[0227] ML techniques can be used independently of one another or in combination. Different problems and / or datasets may benefit from using different ML techniques (e.g., combinations of ML techniques). Different training types for a model may be more appropriate for a particular problem and / or dataset. The optimal algorithm (e.g., combination of ML techniques) and / or training type can be determined for a particular application, problem, and / or dataset. For example, a process can be performed for one or more of: selecting a data reduction type, selecting a configuration for a model and / or algorithm, determining where to reduce data, selecting the efficiency of the reduction and / or results, and / or the like.

[0228] For example, an ML technique and / or combination of ML techniques may be determined for a particular problem and / or use case. Multiple data reduction and / or data analysis processes may be performed to determine the accuracy, efficiency, and / or compatibility associated with a data set. For example, a first ML technique (e.g., a first set of combined ML techniques) may be used on a data set to perform data reduction and / or data analysis. The first ML technique may produce a first output. Similarly, a second ML technique (e.g., a second set of combined ML techniques) may be used on a data set (e.g., the same data set) to perform data reduction and / or data analysis. The second ML technique may produce a second output. The first output and the second output may be compared to determine which ML technique produces a more desirable result (e.g., a more efficient result, a more accurate result). Multiple ML techniques may be compared on the same data set to determine the best ML technique to use on similar data sets and / or problems in the future.

[0229] In an example, in a medical context, a surgeon or healthcare professional can provide feedback to an ML technique and / or model used on a dataset. The surgeon can input the feedback into the weighted results of the ML model. The feedback can be used as input by the model to determine reduction methods for future analysis.

[0230] In an example, a data analysis method (e.g., an ML technique to be used in the data analysis method) can be determined based on the data set itself. For example, the source of the data may affect the type of data analysis method to be used on the data set. Available system resources can be used to determine the data analysis method to be used on a given data set. For example, the amount of data can be considered in determining the data analysis method. For example, the need for a data set beyond the local processing level or operational response amplitude can be considered (e.g., small device changes can be made using local data, while major device operational changes may require global compilation and verification).

[0231] Such ML techniques can be applied to surgical information (e.g., 7A to 7D The combination of information flows of surgical information in the ML model to generate useful ML models.

[0232] A computing device, such as a surgical hub, may configure an operating range (e.g., an allowable operating range) associated with a surgical device. The operating range may be or may include an upper envelope and a lower envelope of allowable inputs for controlling the surgical device. The computing device may determine the operating range to control the surgical device to perform a surgical step associated with a surgical procedure. The computing device may analyze data (e.g., collected data associated with an operating range for a surgical step performed by one or more healthcare professionals (HCPs). The computing device may use the data to train machine learning and may provide an operating range appropriate for the surgical step. The computing device may provide the operating range, e.g., the allowable operating range, to the HCP who is about to perform the surgical step.

[0233] A device (such as a surgical device) may receive and / or be configured with an operating range, e.g., an allowable operating range, for controlling the surgical device to perform a surgical procedure. For example, a surgical device being used in a surgical procedure in an operating room may be configured with an operating range. The operating range may have an upper range and a lower range for controlling the surgical device. The configured operating ranges described herein may be or may include a predefined envelope.

[0234] The configured operating range and / or predefined envelope may be associated with the magnitude of the functional adaptation. For example, the device may be a motor-controlled surgical device. The motor-controlled surgical device may have a predefined operating program and the ability to change the operation of the motor-controlled surgical device based on the current surgical step and / or the current situation during the surgical operation. The motor-controlled surgical device may be defined by an operating range, a predefined envelope, and / or an adjustment window. For example, the motor-controlled surgical device may allow changes in the operation of the device within the operating range, the predefined envelope, and / or the adjustment window. If the motor-controlled surgical device determines that the change in the operation of the device falls outside the operating range, the predefined envelope, and / or the adjustment window, the motor-controlled surgical device may prevent the change in operation. The motor-controlled surgical device may send an alert (e.g., an alert message) to the HCP.

[0235] The device may have different operating ranges, predefined envelopes, and / or adjustment windows. For example, the device may have a larger range adjustment for the operating range, predefined envelope, and / or adjustment window in response to an HCP-initiated update and / or after receiving a positive response from the HCP. If the adjustment is generated based on an ML model (e.g., using an ML algorithm), the device may have a smaller adjustment range (e.g., smaller than that involving HCP control).

[0236] In an example, a device may collect operational data. The device may use the collected operational data to train an ML model. Based on the ML-trained model, the device may generate configuration information associated with an operating range, a predefined envelope, and / or an adjustment window. Additionally and / or alternatively, one or more other devices may receive aggregated data associated with the surgical device. The one or more other devices may use the aggregated data to train an ML model. The device may use the ML model to generate configuration information associated with an allowable operating range, an allowable predefined envelope, and / or an allowable adjustment window.

[0237] In an example, the device may receive configuration information from a computing device, such as a surgical hub. For example, the computing device may send configuration information including an operating range, a predefined envelope, and / or an adjustment window to the device. Based on the configuration information, the device may operate within the operating range, the predefined envelope, and / or the adjustment window.

[0238] In an example, the device may determine and / or receive from a computing device a determination that an increased risk of collateral damage is imminent. In an example, the device may determine that the patient's condition has changed (e.g., suddenly changed) and / or an emergency situation has occurred. Based on this determination, the device may have adaptive configuration information associated with the operating range, predefined envelope, and / or adjustment window. For example, if the patient's condition has changed and / or if an emergency situation has occurred, the device may allow for a larger operating range window, predefined envelope, and / or adjustment window.

[0239] A device, such as a surgical device, may, for example, receive and / or be configured with an allowable operating range for a surgical procedure based on a trained ML model. For example, a surgical device may receive and / or be configured with an allowable operating range based on a trained ML model to control the device. The allowable operating range may be generated based on and / or using data from the trained ML model. The allowable operating range may be changed (e.g., adaptively changed) based on data available to the device (e.g., a computing device) and / or information from the trained ML model (e.g., generated using an ML process and / or ML algorithm). The allowable operating range may be used and / or the allowable operating range may be configured to provide a predetermined and / or estimated allowable control range for controlling the device during a surgical procedure, e.g., based on information from the trained ML model.

[0240] The data for the allowable operating range may be derived from an ML-trained model. For example, the data from the ML-trained model may be based on data associated with at least one of the patient, the HCP, the surgical device, the current surgical procedure, the risks involved in the surgical procedure, user input, the magnitude of the risk of failure, the risk of expected and / or unintended consequences, and / or the like.

[0241] In an example, the data associated with the patient may be or may include body mass index (BMI), height, weight, medical history, and / or the like. In an example, the data associated with the HCP may be or may include experience, such as the number of surgical procedures performed, success rates and / or failure rates, preferred settings for using a device, adjusting device configurations and / or data associated with success rates, trends in data from other HCPs performing the same surgical procedure, and / or the like.

[0242] ML processes and / or ML algorithms can be used to analyze data as described herein. For example, a device can use the data to train an ML model and provide an allowable operating range. The allowable operating range generated by the trained ML model and / or ML algorithm can limit control of the device based on, for example, frequency, success rate, magnitude of past changes, and / or the like. For example, the allowable operating range can be used to prevent changes to control devices from cascading effects (e.g., causing unintended large effects and / or self-propagating problems).

[0243] In an example, a device may be configured with an estimated allowable operating range to control the device. For example, a surgical device may receive the estimated allowable operating range to control the surgical device externally (e.g., from a computing device, a surgical hub, a cloud network, and / or the like) for a surgical procedure. In an example, the device may be configured and / or use an ML algorithm and determine the estimated allowable operating range to control the device, e.g., as described herein.

[0244] In an example, a device may configure an ML algorithm to analyze data, such as operational data. Based on the analysis of the data, the device may use the data to train an ML model. The device may use the data from the ML model to adjust (e.g., automatically adjust) the behavior of a control algorithm operation. The control algorithm operation may be configured and / or configured to provide an allowable operating range as described herein. For example, the device may configure the ML algorithm to adjust the behavior of future control algorithm operations that provide an allowable operating range based on patterns determined from the data. The device may configure the ML algorithm to have limits on the magnitude and / or frequency of adjustments. In an example, the device may configure the ML algorithm to have fixed limits on adjustments. For example, the device may configure the ML algorithm to allow no more than two adjustments per week, no more than 5% upward adjustments, 10% downward adjustments, and / or the like. The device may configure the ML algorithm to limit adjustments based on an aspect of the surgical procedure (such as risk, overall benefit, user interface operation issues, and / or the like). In an example, the device may configure the ML algorithm to have adjustable limits (e.g., adaptive limits) based on an aspect of the surgical procedure as described herein.

[0245] In an example, a device may configure an ML algorithm to limit adjustments based on the effect and / or frequency of previous adjustments. For example, if the device determines that one or more large adjustments occurred in the past (e.g., recently), the device may configure the ML algorithm to limit adjustments (e.g., one or more future adjustments). Based on limiting adjustments, the device may reduce the potential impact of previous adjustments for a predetermined amount of time. For example, limiting adjustments may limit the harmful magnitude of previous adjustments, the type and / or timing of future adjustments, and / or the like. Limiting adjustments may provide improvements and / or, for example, use of past adjustments to make larger directional adjustments.

[0246] In an example, a device may configure an ML algorithm to limit adjustments based on historical adaptation. For example, the device may be configured to limit adjustments based on a user, a surgical procedure, past usage data of the device, and / or the like. The device may configure an ML algorithm to compare an output (e.g., actual performance of the device using a configured allowable operating range) with one or more similar previous outputs. Based on the comparison, the device determines that the current output is within normal limits (e.g., thresholds and / or acceptable operating limits). For example, the device may be configured (e.g., initially configured) for a scalpel device where the allowable operating range may be 30 mm / s. The device may compare the allowable operating range of 30 mm / s with one or more historical output change recommendations for the same scalpel device used in the same surgical procedure. For example, the device may compare the allowable operating range, for example, before displaying the allowable operating range to the HCP. The device may compare historical output change recommendations from a local database, an edge and / or fog network, a cloud network, and / or the like. Based on the comparison, the device may perform checks (e.g., additional checks) on the allowable operating range. The device can adjust and / or compensate the allowable operating range based on the comparison and present the adjusted / compensated allowable operating range to the HCP. For example, as described herein, the device can be configured (e.g., initially configured) with an allowable operating range of 30 mm / s for controlling a surgical knife. Based on a comparison with historical data associated with the device, the patient, and the target action, the device can determine that the allowable operating range can be adjusted to 18 mm / s, for example, to have a higher success rate.

[0247] Figure 9 Flowchart 50900 illustrates a device (such as a computing device) determining an allowable range of operation to control a surgical device. As illustrated in 50902, a computing device such as a surgical hub may receive surgical operation data. The surgical operation data may be or may include data and / or information associated with a surgical operation. For example, the surgical operation data may be associated with surgical information and / or may include surgical information (e.g., information about 7A to 7D ). In an example, the surgical procedure data may be or may include at least one of patient information, HCP information, surgical procedure information, information associated with a surgical device to be used for the surgical procedure, and / or the like.

[0248] As described herein, patient information may be or may include BMI, weight, height, blood type, medical history, scans, lab results, and / or the like. HCP information may be or may include experience associated with the HCP, expertise associated with the HCP, the number of times the HCP has performed the current surgical procedure, preferred settings for the HCP, and / or the like. Surgical procedure information may be or may include one or more surgical procedures associated with the surgical procedure, one or more surgical devices associated with the surgical procedure, patient information associated with the surgical procedure, one or more HCPs associated with the surgical procedure, and / or the like. Information associated with a surgical device may be or may include manufacturer, usage history (e.g., number of failures associated with the device), service history of the surgical device, battery level, and / or the like.

[0249] As illustrated in 50904, the computing device may identify a surgical device to be used in a surgical procedure and / or a surgical step to be performed in the surgical procedure. For example, based on the surgical procedure data, the computing device may identify a surgical device to be used in a surgical procedure and / or a surgical step to be performed in the surgical procedure.

[0250] A computing device may determine an allowable operating range associated with a surgical device to be used for a surgical procedure. For example, as illustrated in 50906, the computing device may determine the allowable operating range based on at least one of an identified surgical device (e.g., as illustrated in 50904), an identified surgical step (e.g., as illustrated in 50904), and / or received surgical procedure data (e.g., as illustrated in 50902). As described herein, the computing device may train an ML model (e.g., using an ML algorithm and / or ML process) using the surgical procedure data, the identified surgical device, and / or the identified surgical step. Based on the data associated with the ML model, the computing device may determine the allowable operating range. For example, based on data from the ML-trained model, the computing device may analyze the usage history associated with the surgical device used for a surgical step performed by a HCP. The trained ML model data may provide a control input range with a high success rate for the current surgical step. Based on this analysis, the computing device may provide an allowable operating range for controlling the surgical device to perform the surgical step.The provision of an allowable operating range for controlling a surgical device disclosed herein may be further described in at least one of the following: U.S. patent application No. 16 / 209,423, filed on December 4, 2018, entitled “Method Of Compressing Tissue Within A Stapling Device And Simultaneously Displaying The Location Of The Tissue Within The Jaws”; U.S. Patent No. 10,881,399, filed on January 5, 2021, entitled “Techniques For Adaptive ControlOf Motor Velocity Of A Surgical Stapling And Cutting Instrument”; U.S. patent application No. 16 / 458,103, filed on June 30, 2019, entitled “Packaging For A Replaceable ComponentOf A Surgical Stapling System”; U.S. patent application No. 16 / 458,103, filed on June 30, 2019, entitled “Control Of Advancement Rate And Application Force Based On No. 10,390,895, entitled “Methods, Systems, And Devices For Controlling Electrosurgical Tools,” issued on March 2, 2021; U.S. patent application Ser. No. 16 / 209,458, entitled “Method For Smart Energy Device Infrastructure,” filed on December 4, 2018; and U.S. Patent No. 10,842,523, entitled “Modular Battery Powered Handheld Surgical Instrument And Methods Therefor,” issued on November 24, 2020, the entireties of which are incorporated herein by reference.

[0251] As described herein, a computing device may use and / or be configured to use data to train an ML model, and the computing device may utilize data from the trained ML model to determine an allowable range of operation. Surgical information associated with the same surgical procedure performed by other HCPs using the same surgical device (e.g., as described herein with respect to 7A to 7DThe 726, 727, 762, 766 described herein may be configured as one or more inputs to the ML model. These inputs may be used, for example, to train the ML model using one or more training methods suitable for training surgical information. For example, the computing device may use the surgical information to train the ML model using supervised learning (such as a supervised learning algorithm as described herein (e.g., Figures 8A to 8B )) trains an ML model. The output of the ML-trained model (e.g., a supervised learning algorithm) may be or may include appropriate information for a computing device to determine an allowable operating range for a surgical procedure using a surgical device as described herein. For example, the output of the ML-trained model may be or may include a labeled output that provides supervisory feedback that provides an allowable operating range for a surgical procedure using the surgical device.

[0252] As shown in 50908, the computing device may receive an adjustment input configuration. The adjustment input configuration may be configured to be able to control a surgical device to perform a surgical step. In an example, the adjustment input configuration may be an input to increase / decrease a motor associated with a surgical stapler. In an example, the adjustment input configuration may be an input to increase / decrease a current associated with a surgical cutter and / or a cauterization device. The adjustment input configuration may be generated by an ML trained model. As described herein, the computing device may use and / or may be configured to be able to use data from an ML trained model and generate / receive the adjustment input configuration. The computing device may use appropriate data and / or surgical information (e.g., information about 7A to 7D ) as input to train the ML model. For example, the computing device may use surgical data associated with a surgical device used by other HCPs for the same surgical step to train the ML model. The computing device may use and / or input surgical data associated with the surgical device to train the ML model. As described herein, the computing device may use one or more training methods suitable for training the ML model. For example, the computing device may use surgical data associated with the surgical device and use supervised learning (such as a supervised learning algorithm as described herein (e.g., about Figures 8A to 8B )) to train an ML model. The output of the ML trained model (e.g., supervised learning) may be or may include an adjustment input configuration suitable for the current surgical step. For example, the output of the ML data may be configured to provide an adjustment input configuration to control a surgical device to perform the current surgical step. The output of the ML model may provide an adjustment input configuration to increase or decrease the control input for the surgical device.

[0253] The computing device may determine that the adjustment input configuration is outside the determined allowable operating range, as shown in 50910. The computing device may determine that the adjustment input configuration is within the determined allowable operating range.

[0254] As illustrated in 50912, if the computing device determines that the adjustment input configuration is outside the determined allowable operating range, the computing device may prevent the adjustment input configuration from controlling the surgical device. The computing device may send an alert (e.g., an alert message) to the HCP. In an example, the computing device may send an alert message indicating that the adjustment input configuration is outside the allowable operating range. In an example, the computing device may send an alert message indicating that the adjustment input configuration is outside the allowable operating range. The computing device may send an alert message indicating the risk associated with adjusting the input to an adjustment input configuration outside the allowable operating range. In an example, the computing device may send a message indicating that the adjustment input configuration is within the allowable operating range.

[0255] The computing device may determine the origin of the adjusted input configuration. For example, the computing device may determine whether the adjusted input configuration comes from the HCP (e.g., a surgeon using the surgical device) or is generated by ML data (e.g., by another computing device, a remote server, a cloud, and / or the like).

[0256] In an example, if the computing device determines that the adjustment input configuration is from the HCP, the computing device may send a message to the HCP. The message may include, for example, whether to use an adjustment input configuration outside the allowable operating range to adjust the input to control the surgical device. The computing device may receive a feedback message and / or response from the HCP. For example, the feedback message and / or response may confirm that the adjustment input configuration should be used (e.g., despite being outside the allowable operating range). As described herein, the computing device may request a basis for confirmation from the HCP (e.g., to use an adjustment input configuration outside the allowable operating range). For example, the computing device may inquire / request additional information regarding the adjustment input configuration (e.g., a change in the patient's condition, a malfunction of the surgical device, switching and / or erroneous scan data, erroneous patient information, etc.).

[0257] For example, based on the feedback message, a response from the HCP, and / or a basis, the computing device may allow the adjustment input configuration as input to control the surgical device for the current surgical step. The computing device may send a request message to the HCP. The request message may indicate whether the allowable operating range needs to be revised, for example, based on the adjustment input configuration. In an example, the HCP may indicate that the adjustment input configuration is temporary (e.g., one-time), and that the allowable operating range does not need to be revised. In an example, the HCP may indicate that the adjustment input configuration is permanent, and that the allowable operating range needs to be revised, for example, based on the adjustment input configuration and / or current operating data.

[0258] In an example, if the computing device determines that the adjusted input configuration was generated by the ML model, the computing device may send ML data to the HCP. The ML data may include or include information and / or analysis that led to the adjustment of the input configuration. For example, the ML data may include or include at least one of information about how often other HCPs use the adjusted input configuration for the current surgical step, or the success rate of surgical procedures using the adjusted input configuration.

[0259] If the computing device determines that the adjustment input configuration is within the allowable operating range, the computing device may, for example, use the adjustment input configuration to adjust the input to control the surgical device to perform the surgical step.

[0260] Figure 10 Flowchart 50920 illustrates a device (such as a surgical device) determining an allowable range of operation to control the surgical device. As illustrated in 50922, a device such as a surgical device may receive surgical procedure data. The surgical procedure data may be or may include data and / or information associated with the surgical procedure. In an example, the surgical procedure data may be or may include at least one of patient information, HCP information, surgical procedure information, and / or the like.

[0261] As described herein, patient information may include or include BMI, weight, height, blood type, medical history, scans, laboratory results, and / or the like. HCP information may include or include experience associated with the HCP, expertise associated with the HCP, the number of times the HCP has performed the current surgical procedure, preferred settings for the HCP, and / or the like. Surgical procedure information may include or include one or more surgical procedures associated with the surgical procedure, one or more surgical devices associated with the surgical procedure, patient information associated with the surgical procedure, one or more HCPs associated with the surgical procedure, and / or the like.

[0262] The surgical device may identify a surgical step to be performed in the surgical procedure, as illustrated in 50924. For example, based on the surgical procedure data, the surgical device may identify a surgical step to be performed in the surgical procedure.

[0263] The surgical device may determine an allowable operating range to control the surgical device to be used for a surgical procedure. For example, as illustrated in 50926, the surgical device may determine the allowable operating range based on an identified surgical step (e.g., as illustrated in 50924) and / or received surgical operation data (e.g., as illustrated in 50922). As described herein, the surgical device may use data (e.g., surgical steps and / or surgical operation data) to train an ML model. The surgical device may use data from the ML-trained model to determine the allowable operating range. For example, based on the data from the ML-trained model, the surgical device may analyze the usage history associated with the surgical device used for a surgical step performed by the HCP. The data from the ML model may be configured to provide a control input range that has a high success rate for the current surgical step. Based on this analysis, the surgical device may provide an allowable operating range to control the surgical device to perform the surgical step.

[0264] As shown in 50928, the surgical device can receive an adjustment input configuration. The adjustment input configuration can be configured to control the surgical device to perform a surgical step. In an example, the adjustment input configuration can be an input to increase / decrease the motor associated with the surgical stapler. In an example, the adjustment input configuration can be an input to increase / decrease the current associated with the surgical cutter and / or cautery device.

[0265] The surgical device may determine that the adjustment input configuration is outside of the determined allowable operating range, as shown in 50930. The surgical device may determine that the adjustment input configuration is within the determined allowable operating range.

[0266] As illustrated in 50932, if the surgical device determines that the adjustment input configuration is outside the determined allowable operating range, the surgical device may prevent the adjustment input configuration from controlling the surgical device. The surgical device may send an alert (e.g., an alert message) to the HCP. In an example, the surgical device may send an alert message indicating that the adjustment input configuration is outside the allowable operating range. In an example, the surgical device may send an alert message indicating that the adjustment input configuration is outside the allowable operating range. The surgical device may send an alert message indicating the risk associated with adjusting the input to an adjustment input configuration outside the allowable operating range. In an example, the surgical device may send a message indicating that the adjustment input configuration is within the allowable operating range.

[0267] The surgical device may determine the origin of the adjusted input configuration. For example, the surgical device may determine whether the adjusted input configuration comes from an HCP (e.g., a surgeon using the surgical device) or is generated by an ML model (e.g., by a computing device, a remote server, a cloud, and / or the like).

[0268] In an example, if the surgical device determines that the adjustment input configuration is from the HCP, the surgical device may send a message to the HCP. The message may include, for example, whether the adjustment input configuration outside the allowed operating range is used to adjust the input to control the surgical device. The surgical device may receive a feedback message and / or response from the HCP. For example, the feedback message and / or response may confirm that the adjustment input configuration should be used (e.g., although outside the allowable operating range). As described herein, the surgical device may request a basis for confirmation from the HCP (e.g., to use the adjustment input configuration outside the allowable operating range). For example, the surgical device may inquire / request additional information regarding the adjustment input configuration (e.g., a change in patient condition, a malfunction of the surgical device, switching and / or erroneous scan data, erroneous patient information, etc.).

[0269] For example, based on a feedback message, a response from the HCP, and / or a basis, the surgical device may allow the adjustment input configuration as an input to control the surgical device for the current surgical step. The surgical device may send a request message to the HCP. The request message may indicate whether the allowable operating range needs to be revised, for example, based on the adjustment input configuration. In an example, the HCP may indicate that the adjustment input configuration is temporary (e.g., one-time), and that the allowable operating range does not need to be revised. In an example, the HCP may indicate that the adjustment input configuration is permanent, and that the allowable operating range needs to be revised, for example, based on the adjustment input configuration and / or current operating data.

[0270] In an example, if the surgical device determines that the adjusted input configuration was generated by the ML model, the surgical device may transmit ML data to the HCP. The ML data may include or include information and / or analysis that led to the adjustment of the input configuration. For example, the ML data may include or include at least one of information about how often other HCPs use the adjusted input configuration for the current surgical step, or the success rate of surgical procedures using the adjusted input configuration.

[0271] If the surgical device determines that the adjustment input configuration is within the allowable operating range, the surgical device may, for example, use the adjustment input configuration to adjust the input to control the surgical device to perform the surgical step.

[0272] Figure 11A computing device that determines an allowable operating range associated with a surgical device is illustrated. For example, as described herein, a surgical hub 50944 can determine an allowable operating range 50942 associated with a surgical stapler 50940. The allowable operating range 50942 can be a range of allowable inputs for controlling the surgical stapler 50940. As described herein, a computing device such as a surgical hub 50944 can configure the data described herein to train an ML model and use data from the ML model to determine the allowable operating range 50942 based on, for example, the surgical stapler 50942, surgical steps, and / or surgical operation data.

[0273] Figure 12 A computing device is illustrated that adjusts the permissible operating range associated with a surgical device based on an adjustment input configuration from a healthcare professional. For example, a surgical hub 50954 may receive an adjustment input configuration 50952 from a surgical stapler 50950. As described herein, the surgical hub 50954 may determine whether the adjustment input configuration 50952 is outside the permissible operating range 50956. If the surgical hub 50954 determines that the adjustment input configuration 50952 is outside the permissible operating range 50956, the surgical hub 50954 may determine whether the adjustment input configuration 50952 can be initiated by a healthcare professional (such as a surgeon using the device). If the surgical hub 50954 determines that the adjustment input configuration 50952 is initiated by a surgeon, the surgical hub 50954 may adjust the permissible operating range 50956. For example, the surgical hub 50954 may be configured with a revised allowable operating range 50958 that extends from the allowable operating range 50956 and takes into account the adjusted input configuration 50952 from the surgeon.

[0274] Figure 13A computing device is illustrated that receives an adjustment input configuration that is outside of an allowable operating range, and the adjustment input configuration is from an ML model (e.g., an ML-trained model using an ML process and / or ML algorithm as described herein). For example, a surgical hub 50964 may receive an adjustment input configuration 50962 from a surgical stapler 50960. As described herein, the surgical hub 50964 may determine whether the adjustment input configuration 50962 is outside of an allowable operating range 50966. If the surgical hub 50964 determines that the adjustment input configuration 50962 is outside of the allowable operating range 50966, the surgical hub 50964 may determine whether the adjustment input configuration 50962 can be initiated by the ML-trained model. As described herein, if the surgical hub 50964 determines that the adjustment input configuration 50962 is based on an ML-trained model and is outside of the allowable operating range 50966, the surgical hub 50964 may block the adjustment input configuration 50962. For example, the surgical hub 50964 may configure the surgical stapler 50960 to maintain the allowable operating range 50966 and prevent the adjustment input configuration 50962. In an example, the surgical hub 50964 may resend the allowable operating range 50966 to the surgical stapler 50960, e.g., to confirm that the allowable operating range 50966 has not changed based on the adjustment input configuration 50962.

[0275] In an example, as described herein, a device may receive and / or be configured with an allowable operating range. In an example, as described herein, a device may determine the allowable operating range using, for example, data from an ML trained model. A device, such as a surgical device, may receive control input from a HCP and / or an original equipment manufacturer (OEM) intermediary device to control the surgical device to perform a surgical procedure. The device may include a process for determining and / or user approval, rejection, restriction, and / or adjustment of the received / configured and / or determined allowable operating range.

[0276] The device may receive a control input to control the device. For example, the surgical device may receive a control input to control the device from an HCP who is performing a current surgical step. As described herein, the surgical device may be configured with and / or determine an allowable operating range to control the surgical device. The surgical device may determine whether the control input from the HCP is within the allowable operating range or outside the allowable operating range. If the surgical device determines that the control input is within the allowable operating range, the surgical device may allow the control input to control the device. If the surgical device determines that the control input is outside the allowable operating range, the surgical device may prevent the control input from controlling the device. The surgical device may send an alert (e.g., an alert message) to the HCP indicating that the control input provided is outside the allowable operating range.

[0277] The surgical device may receive feedback from the HCP. The feedback from the HCP may indicate confirmation from the HCP that the control input is outside the allowable operating range. The feedback may include confirmation that the control input is allowed (e.g., if it is outside the allowable operating range) and / or that the allowable operating range is revised based on the HCP's confirmation.

[0278] In an example, the HCP may receive an allowable operating range, for example, before providing a control input. As described herein, the HCP may accept the configured allowable operating range. In an example, the HCP may reject the configured allowable operating range. The HCP may provide an updated operating range. The device may use the updated operating range and configuration data to train an ML model and be configured to provide the updated allowable operating range.

[0279] In an example, the device may use third-party validation. For example, the device may use third-party validation to enable one or more sequential algorithm adjustments. The device may display data from an ML-trained model, results (e.g., relationships, recommendations, control system changes, etc.), and / or the like. Additionally and / or alternatively, the device may display the data (e.g., a simplified version of the data) and enable a third party (e.g., a third-party device) to determine whether utilization of the results is valid and / or warranted.

[0280] In an example, the device may display the results (e.g., data) of the output parameters (such as the allowable operating range) recommended by the ML trained model to complete the task. For example, the device may display the minimized actuation time, the step shift in the output parameter, one or more adjustments made by the HCP based on the HCP's experience, visual, tactile and / or device feedback, sensory feedback, and / or the like. This illustration may allow a third party (e.g., a third party device) and / or the HCP to override, reduce, or eliminate the proposed adjustment data (e.g., the allowable operating range) from the ML model. Feedback from the third party and / or the user may be provided to the device via a display (e.g., a screen associated with the device) and / or via a computing device (e.g., a hub and / or a display associated with the hub).

[0281] If the device determines that the adjustment from the HCP exceeds the preconfigured adjustment (e.g., a large adjustment), the device may interact with the HCP. The device may ask the HCP, supervisor, and / or the like for confirmation of the adjustment as described herein. The device may request the basis for such adjustment. In an example, during a surgical procedure, the device may provide an allowable operating range for the device based on the device moving to the right. The device may receive an adjustment (e.g., a large adjustment) that exceeds a preconfigured threshold from the HCP. The device may request the basis for the adjustment from the HCP. The device may ask the HCP whether the device is moving to the left (e.g., instead of the recommended right movement). If the device receives an affirmative answer from the HCP, the device may provide an updated allowable operating range (e.g., based on a reversal from right to left of the entire position). In an example, the device may ask the HCP whether the switch in movement (e.g., from right to left) is a one-time event, or whether the surgery and / or the allowable operating range needs to be updated (e.g., before the next step is performed).

[0282] In an example, if the HCP makes an adjustment that is greater than a preconfigured threshold as described herein, the device may request a basis from the HCP. The device may inquire whether the scan data was incorrectly labeled and / or entered into the device. The device may inquire whether right and left were switched and whether the change was not incorrectly labeled in time (e.g., before the data was entered into the device). The device may ask for confirmation whether the patient underwent surgery before it was not recorded, entered, and / or forgotten. The device may inquire whether one or more marks that should have been present for the patient are not present. For example, the device may ask for confirmation whether the patient is missing a kidney and / or whether the kidney is being used as a reference for another operation. The device may inquire whether the wrong patient is on the operating table. The device may display that the surgical plan and / or scan does not match the input to the device. The device may request confirmation from the HCP and / or the device may ask whether to continue with the surgical plan or make adjustments based on the input.

[0283] The device may determine that a potential input from the HCP may have catastrophic consequences. The device may notify the HCP of the possibility of catastrophic consequences and / or indicate that the potential input may be outside of standard operating ranges. The device may query whether the device determines that a change in the patient's condition has occurred between scans and / or assessments and when a surgical procedure occurred.

[0284] If an update to the surgical plan is needed, the device may request feedback from the HCP. In one example, based on an MRI scan, the device may determine that the patient has a meniscus tear and provide a surgical plan and / or permissible operating range for repairing the meniscus tear. After the scan and / or during the surgical procedure, the device may determine that the remaining portion of the meniscus is completely torn. The device may confirm with the HCP that an update to the surgical plan and / or permissible operating range is needed.

[0285] In one example, the device may provide a surgical plan and / or permissible operating range for gallbladder surgery for a patient. During surgery, based on the input, the device may determine that the patient has cancer. The device may request confirmation from the HCP that an update to the surgical plan and / or permissible operating range is needed.

[0286] The device may configure data from the ML model to determine weighted adjustments (e.g., adjustments to the allowable operating range). For example, the device may configure data from the ML model to determine weighted adjustments based on one or more feedback from the HCP and / or a third party, as described herein. The weighted adjustments may be based on time and / or frequency. In an example, if the device determines that one or more adjustments provide an improvement, the device may increase the frequency of adjustments and / or allow less time for adjustments to be made. In an example, if the device determines that one or more adjustments result in harmful and / or unsuccessful outcomes, the device may decrease the frequency of adjustments and / or allow more time for adjustments to be made.

[0287] The device may determine a weighted adjustment (e.g., an adjustment to the permissible operating range) based on the type of change. In an example, the device may determine a permissible operating range that may be required for surgical steps (e.g., less critical and / or non-life-threatening). The device may provide an adjusted permissible operating range (e.g., more frequent and / or less frequent checklists). In an example, the device may determine a permissible operating range for critical steps and / or important procedures (e.g., surgical and / or non-surgical steps that may involve risk). The device may require more information, one or more confirmation steps, and / or more user confirmations before providing the adjusted permissible operating range.

[0288] The device may utilize weighted responses, for example, to control the magnitude of algorithm adaptation. For example, the device may compile and / or aggregate one or more results (e.g., ideal results). The device may use the compiled and / or aggregated results to have a weighted and / or predefined aggregation list. The device may combine the weighted and / or predefined aggregation list with the current data (e.g., a portion of the current data). The device may request verification (e.g., remote verification) and / or validation. The device may require verification and / or validation and may upload the data to the cloud and / or a remote server for review and / or combination with other system results (e.g., results from other locations and / or facilities).

[0289] Devices, such as computing devices, can collect data from remote servers and / or the cloud. The computing device can use ML models to provide conclusions and provide global device operational changes (e.g., global permissible operating ranges). Global device operational changes (e.g., global permissible operating ranges) can control devices (e.g., one or more facilities using the device). Global device operational changes (e.g., global permissible operating ranges) can verify recommended permissible operating ranges for the device and / or provide one or more proposed changes to the device in a controlled and / or global manner. Global device operational changes can prevent unintentional and / or uncontrolled changes to local devices (e.g., local operations and / or local environment).

[0290] The computing device may compare a proposed change (e.g., a global device operation change) with, for example, competing changes proposed for related devices, related steps, related technologies, and / or the like. The computing device may, for example, prevent a constant cycle of changes based on the comparison and / or modifications from related devices.

[0291] In an example, the device may be configured with data from an ML-trained model and process the output based on the collected parameters. If the device determines that the output is complete, the device may have one or more parameters (e.g., one or more additional parameters). The device may return one or more parameters (e.g., including one or more additional parameters) and / or defined parameters and weight the one or more parameters and / or defined parameters. The device may modify the output based on a weighting factor. For example, the device may have a tissue disease state and / or identification of a blood vessel and / or artery. The disease state and / or identification may modify the output and / or weight the output, for example, to compensate for the parameter.

[0292] The device may determine a threshold (e.g., a threshold function) that limits the boundaries of feasible adjustments. The device may determine threshold boundaries for the magnitude of algorithm changes. For example, the device may configure a functional algorithm to determine one or more boundaries of a functional range. The device may determine whether an adjustment is out of bounds or within bounds. For example, the device may determine whether an adjustment is out of bounds based on data (e.g., data entering / leaving an ML model). The ML model may use patient information such as BMI, height, weight, and / or the like. The ML model may use the patient information and generate a predictive model configuration. For example, the ML model may be configured to be able to guess, based on the patient information, what properties of the tissue, the functional range within which the device may operate, and / or the like before a surgical procedure. During the surgical procedure, the device may compare the device's performance data with the predictive modeling configuration. The device may determine whether there is drift and / or error in the predicted model and / or the procedure. Based on the determination that there is drift and / or error, the device may adjust the predicted model to within acceptable limits.

[0293] In an example, the device may determine whether the patient's tissue is more difficult to transect once or more difficult to transect consistently. In an example, the device may determine whether there is a problem with the device and / or sensor. In an example, the device may determine whether there is a problem with a predictive model of the patient's tissue composition. Based on this determination, the device may provide an alert to the HCP. For example, the device may issue a warning to the HCP, recommending a different cartridge for the current surgical procedure (e.g., relative to the cartridge used for the current surgical procedure).

[0294] The device can use historical data and predict one or more devices that are most effective for one or more surgical steps. Based on historical data, the device can provide and / or integrate product recommendations to purchasing and hospital inventory management systems. For example, the device can ensure that the hospital has sufficient warehouses (e.g., blue, white, gold and / or similar warehouses) in stock. If one or more warehouses are insufficiently stocked, the device can send an alert (e.g., an alert message). The device can be adjusted in the event of a supply chain disruption. The device can change one or more recommendations, for example, to allocate resources more efficiently. For example, if surgery A uses blue and / or gold warehouses and surgery B would be more effective with gold warehouses, the device can recommend using blue warehouses for surgery A (e.g., instead of using gold warehouses). The device can integrate and / or provide product recommendations to higher-level management, for example, for the entire hospital network to more efficiently deploy resources and / or supplies when needed.

[0295] When using a modern surgical device to perform a surgical step in a surgical procedure, there are typically many possible operational configurations for the surgical device for its various operational inputs. For example, a surgical device may have an output power delivery between 0W and 50W, a power delivery time of 10 seconds to 15 seconds, and a maximum temperature setting of 45°C. Determining how to adjust the operational input for a surgical step is not straightforward because it can vary depending on the patient undergoing the surgical procedure, the operator performing the surgical procedure, and the surgical device used to perform the surgical procedure.

[0296] Typically, operators use their experience with past surgical procedures to tailor operating inputs to a specific patient using a specific device. However, reliance on operator experience leaves open the possibility that the operating inputs may be inappropriate for the patient or surgical device, potentially leading to a poor surgical outcome for the patient. Furthermore, operating inputs may be accidentally set to unsafe levels by the operator. However, there are no mechanisms to prevent the use of inappropriate or unsafe operating inputs on surgical devices used for surgical procedures.

[0297] As used herein, the terms "operator" and "healthcare professional (HCP)" are interchangeable.

[0298] As used herein, the terms "machine learning process" and "machine learning model" are interchangeable. As used herein, the terms "machine learning technique" and "machine learning algorithm" are interchangeable. Where the term "machine learning" is used herein, this is a reference to either a machine learning model or a machine learning algorithm, as will be apparent from the context in which the term is used.

[0299] As used herein, the term "machine learning data" refers to data output from a machine learning model. As used herein, the terms "machine learning model metadata" and "machine learning data information" are interchangeable.

[0300] As used herein, the term "operating range" defines the range of operational inputs used to control a surgical device, preferably for performing a surgical step. The range may correspond to all possible operational configurations of the device for a particular operational input. For example, a surgical device may have an output power delivery between 0 W and 50 W, and thus 0 W to 50 W is the operating range of the surgical device for output power. The operating range may have an upper boundary but no lower boundary. Alternatively, the operating range may have a lower boundary but no upper boundary. As another alternative, the operating range may have an upper boundary and a lower boundary that define an envelope. In some embodiments, the operating range may include an upper envelope and a lower envelope. The terms "predefined envelope" and "adjustment window" may also be used herein to refer to an "operating range."

[0301] As used herein, the term "allowable operating range" defines the range of operational inputs used to control a surgical device, preferably for performing a surgical procedure. The allowable operating range is a subset of all possible operational configurations of a device for a particular operational input. The allowable operating range may be based on an operating range that is safe for performing a particular surgical procedure. For example, power delivery may have an allowable operating range of 45W to 50W (of an operating range of 0W to 50W). The allowable operating range may be based on previous operating ranges for performing a particular surgical procedure, i.e., from historical surgical operation data. In embodiments where the allowable operating range is based on previous operating ranges for performing a particular surgical procedure, machine learning may be used to train a model to determine the allowable operating range. Like the operating range, the allowable operating range may have an upper boundary but no lower boundary. Alternatively, the allowable operating range may have a lower boundary but no upper boundary. As another alternative, the allowable operating range may have upper and lower boundaries that define an envelope. In some embodiments, the allowable operating range may include an upper envelope and a lower envelope.

[0302] The term "adjustment input configuration" is a change in (i.e., an adjustment to) the operational configuration of a particular operational input of a surgical device. The adjustment input configuration is within the operational range of the surgical device (i.e., one of the possible operational configurations of the device for the particular operational input). However, the adjustment input configuration may not be within the permissible operational range of the surgical device, i.e., within a subset of all possible operational configurations of the device for the particular operational input. The adjustment may be a manual adjustment (i.e., initiated by an operator) or an automatic adjustment (i.e., initiated by the device). The automatic adjustment may have one or more of the following: fixed limits on the adjustment; limits on the magnitude and / or frequency of the adjustment; and adjustments based on an aspect of the surgical procedure (such as surgical risk, overall benefit of the surgical outcome, or issues with user interface operation). The ML model may be trained to determine the adjustment input configuration as input to control the surgical device to perform the surgical step using previous adjustment input configurations (i.e., from historical surgical operation data).

[0303] The following is a non-exhaustive list of embodiments that may or may not be claimed: 1. An apparatus comprising:

[0304] A processor configured to:

[0305] receiving surgical procedure data associated with a surgical procedure, wherein the surgical procedure data includes at least one of patient information, healthcare professional (HCP) information, or information associated with a surgical device to be used for the surgical procedure;

[0306] identifying the surgical device to be used for the surgical procedure and surgical steps associated with the surgical procedure based on the surgical procedure data;

[0307] determining an allowable operating range associated with the surgical device based on the surgical device, the surgical step, and the surgical operation data, wherein the allowable operating range is an operating range for controlling the surgical device to perform the surgical step;

[0308] receiving an adjustment input configuration, wherein the adjustment input configuration is configured to control the surgical device to perform the surgical step;

[0309] determining that the adjustment input configuration is within the determined allowable operating range; and

[0310] Based on a determination that the adjustment input configuration is outside the allowable operating range, the adjustment input configuration is prevented from controlling the surgical device.

[0311] 2. The apparatus of embodiment 1, wherein the processor is configured to:

[0312] determining whether the adjustment input configuration is from the HCP or generated by a machine learning (ML) model; and

[0313] An alert is sent to the HCP, wherein the alert indicates that the adjusted input configuration is outside of an allowed operating range, and wherein the alert further indicates a risk associated with adjusting an input to the adjusted input configuration outside of the allowed operating range.

[0314] 3. The apparatus of embodiment 2, wherein the processor is configured to:

[0315] sending a message to the HCP based on a determination from the HCP of the adjustment input configuration, wherein the message includes whether the input to control the surgical device is adjusted using the adjustment input configuration that is outside of the allowed operating range;

[0316] receiving a feedback message from the HCP, wherein the feedback message confirms using the adjusted input configuration as the input to control the surgical device; and

[0317] Based on the received feedback message, the adjustment input configuration is enabled as the input to control the surgical device to perform the surgical step.

[0318] 4. The apparatus of embodiment 3, wherein the processor is configured to:

[0319] A request message is sent to the HCP, wherein the request message includes whether to determine a revised allowable operating range based on the adjustment input configuration.

[0320] 5. The apparatus of embodiment 2, wherein the processor is configured to:

[0321] Based on a determination that the adjusted input configuration was generated by the ML model, ML data is transmitted, wherein the ML data includes at least one of frequency information of how other HCPs used the adjusted input configuration for the surgical step, or a success rate of the surgical procedure using the adjusted input configuration.

[0322] 6. The apparatus of embodiment 1, wherein the processor is configured to:

[0323] Based on a determination that the adjustment input configuration is within the allowable operating range, an adjustment input is applied using the adjustment input configuration to control the surgical device to perform the surgical step.

[0324] 7. A method comprising:

[0325] receiving surgical procedure data associated with a surgical procedure, wherein the surgical procedure data includes information associated with at least one of a patient, a healthcare professional (HCP), or a surgical device to be used in the surgical procedure;

[0326] identifying the surgical device to be used for the surgical procedure and surgical steps associated with the surgical procedure based on the surgical procedure data;

[0327] determining an allowable operating range associated with the surgical device based on the surgical device, the surgical step, and the surgical operation data, wherein the allowable operating range is an operating range for controlling the surgical device to perform the surgical step;

[0328] receiving an adjustment input configuration, wherein the adjustment input configuration is configured to control the surgical device to perform the surgical step;

[0329] determining that the adjustment input configuration is within the determined allowable operating range; and

[0330] Based on a determination that the adjustment input configuration is outside the allowable operating range, the adjustment input configuration is prevented from controlling the surgical device.

[0331] 8. The method according to embodiment 7, wherein the method comprises:

[0332] determining whether the adjustment input configuration is from the HCP or generated by a machine learning (ML) model; and

[0333] An alert is sent to the HCP, wherein the alert indicates that the adjusted input configuration is outside of an allowed operating range, and wherein the alert further indicates a risk associated with adjusting an input to the adjusted input configuration outside of the allowed operating range.

[0334] 9. The method according to embodiment 8, wherein the method comprises:

[0335] sending a message to the HCP based on a determination from the HCP of the adjustment input configuration, wherein the message includes whether the input to control the surgical device is adjusted using the adjustment input configuration that is outside of the allowed operating range;

[0336] receiving a feedback message from the HCP, wherein the feedback message confirms using the adjusted input configuration as the input to control the surgical device; and

[0337] The adjustment input configuration is allowed to serve as the input to control the surgical device to perform the surgical step.

[0338] 10. The method according to embodiment 9, wherein the method comprises:

[0339] A request message is sent to the HCP, wherein the request message includes whether to determine a revised allowable operating range based on the adjustment input configuration.

[0340] 11. The method according to embodiment 8, wherein the method comprises:

[0341] Based on a determination that the adjusted input configuration was generated by the ML model, ML data is transmitted, wherein the ML data includes at least one of frequency information of how other HCPs used the adjusted input configuration for the surgical step, or a success rate of the surgical procedure using the adjusted input configuration.

[0342] 12. The method according to embodiment 7, wherein the method comprises:

[0343] Based on a determination that the adjustment input configuration is within the allowable operating range, an adjustment input is applied using the adjustment input configuration to control the surgical device to perform the surgical step.

[0344] 13. A surgical device, comprising:

[0345] A processor configured to:

[0346] receiving surgical procedure data associated with a surgical procedure, wherein the surgical procedure data includes at least one of patient information, healthcare professional (HCP) information, or information associated with the surgical device to be used for the surgical procedure;

[0347] identifying a surgical step associated with the surgical procedure based on the surgical procedure data;

[0348] determining an allowable operating range associated with the surgical device based on the identified surgical step and the surgical operation data, wherein the allowable operating range is an operating range for controlling the surgical device to perform the surgical step;

[0349] receiving an adjustment input configuration, wherein the adjustment input configuration is configured to control the surgical device to perform the surgical step;

[0350] determining that the adjustment input configuration is within the determined allowable operating range; and

[0351] Based on a determination that the adjustment input configuration is outside the allowable operating range, the adjustment input configuration is prevented from controlling the surgical device.

[0352] 14. The surgical device of embodiment 13, wherein the processor is configured to:

[0353] An alert is sent to the HCP, wherein the alert indicates that the adjustment input configuration is outside of an allowed operating range, and wherein the alert further indicates a risk associated with adjusting an input used to control the surgical device to the adjustment input configuration outside of the allowed operating range.

[0354] 15. The surgical device of embodiment 13, wherein the processor is configured to:

[0355] determining that the adjustment input configuration is from the HCP;

[0356] Based on the determination from the HCP of the adjusted input configuration, a message is sent to the HCP, wherein the message includes whether the allowed operating range is used.

[0357] The adjustment input configuration outside the range is used to adjust the input to control the surgical device.

[0358] 16. The surgical device of embodiment 15, wherein the processor is configured to:

[0359] receiving a feedback message from the HCP, wherein the feedback message confirms using the adjusted input configuration as the input to control the surgical device; and

[0360] Based on the received feedback message, the adjustment input configuration is enabled as the input to control the surgical device to perform the surgical step.

[0361] 17. The surgical device of embodiment 15, wherein the processor is configured to:

[0362] A request message is sent to the HCP, wherein the request message includes whether to determine a revised allowable operating range based on the adjustment input configuration.

[0363] 18. The surgical device of embodiment 13, wherein the processor is configured to:

[0364] A determination is made that the adjustment input configuration is generated by a machine learning (ML) model, and based on the determination that the adjustment input configuration is generated by the ML model, ML model metadata is sent to the HCP.

[0365] 19. The surgical device of embodiment 18, wherein the ML data comprises:

[0366] Other HCPs use at least one of frequency information of the adjusted input configuration for the surgical step, or a success rate of the surgical procedure using the adjusted input configuration.

[0367] 20. The surgical device of embodiment 13, wherein the processor is configured to:

[0368] Based on a determination that the adjustment input configuration is within the allowable operating range, an adjustment input is applied using the adjustment input configuration to control the surgical device to perform the surgical step.

Claims

1. A method executed by a processor, the method comprising: receiving surgical procedure data associated with a surgical procedure, wherein the surgical procedure data includes data associated with at least one of a patient, an operator, or a surgical device to be used in the surgical procedure; identifying a surgical step associated with the surgical procedure based on the surgical procedure data; determining an allowable operating range associated with the surgical device based on the surgical step and the surgical operation data, wherein the allowable operating range is an operating range for controlling the surgical device to perform the surgical step; receiving an adjustment input configuration, wherein the adjustment input configuration is configured to control the surgical device to perform the surgical step; determining whether the adjustment input configuration is within the determined allowable operating range; and Based on a determination that the adjustment input configuration is not within the allowable operating range, the adjustment input configuration is prevented from controlling the surgical device.

2. The method according to claim 1, wherein Determining the allowable operating range associated with the surgical device includes: inputting the surgical operation data into a machine learning (ML) model, wherein the ML model is trained to use historical surgical operation data to determine the allowable operating range of the surgical device for performing the surgical step.

3. The method according to claim 1, wherein Determining the allowable operating range associated with the surgical device includes retrieving the allowable operating range of the surgical device for performing the surgical step from historical surgical procedure data.

4. The method according to claim 1, further comprising: identifying the surgical device to be used for the surgical procedure based on the surgical procedure data, Wherein determining the allowable operating range associated with the surgical device is further based on the surgical device.

5. The method according to any one of claims 1 to 4, further comprising: An alert is sent to the operator, wherein the alert indicates that the adjustment input configuration is not within the allowable operating range, and wherein the alert further indicates a surgical risk associated with adjusting an input to the adjustment input configuration that is not within the allowable operating range.

6. The method according to any one of claims 1 to 5, wherein The adjustment input configuration is received from the operator or generated by an ML model, wherein the ML model is trained to use historical surgical operation data to determine the adjustment input configuration as the input to control the surgical device to perform the surgical step.

7. The method according to claim 6, further comprising: determining whether the adjustment input configuration is from the operator; Based on the determination from the operator that the adjustment input configuration is present, a message is sent to the operator to confirm whether to adjust the input to control the surgical device using the adjustment input configuration that is not within the allowable operating range.

8. The method according to claim 7, further comprising: receiving a feedback message from the operator confirming use of the adjusted input configuration as the input for controlling the surgical device; as well as The adjustment input configuration is allowed to serve as the input to control the surgical device to perform the surgical step.

9. The method according to claim 8, further comprising: A request message is sent to the operator to confirm whether to modify the allowable operating range based on the adjustment input configuration.

10. The method according to claim 6, further comprising: determining whether the adjustment input configuration is generated by a machine learning (ML) model, Based on the determination that the adjustment input configuration was generated by the ML model, ML model metadata is sent to the operator.

11. The method according to claim 10, wherein: The ML model metadata includes at least one of: frequency information of other operators using the adjusted input configuration for the surgical step, or a surgical success rate of the surgical operation using the adjusted input configuration.

12. The method according to any one of claims 1 to 11, further comprising: Based on the determination that the adjustment input configuration is within the allowable operating range, an adjustment input is used to control the surgical device to perform the surgical step using the adjustment input configuration.

13. A surgical device comprising: A processor configured to execute the method according to any one of claims 1 to 12.

14. An apparatus comprising: A processor configured to execute the method according to any one of claims 1 to 12. 15 . A computer program which, when executed by a processor, causes the processor to perform the method according to claim 1 .

Images