Adaptable range of motion for surgical devices

Abstract

A device, such as a computing device or a surgical device, can receive surgical data associated with a surgical procedure. Based on the surgical data, the device can identify a surgical device to be used in the surgical procedure and / or one or more surgical steps associated with the surgical procedure. Based on the identified surgical device, the one or more surgical steps, and / or the surgical data, the device can determine an allowable operating range for controlling the surgical device for the surgical procedure. The device can receive an adjustment input configuration for controlling the surgical device for the surgical step. The device can determine whether the adjustment input configuration is outside the determined allowable operating range. Based on a determination that the adjustment input configuration is outside the determined allowable operating range, the device can prevent the adjustment input configuration for controlling the surgical device.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is related to the following concurrently filed applications, the contents of each of which are incorporated herein by reference: • U.S. patent application filed with attorney docket number END9438USNP1 entitled "A Method for Advanced Algorithm Support." [Background technology]

[0002] Patient care, in general, improves when it is tailored to the individual. Because every person has different needs, surgical and interventional solutions that center every patient's unique journey can represent an efficient and innovative path to healing. At the same time, the high stakes of patient care, particularly the surgical process, often focus on conservative, repeatable activities.

[0003] For example, innovative medical technologies such as advanced surgical support computing systems and intelligent surgical instruments can improve approaches to patient care and address the specific needs of healthcare providers.

[0004] The ever-increasing availability of data and computing resources makes non-traditional algorithms, such as machine learning algorithms, a particular technological opportunity in the healthcare system. However, incorporating such non-traditional algorithms into any medical technology presents many challenges.

[0005] A computing system, which may include a surgical device and / or a surgical hub, may be configured with machine learning to train data and provide acceptable control input ranges for controlling one or more surgical devices. The acceptable control input ranges may provide recommended input ranges for controlling one or more surgical devices for a healthcare provider (HCP) during a surgical procedure. Machine learning may be used to improve data such as acceptable control input ranges for surgical devices. However, training machine learning may be time consuming and inconvenient. Summary of the Invention [Means for solving the problem]

[0006] According to an embodiment of the present invention, there is provided a method executed by a processor, the method including: receiving surgical data associated with a surgical procedure, the surgical data including data associated with at least one of a patient, an operator, or a surgical device used in the surgical procedure; identifying a surgical step associated with the surgical procedure based on the surgical data; determining an allowable operating range associated with the surgical device based on the surgical step and the surgical data, the allowable operating range being an operating range for controlling the surgical device for the surgical step; receiving an adjustment input configuration, the adjustment input configuration configured to control the surgical device for the surgical step; determining whether the adjustment input configuration is within the determined allowable operating range; and preventing the adjustment input configuration from controlling the surgical device based on a determination that the adjustment input configuration is not within the allowable operating range.

[0007] In this way, surgical data can be utilized to determine the acceptable operating range of the surgical device, rather than simply relying on the operator's experience with past surgeries, particularly surgical steps. By preventing adjustment input configurations that are outside the acceptable operating range, inappropriate or unsafe operating inputs can be prevented from being used in the surgical device for a surgical step.

[0008] In embodiments, data associated with the patient may include body mass index (BMI), height, weight, and / or medical history. In embodiments, data associated with the operator may include the number of times to perform the surgical procedure and / or surgical step, success and / or failure rates for performing the surgical procedure and / or surgical step, preferred input configurations using the surgical device to perform the surgical procedure and / or surgical step, possibilities for adjusting the input configurations to perform the surgical procedure and / or surgical step, and / or data associated with other operators performing the same surgical procedure and / or surgical step.

[0009] In embodiments, the surgical data is received from a surgical device. In embodiments, the surgical data is received from a local database, an edge and / or fog network, and / or a cloud network. The surgical device, the local database, the edge and / or fog network, and / or the cloud network may collect the surgical data.

[0010] In embodiments, surgical procedure data (i.e., data associated with at least one of the patient, the operator, or the surgical device used in the surgery) may be used to identify the surgical device used in the surgery and the surgical steps associated with the surgery by searching a database of planned surgeries using the surgical procedure data. In embodiments, the identified surgical device may be a particular class of surgical device, a particular model of surgical device, or may be a particular surgical device.

[0011] In an embodiment, preventing the adjustment input configuration includes not allowing the input configuration to exceed an acceptable operating range.

[0012] Determining the allowable operating range associated with the surgical device can include inputting the surgical data into a machine learning (ML) model, where the ML model is trained to determine the allowable operating range of the surgical device for performing the surgical step using the historical surgical data.

[0013] In this way, the acceptable operating ranges are not only set by the operator, but also by using past surgical procedure data, including from many different operators. Furthermore, the ML model can use the success of surgical steps as part of its training data (i.e., surgical outcomes) and thus generalize previously used operating ranges to establish acceptable operating ranges that improve surgical outcomes.

[0014] In embodiments, a supervised machine learning algorithm may be used to determine the acceptable operating range. This type of algorithm is trained on labeled data and provides the correct output for each input. By learning the relationship between the input data and the desired outcome, the algorithm can make predictions about the acceptable operating range of the device in different surgical scenarios.

[0015] Determining the allowable range of motion associated with the surgical device can include retrieving the allowable range of motion of the surgical device for performing the surgical step from historical surgical data.

[0016] In this way, the acceptable operating range is not only set by the operator, but also by using past surgical procedure data, including from many different operators.

[0017] The method may further include identifying a surgical device to be used in the surgery based on the surgical data, wherein determining an allowable operating range associated with the surgical device is also based on the surgical device.

[0018] In this way, any unique characteristics of the surgical device can be taken into account in the allowable operating range. For example, if the surgical device is not properly calibrated, the allowable operating range can compensate for this.

[0019] In embodiments, the identified surgical device may be a particular class of surgical device, a particular model of surgical device, or may be a particular surgical device, In embodiments, the allowable operating range may depend on the particular class of surgical device, the particular model of surgical device, and / or the particular surgical device.

[0020] The method may further include sending an alert to an operator, the alert indicating that the adjustment input configuration is not within an acceptable operating range, the alert further indicating a surgical risk associated with adjusting the input to the adjustment input configuration that is not within the acceptable operating range.

[0021] In this way, the operator is informed of the surgical risks of proceeding with the adjustment input configuration, which means that the operator cannot mistakenly use an adjustment input configuration that has a surgical risk associated with it.

[0022] The adjustment input configuration may be received from an operator or generated by an ML model that has been trained using past surgical data to determine the adjustment input configuration as an input for controlling the surgical device for the surgical step.

[0023] The method may further include determining whether the adjustment input configuration is from an operator, and based on a determination that the adjustment input configuration is from the operator, sending a message to the operator to confirm whether to adjust the input for controlling the surgical device using the adjustment input configuration that is not within the allowable operating range.

[0024] In this way, the operator is provided with a check as to whether the adjustment input configuration was intentional, and therefore the possibility of starting a surgical step with an accidental adjustment input configuration is significantly reduced.

[0025] In embodiments, the surgical device has a display configured to display a message sent to the operator. In embodiments, the message may ask the operator to confirm whether to adjust the input by activating a button on the surgical device. In embodiments, the display may be a touchscreen and the message may ask the operator to confirm whether to adjust the input using the touchscreen.

[0026] The method may further include receiving a feedback message from the operator confirming use of the adjusted input configuration as an input for controlling the surgical device, and accepting the adjusted input configuration as an input for controlling the surgical device for the surgical step.

[0027] In embodiments, the operator can send the feedback message by activating a button on the surgical device. In embodiments, the operator can send the feedback message using the touch screen of the device.

[0028] In an embodiment, allowing the adjusted input configuration as an input for controlling the surgical device for the surgical step includes changing the input configuration beyond an allowable operating range.

[0029] The method may further include sending a request message to an operator to confirm whether to modify the allowable operating range based on the adjustment input configuration.

[0030] In this way, feedback from the operator can be used to improve the acceptable operating range.

[0031] In embodiments, once the operator confirms the modification of the acceptable operating range, the modified acceptable operating range is stored in a database of surgical data for future surgical procedures. In embodiments, the modified acceptable operating range may be provided to the ML model that determined the acceptable operating range to further train the ML model.

[0032] The method may further include determining whether the adjusted input configuration was generated by a machine learning (ML) model, and, based on a determination that the adjusted input configuration was generated by the ML model, sending the ML model metadata to an operator.

[0033] In this manner, the ML model metadata may be used by an operator to determine whether the adjusted input configuration generated by the ML model is appropriate. For example, an operator can use ML model metadata, such as frequency information of other operators using the adjusted input configuration for a surgical step, or the surgical success rate of surgical procedures using the adjusted input configuration, to determine whether the adjusted input configuration generated by the ML model should be used.

[0034] In an embodiment, if an operator determines that the adjustment input configuration generated by the ML model may be provided to the ML model that determines the acceptable operating range for further training.

[0035] The ML model metadata can include at least one of frequency information of other operators using the adjusted input configuration for the surgical step or a surgical success rate for surgical procedures using the adjusted input configuration.

[0036] The method may further include adjusting an input for controlling the surgical device for the surgical step using the adjustment input configuration based on determining that the adjustment input configuration is within an acceptable operating range.

[0037] According to a further embodiment of the present invention there is provided a surgical device comprising a processor configured to perform any one of the above methods.

[0038] According to a further embodiment of the present invention there is provided a device (eg, a non-surgical device) comprising a processor configured to perform any one of the above methods.

[0039] According to a further embodiment of the present invention there is provided a computing program which, when executed by a processor, causes the processor to perform any one of the methods described above.

[0040] Disclosed herein are methods, systems, and apparatus for a device, such as a computing device or a surgical device, to determine an allowable operating range for controlling an input associated with a surgical device. The device can use data from a machine learning (ML) model to determine an allowable operating range associated with the surgical device. The device can utilize data from the ML model to improve an artificial intelligence algorithm, reduce the number of iterations used to train the artificial intelligence algorithm, and / or reduce the time it takes to train the machine learning. An adaptive learning algorithm can be used to aggregate one or more data streams. The adaptive learning algorithm can be used to generate and / or determine metadata from the data aggregation. The adaptive learning can be used to determine one or more improvements from a previous machine learning analysis. Improvements in the collection and / or processing of data feeds can be used, for example, to determine an allowable operating range for controlling one or more surgical devices.

[0041] The device can receive surgical data associated with the surgical procedure. The surgical data can be or include information associated with the surgical procedure. For example, the surgical data can be or include at least one of patient information, healthcare professional (HCP) information, or information associated with a surgical device used in the surgical procedure.

[0042] The device can identify a surgical device to be used in the surgical procedure and / or one or more surgical steps associated with the surgical procedure based on the surgical data. The device can determine an allowable operating range associated with controlling the surgical device based on the identified surgical device, the identified surgical step, and / or the received surgical data. For example, the device can provide the identified surgical device, the one or more surgical steps, and / or the surgical data to an ML model to train the model. The device can use data from the ML training model to determine the allowable operating range. The allowable operating range may be a range of control inputs for controlling the surgical device for the surgical step. For example, the allowable operating range can have an upper operating range and a lower operating range within which the surgical device can safely operate (e.g., recommended) based on the ML training model using, for example, an ML process and / or algorithm.

[0043] The device may receive an adjustment input configuration. The adjustment input configuration may provide an input for controlling the surgical device. The device may determine whether the adjustment input configuration is within an acceptable operating range or outside of an acceptable operating range. Based on a determination that the adjustment input configuration is outside the determined acceptable operating range, the device may interfere with the adjustment input configuration to control the surgical device (e.g., to prevent inadvertent input and / or to avoid catastrophic consequences). The device may send an alert (e.g., an alert message) to the HCP informing them that the adjustment input configuration has been received and is outside the determined acceptable operating range.

[0044] This Summary is provided to introduce a selection of concepts in a concise form that are further described herein in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Other features are described herein. [Brief explanation of the drawings]

[0045] [Figure 1] FIG. 1 is a block diagram of a computer-implemented surgical system. [Figure 2] 1 illustrates an exemplary surgical system in an operating room. [Figure 3] 1 illustrates exemplary surgical hubs paired with various systems. [Figure 4] Illustrates a surgical data network having a set of communicating surgical hubs configured to connect with a set of sensing systems, an environmental sensing system, a set of devices, and the like. [Figure 5] 1 illustrates a logic diagram of a control system for a surgical tool. [Figure 6] 1 illustrates an exemplary surgical system including a handle having a controller and a motor, an adapter releasably coupled to the handle, and a loading unit releasably coupled to the adapter. [Figure 7A] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7B] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7C] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7D] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 8A]1A and 1B show an exemplary supervised learning framework and an exemplary unsupervised learning framework, respectively. [Figure 8B] 1A and 1B show an exemplary supervised learning framework and an exemplary unsupervised learning framework, respectively. [Figure 9] 1 illustrates a flow diagram of a device, such as a computing device, that determines an allowable operating range for controlling a surgical device. [Figure 10] 1 illustrates a flow diagram of a device, such as a surgical device, that determines an allowable operating range for controlling the surgical device. [Figure 11] 1 illustrates a computing device that determines an allowable operating range associated with a surgical device. [Figure 12] 1 illustrates a computing device that adjusts an allowable operating range associated with a surgical device based on an adjustment input configuration from a medical professional. [Figure 13] Illustrates a computing device that receives an adjustment input configuration that is outside of an acceptable operating range, the adjustment input configuration being derived from machine learning (ML) training data and / or an ML algorithm. DETAILED DESCRIPTION OF THE INVENTION

[0046] 1 is a block diagram of a computer-implemented surgical system 100. An exemplary surgical system, such as surgical system 100, may include one or more surgical systems (e.g., surgical subsystems) 102, 103, 104. For example, surgical system 102 may include a computer-implemented bidirectional surgical system. For example, surgical systems 102, 103, 104 may include a surgical computing system, such as a surgical hub 106 and / or a computing device 116, that communicates with a cloud computing system 108. Cloud computing system 108 may include a cloud server 109 and a cloud storage unit 110.

[0047] The surgical systems 102, 103, 104 may each be computer-enabled surgical instruments and devices. For example, the surgical systems 102, 103, 104 may include a wearable sensing system 111, a human interface system 112, a robotic system 113, one or more intelligent instruments 114, an environmental sensing system 115, and / or others. The wearable sensing system 111 may include one or more devices used to sense aspects of an individual's condition and activity within the surgical environment. For example, the wearable sensing system 111 may include a healthcare provider sensing system and / or a patient sensing system.

[0048] The human interface system 112 may include devices that allow individuals to interact with the surgical systems 102, 103, 104 and / or the cloud computing system 108. The human interface system 112 may include human interface devices.

[0049] The robotic system 113 may include a surgical robotic device, such as a surgical robot. The robotic system 113 may enable robotic surgical procedures. The robotic system 113 may receive information, settings, programming, control, etc. from the surgical hub 106; for example, the robotic system 113 may transmit data, such as sensor data, feedback information, video information, and operation logs, to the surgical hub 106.

[0050] Environmental sensing system 115 may include, for example, one or more devices used to measure one or more environmental attributes, for example, as further described in Figure 2. Robotic system 113 may include multiple devices used to perform a surgical procedure, for example, as further described in Figure 2.

[0051] The surgical system 102 may communicate with a remote server 109, which may be part of a cloud computing system 108. In one example, the surgical system 102 may communicate with the remote server 109 via a networked connection, such as an Internet connection (e.g., business internet service, T3, cable / FIOS networking node, etc.). The surgical system 102 and / or its components may communicate with the remote server 109 via a cellular transmission / reception point (TRP) or base station using one or more of the following cellular protocols: GSM / GPRS / EDGE (2G), UMTS / HSPA (3G), long term evolution (LTE) or 4G, LTE-Advanced (LTE-A), new radio (NR), or 5G.

[0052] In one example, the surgical hub 106 may facilitate displaying images from a surgical imaging device, such as a laparoscope. The surgical hub 106 may have collaborative interaction with other local systems to facilitate displaying information related to those local systems. The surgical hub 106 may interact with one or more sensing systems 111, 115, one or more intelligent instruments 114, and / or multiple displays. For example, the surgical hub 106 may be configured to collect measurement data from one or more sensing systems 111, 115 and send notification or control messages to one or more sensing systems 111, 115. The surgical hub 106 may send and / or receive information, including notification information, to and / or from a human interface system 112. The human interface system 112 may include one or more human interface devices (HIDs). The surgical hub 106 may send and / or receive notification or control information for audio, display, and / or control information to various devices in communication with the surgical hub.

[0053] For example, the sensing systems 111, 115 may include a wearable sensing system 111 (which may include one or more HCP sensing systems and one or more patient sensing systems) and an environmental sensing system 115. The one or more sensing systems 111, 115 may measure data related to various biomarkers. The one or more sensing systems 111, 115 may measure the biomarkers using one or more sensors, such as optical sensors (e.g., photodiodes, photoresistors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, thermoelectric sensors, infrared sensors, etc. The one or more sensors may measure the biomarkers as described herein using one of many of the following sensing technologies: photoplethysmography, electrocardiography, electroencephalography, colorimetric, impedimentary, potentiometric, amperometric, etc.

[0054] Biomarkers measured by one or more sensing systems 111, 115 may include, but are not limited to, sleep, core body temperature, maximal oxygen consumption, physical activity, alcohol intake, respiratory rate, oxygen saturation, blood pressure, blood glucose, heart rate variability, blood potential of hydrogen, hydration status, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, coughing and sneezing, gastrointestinal motility, gastrointestinal imaging, airway bacteria, edema, mental status, sweat, circulating tumor cells, autonomic tone, circadian rhythm, and / or menstrual cycle.

[0055] The biomarkers may relate to physiological systems, which may include, but are not limited to, behavioral and psychological, cardiovascular, renal, dermatological, nervous, gastrointestinal, respiratory, endocrine, immune, oncological, musculoskeletal, and / or reproductive systems. Information from the biomarkers may be determined and / or used, for example, by the computer-implemented patient and surgical system 100. Information from the biomarkers may be determined and / or used, for example, by the computer-implemented patient and surgical system 100 to improve the system and / or improve patient outcomes. One or more sensing systems 111, 115, biomarkers, and physiological systems are described in more detail in U.S. patent application Ser. No. 17 / 156,287 (Attorney Docket No. END9290USNP1), entitled "METHOD OF ADJUSTING A SURGICAL PARAMETER BASED ON BIOMARKER MEASUREMENTS," filed on January 22, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0056] FIG. 2 shows an example of a surgical system 202 in an operating room. As illustrated in FIG. 2, a patient is being operated on by one or more health care professionals (HCPs). The HCPs are monitored by one or more HCP sensing systems 220 worn by the HCPs. The HCPs and the environment surrounding the HCPs may also be monitored by one or more environmental sensing systems including, for example, a set of cameras 221, a set of microphones 222, and other sensors that may be deployed in the operating room. The HCP sensing systems 220 and the environmental sensing systems may communicate with a surgical hub 206, which may then communicate with one or more cloud servers 209 of a cloud computing system 208, as shown in FIG. 1. The environmental sensing systems may be used to measure one or more environmental attributes, such as HCP position within the operating room, HCP movement, ambient noise within the operating room, temperature / humidity within the operating room, etc.

[0057] As illustrated in FIG. 2 , a primary display 223 and one or more audio output devices (e.g., speaker 219) are positioned in the sterile field for visibility to the operator at the operating table 224. Additionally, a visualization / notification tower 226 is positioned outside the sterile field. The visualization / notification tower 226 may include a first non-sterile human interactive device (HID) 227 and a second non-sterile HID 229 facing opposite each other. The HIDs may be displays or displays with touchscreens that allow a human to directly interface with the HIDs. A human interface system guided by the surgical hub 206 may be configured to utilize the HIDs 227, 229, and 223 to coordinate information flow to operators inside and outside the sterile field. In one example, the surgical hub 206 may cause an HID (e.g., primary HID 223) to display notifications and / or information regarding the patient and / or surgical procedure steps. In one example, the surgical hub 206 may prompt and / or receive input from a person in the sterile field or non-sterile area. In one example, the surgical hub 206 may cause the HID to display a snapshot of the surgical site as recorded by the imaging device 230 on the non-sterile HID 227 or 229 while maintaining a live view of the surgical site on the main HID 223. The snapshot on the non-sterile display 227 or 229 may, for example, allow a non-sterile operator to perform diagnostic steps related to the surgical procedure.

[0058] In one aspect, the surgical hub 206 may be configured to send diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 to a primary display 223 in the sterile field for viewing by a sterile operator at the operating table. In one example, the input may be in the form of a correction to a snapshot displayed on the non-sterile display 227 or 229, which may be sent by the surgical hub 206 to the primary display 223.

[0059] 2, a surgical instrument 231 is used as part of a surgical system 202 in a surgical procedure. A hub 206 may be configured to coordinate information flow to the display of the surgical instrument 231, as described, for example, in U.S. Patent Application Publication No. 2019-0200844(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 can be sent by the hub 206 to a surgical instrument display in the sterile field, where it can be viewed by the operator of the surgical instrument 231. Exemplary surgical instruments suitable for use with surgical system 202 are described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety.

[0060] FIG. 2 illustrates an example of a surgical system 202 being used to perform a surgical procedure on a patient lying on an operating table 224 in an operating room 235. A robotic system 234 may be used as part of the surgical system 202 in the surgical procedure. The robotic system 234 may include a surgeon's console 236, a patient side cart 232 (surgical robot), and a surgical robot hub 233. The patient side cart 232 can manipulate at least one detachably coupled surgical tool 237 through a minimally invasive incision within the patient's body while the surgeon views the surgical site through the surgeon's console 236. Images of the surgical site can be acquired by a medical imaging device 230, which can be manipulated by the patient side cart 232 to orient the imaging device 230. The robotic hub 233 can be used to process the images of the surgical site and then display them to the surgeon through the surgeon's console 236.

[0061] Other types of robotic systems can be readily adapted for use with surgical system 202. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0201137(A1), entitled "METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety.

[0062] Various examples of cloud-based analytics methods that may be performed by cloud computing system 208 and that are suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0206569(A1), entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,403), the disclosure of which is incorporated herein by reference in its entirety.

[0063] In various embodiments, the imaging device 230 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, charge-coupled device (CCD) sensors and complementary metal-oxide semiconductor (CMOS) sensors.

[0064] The optics of the imaging device 230 may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate a portion of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0065] The one or more illumination sources can be configured to emit electromagnetic energy in the visible spectrum as well as the invisible spectrum. The visible spectrum, sometimes referred to as the optical spectrum or luminous spectrum, is the portion of the electromagnetic spectrum that is visible to (i.e., detectable by) the human eye and is sometimes referred to as visible light or simply light. The human eye typically responds to wavelengths in air between about 380 nm and about 750 nm.

[0066] The invisible spectrum (e.g., non-radiative spectrum) is the portion of the electromagnetic spectrum located below and above the visible spectrum (i.e., wavelengths less than about 380 nm and greater than about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, which constitutes invisible infrared (IR), microwave, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, which constitutes invisible ultraviolet, x-ray, and gamma-ray electromagnetic radiation.

[0067] In various aspects, imaging device 230 is configured for use in minimally invasive procedures. Examples of imaging devices suitable for use with the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, cholangioscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophagogastroduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngo-neproscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0068] The imaging device may use multispectral monitoring to distinguish between topography and underlying structures. Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. Wavelengths can be separated by filters or by using instruments sensitive to specific wavelengths, including frequencies beyond the visible light range, e.g., IR and UV light. Spectral imaging allows for the extraction of additional information that cannot be captured by the red, green, and blue receptors of the human eye. The use of multispectral imaging is described in detail under the heading "Advanced Imaging Acquisition Module" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Multispectral monitoring can be a useful tool for repositioning the surgical field after the surgical task is complete to perform one or more of the tests described above on the treated tissue. It is self-evident that strict sterilization of the operating room and surgical equipment is necessary during any surgical procedure. The strict hygiene and sterilization conditions required in an "operating room," i.e., an operating room or procedure room, require the highest possible sterility of all medical devices and equipment. Part of the sterilization process is the need to sterilize everything that comes into contact with the patient or enters the sterile field, including the imaging device 230 and its accessories and components. It will be understood that the sterile field may be considered a specific area, such as in a tray or on a sterile towel, that is deemed free of microorganisms, or the sterile field may be considered the area immediately surrounding the patient being prepared for the surgical procedure. The sterile field may include properly clothed and hand-washed team members, as well as all equipment and fixtures in the area.

[0069] The wearable sensing system 211 illustrated in FIG. 1 may include one or more sensing systems, such as an HCP sensing system 220 as shown in FIG. 2. The HCP sensing system 220 may include a sensing system that monitors and detects a set of physical conditions and / or a set of physiological conditions of a healthcare professional (HCP). An HCP may generally be one or more healthcare professionals assisting a surgeon or other healthcare provider. In one example, the sensing system 220 may measure a set of biomarkers to monitor the HCP's heart rate. In one example, the sensing system 220 worn on the surgeon's wrist (e.g., a watch or wristband) may use an accelerometer to detect hand movement and / or shaking and determine tremor magnitude and frequency. The sensing system 220 may transmit measurement data associated with the set of biomarkers and data associated with the surgeon's physical condition to the surgical hub 206 for further processing. One or more environmental sensing devices may transmit environmental information to the surgical hub 206. For example, the environmental sensing device may include a camera 221 for detecting the HCP's hand / body position. The environmental sensing devices may include a microphone 222 for measuring ambient noise within the operating room. Other environmental sensing devices may include devices such as a thermometer to measure temperature and a hygrometer to measure ambient humidity within the operating room. The surgical hub 206, alone or in communication with a cloud computing system, may use the surgeon biomarker measurement data and / or environmental sensing information to modify the control algorithms of handheld instruments or the average latency of a robotic interface, for example, to minimize tremor. In one example, the HCP sensing system 220 may measure one or more surgeon biomarkers associated with the HCP and transmit measurement data associated with the surgeon biomarkers to the surgical hub 206. The HCP sensing system 220 may use one or more RF protocols, such as Bluetooth, Bluetooth Low-Energy (BLE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low-Power Wireless Personal Area Network (6LoWPAN), or Wi-Fi, to communicate with the surgical hub 206.Surgeon biomarkers may include one or more of stress, heart rate, etc. Environmental measurements from the operating room may include ambient noise levels associated with the surgeon or patient, surgeon and / or staff movement, surgeon and / or staff attention level, etc.

[0070] The surgical hub 206 may use the surgeon biomarker measurement data associated with the HCP to adaptively control one or more surgical instruments 231. For example, the surgical hub 206 may send a control program to the surgical instrument 231 to control its actuators to limit or compensate for fatigue and the use of fine motor skills. The surgical hub 206 may send the control program based on situational awareness and / or context regarding the importance or criticality of the task. The control program may instruct the instrument to change operation to provide more control when control is needed.

[0071] FIG. 3 illustrates an exemplary surgical system 302 having a surgical hub 306. The surgical hub 306 may be paired with a wearable sensing system 311, an environmental sensing system 315, a human interface system 312, a robotic system 313, and an intelligent instrument 314 via a modular control. The hub 306 includes a display 348, an imaging module 349, a generator module 350, a communications module 356, a processor module 357, a storage array 358, and an operating room mapping module 359. In certain embodiments, as illustrated in FIG. 3, the hub 306 further includes a smoke evacuation module 354 and / or a suction / irrigation module 355. The various modules and systems may be connected to the modular control either directly via a router or via the communications module 356. Operating room devices may be coupled to cloud computing resources and data storage via the modular control. The human interface system 312 may include a display subsystem and a notification subsystem.

[0072] The modular controller may be coupled to a non-contact sensor module. The non-contact sensor module may use ultrasonic, laser-type, and / or similar non-contact measurement devices to measure the dimensions of the operating room and generate a map of the operating room. Other distance sensors may be used to determine the boundaries of the operating room. As described in U.S. Provisional Patent Application No. 62 / 611,341, filed December 28, 2017, entitled "INTERACTIVE SURGICAL PLATFORM," which is incorporated herein by reference in its entirety, under the heading "Surgical Hub Spatial Awareness Within an Operating Room," an ultrasound-based non-contact sensor module may scan the operating room by transmitting bursts of ultrasound and receiving echoes as the bursts reflect off the exterior walls of the operating room. The sensor module may be configured to determine the size of the operating room and adjust Bluetooth pairing distance limits. For example, a laser-based non-contact sensor module can scan an operating room by transmitting laser light pulses, receiving the laser light pulses that reflect off the exterior walls of the operating room, and comparing the phase of the transmitted pulses with the received pulses to determine the size of the operating room and adjust the Bluetooth pairing distance limit.

[0073] During surgical procedures, the application of energy to tissue for sealing and / or cutting is generally associated with smoke evacuation, aspiration of excess fluid, and / or irrigation of tissue. Fluid, power, and / or data lines from different sources often become tangled during surgical procedures. Addressing this issue can result in valuable time being lost during a surgical procedure. Untangling the lines may require unplugging them from their corresponding modules, which may require resetting the modules. The hub modular enclosure 360 ​​reduces the frequency of such line tangles by providing a unified environment for managing power, data, and fluid lines. Aspects of the present disclosure present a surgical hub 306 for use in surgical procedures involving the application of energy to tissue at a surgical site.

[0074] The surgical hub 306 includes a hub enclosure 360 ​​and a combination generator module slidably received within a docking station of the hub enclosure 360. The docking station includes data and power contacts. The combination generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component housed within a single unit. In one aspect, the combination generator module also includes a smoke evacuation component, at least one energy delivery cable for connecting the combination generator module to a surgical instrument, at least one smoke evacuation component configured to evacuate smoke, fluid, and / or particulates generated by the application of therapeutic energy to tissue, and a fluid line extending from the remote surgical site to the smoke evacuation component. In one aspect, the fluid line may be a first fluid line and a second fluid line may extend from the remote surgical site to the suction and irrigation module 355, which is slidably received within the hub enclosure 360. In one aspect, the hub enclosure 360 ​​may include a fluid interface.

[0075] Certain surgical procedures may require the application of two or more energy types to tissue. One energy type may be more beneficial for cutting tissue, while another, different energy type may be more beneficial for sealing tissue. For example, a bipolar generator may be used to seal tissue, while an ultrasonic generator may be used to cut the sealed tissue. Aspects of the present disclosure present a solution in which a hub modular enclosure 360 ​​is configured to house different generators and facilitate bidirectional communication between them. The hub modular enclosure 360 ​​may allow for rapid removal and / or replacement of various modules. Aspects of the present disclosure present a modular surgical enclosure for use in surgical procedures involving the application of energy to tissue. The modular surgical enclosure includes a first energy generator module configured to generate a first energy for application to tissue and a first docking station including a first docking port including first data contacts and first power contacts, wherein the first energy generator module is slidably movable into electrical engagement with the power contacts and the data contacts, and the first energy generator module is slidably movable out of electrical engagement with the first power contacts and the first data contacts. In addition to the above, the modular surgical enclosure also includes a second energy generator module configured to generate a second energy different from the first energy for application to tissue and a second docking station including a second docking port including second data contacts and second power contacts, wherein the second energy generator module is slidably movable into electrical engagement with the power contacts and the data contacts, and the second energy generator module is slidably movable out of electrical engagement with the second power contacts and the second data contacts. In addition, the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port configured to facilitate communication between the first energy generator module and the second energy generator module.Referring to FIG. 3 , an aspect of the disclosure is presented regarding a hub modular enclosure 360 ​​that allows for modular integration of a generator module 350, a smoke evacuation module 354, and a suction / irrigation module 355. The hub modular enclosure 360 ​​further facilitates bidirectional communication between the modules 359, 354, and 355. The generator module 350 can comprise integrated monopolar, bipolar, and ultrasonic components supported within a single housing unit that is slidably insertable into the hub modular enclosure 360. The generator module 350 can be configured to connect to a monopolar device 351, a bipolar device 352, and an ultrasonic device 353. Alternatively, the generator module 350 may comprise a series of monopolar, bipolar, and / or ultrasonic generator modules that interact through the hub modular enclosure 360. The hub modular enclosure 360 ​​may be configured to facilitate the insertion of multiple generators and bidirectional communication between generators docked to the hub modular enclosure 360 ​​so that the multiple generators act as a single generator.

[0076] FIG. 4 illustrates a surgical data network having a set of communication hubs configured to connect a set of sensing systems, environmental sensing systems, and other modular devices located in one or more operating rooms, patient recovery rooms, or rooms within a medical facility specially equipped for surgery to a cloud in accordance with at least one embodiment of the present disclosure.

[0077] 4, surgical hub system 460 may include a modular communications hub 465 configured to connect modular devices located within a medical facility to a cloud-based system (e.g., a cloud computing system 464, which may include a remote server 467 coupled to remote storage 468). The modular communications hub 465 and devices may be connected in a room within the medical facility specially equipped for surgical procedures. In one aspect, modular communications hub 465 may include a network hub 461 and / or a network switch 462 in communication with a network router 466. Modular communications hub 465 may be coupled to a local computer system 463 to provide local computer processing and data manipulation.

[0078] The computer system 463 may include a processor and a network interface. The processor may be coupled to a communication module, storage, memory, non-volatile memory, and input / output (I / O) interfaces via a system bus. The system bus can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures, including, but not limited to, a 9-bit bus, an Industrial Standard Architecture (ISA), a Micro-Channel Architecture (MSA), an Extended ISA (EISA), an Intelligent Drive Electronics (IDE), a VESA Local Bus (VLB), a Peripheral Component Interconnect (PCI), a USB, an Advanced Graphics Port (AGP), a Personal Computer Memory Card International Association (PCMCIA), a Small Computer Systems Interface (SCSI), or any other proprietary bus.

[0079] The processor may be any single-core or multi-core processor, such as that known by Texas Instruments under the trade name ARM Cortex. In one aspect, the processor may be, for example, an LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments, including on-chip memory of 256 KB of single-cycle flash memory or other non-volatile memory up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, 32 KB of single-cycle serial random access memory (SRAM), internal read-only memory (ROM) loaded with StellarisWare® software, 2 KB of electrically erasable programmable read-only memory (EEPROM), and / or one or more pulse width modulation (PWM) modules, one or more Quadrature Encoder Input (QEI) analog, one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, details of which are available in the product data sheet.

[0080] In one example, the processor may include a safety controller, which includes two controller-based families such as the TMS570 and RM4x, also known under the trade name Hercules ARM Cortex R4, manufactured by Texas Instruments. The safety controller may be specifically configured for IEC 61508 and ISO 26262 safety limit applications, among others, to provide advanced integrated safety mechanisms while offering scalable performance, connectivity, and memory options.

[0081] It should be understood that the computer system 463, in a suitable operating environment, may include software that acts as an intermediary between the described users and the basic computer resources. Such software may include an operating system. The operating system, which may be stored on disk storage, may function to control and allocate resources of the computer system. System applications may take advantage of resource management by the operating system through program modules and program data stored either in system memory or on disk storage. It should be understood that the various components described herein may be implemented with various operating systems or combinations of operating systems.

[0082] A user may input commands or information into the computer system 463 through input devices coupled to the I / O interface. Input devices may include, but are not limited to, pointing devices such as a mouse, trackball, stylus, or touchpad; keyboards; microphones; joysticks; gamepads; satellite dishes; scanners; TV tuner cards; digital cameras; digital video cameras; webcams; and the like. These and other input devices connect to the processor through the system bus via interface ports. Interface ports include, for example, serial ports, parallel ports, game ports, and USB. Output devices use some of the same types of ports as input devices. Thus, for example, a USB port may be used to provide input to the computer system 463 and to output information from the computer system 463 to an output device. Output adapters may be provided to illustrate that there may be several output devices, such as monitors, displays, speakers, and printers, among other output devices that may require special adapters. Output adapters may include, by way of example and not limitation, video and sound cards that provide a means of connection between the output device and the system bus. It should be noted that other devices and / or systems of devices, such as remote computers, may provide both input and output capabilities.

[0083] The computer system 463 may operate in a networked environment using logical connections to one or more remote computers, such as a cloud computer, or a local computer. The remote cloud computer may be a personal computer, a server, a router, a network PC, a workstation, a microprocessor-based appliance, a peer device, or other common network node, but typically includes many or all of the elements described with respect to a computer system. For simplicity, only a memory storage device is illustrated with the remote computer. The remote computer may be logically connected to the computer system through a network interface and subsequently physically connected via a communication connection. The network interface may encompass communication networks such as a local area network (LAN) and a wide area network (WAN). LAN technologies may include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet / IEEE 802.3, Token Ring / IEEE 802.5, etc. WAN technologies may include, but are not limited to, point-to-point links, circuit-switched networks such as Integrated Services Digital Networks (ISDN) and its variants, packet-switched networks, and Digital Subscriber Lines (DSL).

[0084] In various examples, computer system 463 may include an image processor, image processing engine, media processor, or any specialized digital signal processor (DSP) used to process digital images. The image processor may use parallel computing using Single Instruction, Multiple Data (SIMD) or Multiple Instruction, Multiple Data (MIMD) techniques to increase speed and efficiency. The digital image processing engine may perform a variety of tasks. The image processor may be a system on a chip with a multi-core processor architecture.

[0085] The communications connection may refer to the hardware / software used to connect the network interface to the bus. The communications connection is shown internal to computer system 463 for clarity of illustration, but can also be external to computer system 463. For purposes of illustration only, the hardware / software required to connect to the network interface may include internal and external technologies such as modems, including regular telephone-grade modems, cable modems, fiber optic modems, and DSL modems, ISDN adapters, and Ethernet cards. In some examples, the network interface may also be provided using an RF interface.

[0086] The surgical data network associated with the surgical hub system 460 may be configured as passive, intelligent, or switched. A passive surgical data network acts as a conduit for data, allowing it to go from one device (or segment) to another and to cloud computing resources. An intelligent surgical data network includes additional features that allow traffic to pass through the monitored surgical data network and configure each port within the network hub 461 or network switch 462. An intelligent surgical data network may be referred to as a manageable hub or switch. A switching hub reads the destination address of each packet and then forwards the packet to the correct port.

[0087] Modular devices 1a-1n located in an operating room may be coupled to a modular communication hub 465. Network hub 461 and / or network switch 462 may be coupled to a network router 466 to connect devices 1a-1n to a cloud computing system 464 or a local computer system 463. Data associated with devices 1a-1n may be transferred to a cloud-based computer via the router for remote data processing and manipulation. Data associated with devices 1a-1n may also be transferred to local computer system 463 for local data processing and manipulation. Modular devices 2a-2m located in the same operating room may also be coupled to network switch 462. Network switch 462 may be coupled to network hub 461 and / or to network router 466 to connect devices 2a-2m to a cloud 464. Data associated with devices 2a-2m may be transferred to cloud computing system 464 via network router 466 for data processing and manipulation. Data associated with the devices 2a-2m may also be transferred to a local computer system 463 for local data processing and manipulation.

[0088] 4, a computing system such as surgical hub system 460 may include a modular communications hub 465 configured to connect modular devices (e.g., surgical devices) located within a medical facility to a cloud-based system (e.g., a cloud computing system 464 that may include a remote server 467 coupled to remote storage 468). The modular communications hub 465 and devices may be connected in a room within the medical facility specially equipped for surgical procedures. In one aspect, the modular communications hub 465 may include a network hub 461 and / or a network switch 462 in communication with a network router 466. The modular communications hub 465 may be coupled to a local computer system (e.g., a computing device) to provide local computer processing and data manipulation.

[0089] FIG. 5 illustrates a logic diagram of a surgical instrument or surgical tool control system 520 according to one or more embodiments of the present disclosure. The surgical instrument or surgical tool may be configurable. The surgical instrument may include surgical fasteners specific to the procedure at hand, such as imaging devices, surgical staplers, energy devices, endocutter devices, etc. For example, the surgical instrument may include any of a power stapler, a power stapler generator, an energy device, an advanced energy device, an advanced energy jaw device, an endocutter clamp, an energy device generator, an operating room imaging system, a smoke evacuation device, a suction irrigation device, an insufflation system, etc. The system 520 may include control circuitry. The control circuitry may include a microcontroller 521 with a processor 522 and a memory 523. For example, one or more of sensors 525, 526, 527 provide real-time feedback to the processor 522. A motor 530 driven by a motor driver 529 operably couples a longitudinally movable displacement member to drive the I-beam knife element. The tracking system 528 may be configured to determine the position of the longitudinally movable displacement member. The position information may be provided to the processor 522, which may be programmed or configured to determine the position of the longitudinally movable drive member, as well as the positions of the firing member, firing bar, and I-beam knife element. Additional motors may be provided to the tool driver interface to control the firing of the I-beam, the movement of the obturator tube, the rotation of the shaft, and the articulation. The display 524 may display various operating states of the instrument and may also include touch screen functionality for data entry. Information displayed on the display 524 may be overlaid with images acquired via the endoscopic imaging module.

[0090] Microcontroller 521 may be any single-core or multi-core processor, such as those known under the trade name ARM Cortex manufactured by Texas Instruments. In one aspect, main microcontroller 521 may be, for example, an LM4F230H5QR ARM Cortex-M4F Processor Core available from Texas Instruments, with on-chip memory of 256 KB of single-cycle flash memory or other non-volatile memory up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, 32 KB of single-cycle SRAM, internal ROM with StellarisWare® software, 2 KB of EEPROM, one or more PWM modules, one or more QEI analog, and / or one or more 12-bit ADCs with 12 analog input channels, details of which are available in the product datasheet.

[0091] The microcontroller 521 may include a safety controller, which includes two controller-based families such as the TMS570 and RM4x, also known under the trade name Hercules ARM Cortex R4, manufactured by Texas Instruments. The safety controller can be specifically configured for IEC 61508 and ISO 26262 safety limit applications, among others, to provide advanced integrated safety mechanisms while offering scalable performance, connectivity, and memory options.

[0092] The microcontroller 521 may be programmed to perform various functions, such as precise control over the speed and position of the knife and articulation system. In one embodiment, the microcontroller 521 may include a processor 522 and memory 523. The electric motor 530 may be a brushed direct current (DC) motor with a gearbox and mechanical linkage to the articulation or knife system. In one embodiment, the motor driver 529 may be an A3941 available from Allegro Microsystems, Inc. Other motor drivers may be easily substituted for use in the tracking system 528 with an absolute positioning system. A detailed description of the absolute positioning system is provided in U.S. Patent Application Publication No. 2017 / 0296213, entitled "SYSTEMS AND METHODS FOR CONTROLLING A SURGICAL STAPLING AND CUTTING INSTRUMENT," published October 19, 2017, which is incorporated herein by reference in its entirety.

[0093] The microcontroller 521 may be programmed to provide precise control over the velocity and position of the displacement members and articulation system. The microcontroller 521 may be configured to calculate a response in the microcontroller 521 software. The calculated response may be compared to the measured response of the actual system to obtain an "observed" response, which is used for actual feedback decision-making. The observed response may be a suitably adjusted value that balances the smooth, continuous nature of the simulated response with the measured response, which can detect external influences on the system.

[0094] The motor 530 may be controlled by a motor driver 529 and may be used by the surgical instrument or tool firing system. In various forms, the motor 530 may be a brushed DC drive motor having a maximum rotational speed of approximately 25,000 RPM. In some examples, the motor 530 may include a brushless motor, a cordless motor, a synchronous motor, a stepper motor, or any other suitable electric motor. The motor driver 529 may include, for example, an H-bridge driver including field-effect transistors (FETs). The motor 530 may be powered by a power supply assembly releasably attached to the handle assembly or tool housing to provide control power to the surgical instrument or tool. The power supply assembly may include a battery, which may include multiple battery cells connected in series, that may be used as a power source to power the surgical instrument or tool. Under certain circumstances, the battery cells of the power supply assembly may be replaceable and / or rechargeable. In at least one example, the battery cells may be lithium-ion batteries, which may be connectable to and detachable from the power supply assembly.

[0095] The motor driver 529 may be the A3941, available from Allegro Microsystems, Inc. The A3941 may be a full-bridge controller for use with external N-channel power metal-oxide semiconductor field-effect transistors (MOSFETs) specifically designed for inductive loads, such as brushed DC motors. The driver 529 may include an intrinsic charge pump regulator, which provides full (>10V) gate drive for battery voltages up to 7V, allowing the A3941 to operate with reduced gate drive down to 5.5V. A bootstrap capacitor may be used to provide the required battery supply voltage above the N-channel MOSFET. An internal charge pump for the high-side drive allows DC (100% duty cycle) operation. The full-bridge may be driven in fast or slow decay mode using diode or synchronous rectification. In slow decay mode, current recirculation is possible through either the high-side or low-side FET. The power FETs may be protected from shoot-through by a resistor-adjustable dead time. Integrated diagnostics indicate undervoltage, overtemperature, and power bridge faults and can be configured to protect the power MOSFETs under most short circuit conditions. Other motor drivers may be easily substituted for use in tracking system 528 with an absolute positioning system.

[0096] The tracking system 528 may include controlled motor drive circuitry including a position sensor 525 according to one aspect of the present disclosure. The position sensor 525 for the absolute positioning system may provide a unique position signal corresponding to the position of the displacement member. In some examples, the displacement member may represent a longitudinally movable drive member including a rack of drive teeth for meshing engagement with a corresponding drive gear of a gear reducer assembly. In some examples, the displacement member may represent a firing member that may be adapted and configured to include a rack of drive teeth. In some examples, the displacement member may represent a firing bar or an I-beam, each of which may be adapted and configured to include a rack of drive teeth. Thus, as used herein, the term displacement member may be used generally to refer to any movable member of a surgical instrument or tool, such as a drive member, firing member, firing bar, I-beam, or any element that can be displaced. In one aspect, a longitudinally movable drive member may be coupled to a firing member, firing bar, and I-beam. Thus, the absolute positioning system may actually track the linear displacement of the I-beam by tracking the linear displacement of the longitudinally movable drive member. In various aspects, the displacement member may be coupled to any suitable position sensor 525 for measuring linear displacement. Thus, the longitudinally movable drive member, firing member, firing bar, or I-beam, or combinations thereof, may be coupled to any suitable linear displacement sensor. The linear displacement sensor may include a contact displacement sensor or a non-contact displacement sensor.The linear displacement sensor may include a linear variable differential transformer (LVDT), a differential variable reluctance transducer (DVRT), a slide potentiometer, a magnetic sensing system with a movable magnet and a series of linearly arranged Hall effect sensors, a magnetic sensing system with a fixed magnet and a series of movable linearly arranged Hall effect sensors, an optical sensing system with a movable light source and a series of linearly arranged photodiodes or photodetectors, an optical sensing system with a fixed light source and a series of movable linearly arranged photodiodes or photodetectors, or any combination thereof.

[0097] The electric motor 530 may include a rotatable shaft operably interfaced with a gear assembly mounted for meshing engagement with a set of drive teeth, or rack, on the displacement member. The sensor element may be operably coupled to the gear assembly such that one rotation of the position sensor 525 element corresponds to several linear longitudinal translations of the displacement member. The gearing and sensor arrangement may be connected to a linear actuator by a rack-and-pinion arrangement or to a rotary actuator by a spur gear or other connection. A power source may provide power to the absolute positioning system, and an output indicator may display the output of the absolute positioning system. The displacement member may represent a longitudinally movable drive member with a rack of drive teeth formed thereon for meshing engagement with a corresponding drive gear of a gear reducer assembly. The displacement member may represent a longitudinally movable firing member, a firing bar, an I-beam, or a combination thereof.

[0098] One revolution of the sensor element associated with the position sensor 525 may correspond to a linear longitudinal displacement d1 of the displacement member, where d1 is the linear longitudinal distance the displacement member travels from point "a" to point "b" after one revolution of the sensor element coupled to the displacement member. The sensor arrangement may be connected via a gear reduction such that the position sensor 525 completes one or more revolutions for a full stroke of the displacement member. The position sensor 525 may complete multiple revolutions for a full stroke of the displacement member.

[0099] A series of switches (where n is an integer greater than 1) may be used alone or in combination with a gear reduction to provide a unique position signal for two or more revolutions of the position sensor 525. The state of the switches may be fed back to the microcontroller 521, which applies logic to determine a unique position signal corresponding to the longitudinal linear displacement d1+d2+...dn of the displacement member. The output of the position sensor 525 is provided to the microcontroller 521. The position sensor 525 of the sensor arrangement may comprise a magnetic sensor, an analog rotary sensor such as a potentiometer, or an array of analog Hall effect elements that output a unique combination of position signals or values.

[0100] The position sensor 525 may comprise any number of magnetic sensing elements, such as magnetic sensors classified according to whether they measure the total magnetic field or vector components of the magnetic field. The technologies used to produce both types of magnetic sensors may encompass many aspects of physics and electronics. Technologies used to sense magnetic fields may include search coils, fluxgates, optical pumping, nuclear precession, SQUIDs, Hall effect, anisotropic magnetoresistance, giant magnetoresistance, magnetic tunnel junctions, giant magnetoimpedance, magnetostrictive / piezoelectric composites, magnetodiodes, magnetotransistors, optical fiber, magneto-optical, and microelectromechanical systems-based magnetic sensors, among others.

[0101] The position sensor 525 of the tracking system 528 with an absolute positioning system may comprise a magnetic rotary absolute positioning system. The position sensor 525 may be implemented as an AS5055EQFT single-chip magnetic rotary position sensor available from Austria Microsystems, AG. The position sensor 525 interfaces with the microcontroller 521 to provide the absolute positioning system. The position sensor 525 may be a low-voltage, low-power component and may include four Hall-effect elements in an area of ​​the position sensor 525 that may be located above the magnet. A high-resolution ADC and a smart power management controller may also be provided on-chip. A coordinate rotation digital computer (CORDIC) processor, also known as the digit-by-digit method and the Boulder algorithm, may be provided to implement simple, efficient algorithms for calculating hyperbolic and trigonometric functions, requiring only addition, subtraction, bit shifting, and table lookup operations. The angular position, alarm bits, and magnetic field information may be transmitted to the microcontroller 521 through a standard serial communications interface, such as a serial peripheral interface (SPI) interface. The position sensor 525 may provide 12-bit or 14-bit resolution and may be an AS5055 chip provided in a small QFN 16-pin 4x4x0.85mm package.

[0102] The tracking system 528, which comprises an absolute positioning system, may include and / or be programmed to implement a feedback controller, such as a PID, state feedback, and adaptive controller. The power supply converts the signal from the feedback controller into a physical input to the system, in this case a voltage. Other examples include PWM of voltage, current, and force. In addition to the position measured by the position sensor 525, other sensors may be provided to measure physical parameters of the physical system. In some embodiments, other sensors may include sensor arrangements such as those described in U.S. Pat. No. 9,345,481, issued May 24, 2016, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety; U.S. Patent Application Publication No. 2014 / 0263552, published September 18, 2014, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety; and U.S. Patent Application No. 15 / 628,175, filed June 20, 2017, entitled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," which is incorporated herein by reference in its entirety. In a digital signal processing system, the absolute positioning system is coupled to a digital data acquisition system, where the output of the absolute positioning system has a finite resolution and sampling frequency. The absolute positioning system may include comparison and combination circuitry to combine the calculated response with the measured response using algorithms such as weighted averages and theoretical control loops that drive the calculated response towards the measured response. The calculated response of the physical system may take into account properties such as mass, inertia, viscous friction, induced drag, etc., in order to predict what the state and output of the physical system will be given knowledge of the input.

[0103] The absolute positioning system may provide the absolute position of the displacement member upon powering up of the instrument without forcing the displacement member to retract or advance to a reset (zero or home) position, as may be required with conventional rotary encoders that simply count the number of forward or backward steps taken by the motor 530 to infer the position of the device actuator, drive bar, knife, etc.

[0104] Sensor 526, such as a strain gauge or micro-strain gauge, may be configured to measure one or more parameters of the end effector, such as the amplitude of strain exerted on the anvil during clamping, which can be indicative of the closure force applied to the anvil. The measured strain may be converted to a digital signal and provided to processor 522. Alternatively, or in addition to sensor 526, sensor 527, such as a load sensor, can measure the closure force applied to the anvil by the closure drive system. For example, sensor 527, such as a load sensor, can measure the firing force applied to the I-beam during the firing stroke of the surgical instrument or tool. The I-beam is configured to engage a wedge-shaped sled, which is configured to cam the staple driver upward and drive the staples into deforming contact with the anvil. The I-beam may also include a sharp cutting edge that can be used to cut tissue as the I-beam is advanced distally by the firing bar. Alternatively, a current sensor 531 can be used to measure the current drawn by motor 530. The force required to advance the firing member may correspond, for example, to the current drawn by motor 530. The measured force may be converted to a digital signal and provided to processor 522.

[0105] For example, a strain gauge sensor 526 can be used to measure the force applied to tissue by the end effector. A strain gauge can be coupled to the end effector to measure the force applied by the end effector to the tissue being treated. A system for measuring the force applied to grasped tissue by the end effector may include a strain gauge sensor 526, such as, for example, a micro-strain gauge, which can be configured to measure one or more parameters of the end effector. In one aspect, the strain gauge sensor 526 can measure the amplitude or magnitude of strain exerted on the jaw members of the end effector during clamping, which can indicate tissue compression. The measured strain can be converted to a digital signal and provided to the processor 522 of the microcontroller 521. A load sensor 527 can measure the force used to operate the knife element, for example, to cut tissue captured between the anvil and the staple cartridge. A magnetic field sensor can be used to measure the thickness of the captured tissue. The magnetic field sensor's measurements can also be converted to a digital signal and provided to the processor 522.

[0106] Measurements of tissue compression, tissue thickness, and / or force required to close the end effector on the tissue, measured by sensors 526, 527, respectively, may be used by microcontroller 521 to characterize a selected position of the firing member and / or a corresponding value of firing member velocity. In one example, memory 523 may store techniques, equations, and / or look-up tables that may be used by microcontroller 521 during the evaluation.

[0107] The surgical instrument or tool control system 520 may also include wired or wireless communication circuitry for communicating with a surgical hub, such as surgical hub 460, as shown in FIG.

[0108] FIG. 6 illustrates an exemplary surgical system 680 according to the present disclosure and may include a surgical instrument 682 that can communicate with a console 694 or a portable device 696 through a local area network 692 and / or a cloud network 693 via wired and / or wireless connections. The console 694 and the portable device 696 may be any suitable computing devices. The surgical instrument 682 may include a handle 697, an adapter 685, and a loading unit 687. The adapter 685 releasably couples to the handle 697, and the loading unit 687 releasably couples to the adapter 685 such that the adapter 685 transmits force from the drive shaft to the loading unit 687. The adapter 685 or the loading unit 687 may include a force gauge (not explicitly shown) disposed therein to measure force applied to the loading unit 687. The loading unit 687 may include an end effector 689 having a first jaw 691 and a second jaw 690. The loading unit 687 may be an in vivo loading or multi-firing loading unit (MFLU) that allows a clinician to fire multiple fasteners multiple times without having to remove the loading unit 687 from the surgical site to reload it.

[0109] The first jaw 691 and the second jaw 690 may be configured to clamp tissue therebetween, fire fasteners through the clamped tissue, and sever the clamped tissue. The first jaw 691 may be configured to fire at least one fastener multiple times, or may be configured to contain a replaceable multi-fire fastener cartridge containing multiple fasteners (e.g., staples, clips, etc.) that may be fired two or more times before being replaced. The second jaw 690 may include an anvil that deforms or otherwise secures fasteners as they are ejected from the multi-fire fastener cartridge.

[0110] The handle 697 may include a motor coupled to the drive shaft to affect rotation of the drive shaft. The handle 697 may include a control interface for selectively activating the motor. The control interface may include buttons, switches, levers, sliders, a touch screen, and any other suitable input mechanism or user interface that may be engaged by a clinician to activate the motor.

[0111] The control interface of the handle 697 may be in communication with a controller 698 of the handle 697 to selectively activate the motors to affect rotation of the drive shaft. The controller 698 may be disposed within the handle 697 and configured to receive input from the control interface and adapter data from the adapter 685 or loading unit data from the loading unit 687. The controller 698 may analyze the input from the control interface and the data received from the adapter 685 and / or the loading unit 687 to selectively activate the motors. The handle 697 may also include a display viewable by a clinician while using the handle 697. The display may be configured to display a portion of the adapter or loading unit data before, during, or after firing of the instrument 682.

[0112] The adapter 685 may include an adapter identification device 684 disposed therein, and the loading unit 687 may include a loading unit identification device 688 disposed therein. The adapter identification device 684 may be in communication with a controller 698, and the loading unit identification device 688 may be in communication with the controller 698. It will be appreciated that the loading unit identification device 688 may be in communication with the adapter identification device 684, which relays or passes communications from the loading unit identification device 688 to the controller 698.

[0113] The adapter 685 may also include multiple sensors 686 (one shown) disposed about its periphery to detect various conditions of the adapter 685 or the environment (e.g., whether the adapter 685 is connected to the loading unit, whether the adapter 685 is connected to the handle, whether the drive shaft is rotating, the torque of the drive shaft, the strain on the drive shaft, the temperature within the adapter 685, the number of times the adapter 685 has been fired, the peak force of the adapter 685 during firing, the total amount of force applied to the adapter 685, the peak retract force of the adapter 685, the number of times the adapter 685 has been paused during firing, etc.). The multiple sensors 686 may provide input to the adapter identification device 684 in the form of data signals. The data signals of the multiple sensors 686 may be stored in the adapter identification device 684 or may be used to update adapter data stored in the adapter identification device 684. The data signals of the multiple sensors 686 may be analog or digital. The multiple sensors 686 may include a force gauge that measures the force exerted on the loading unit 687 during firing.

[0114] The handle 697 and adapter 685 may be configured to interconnect the adapter identification device 684 and the loading unit identification device 688 with the controller 698 via an electrical interface. The electrical interface may be a direct electrical interface (i.e., including electrical contacts that engage with each other to transmit energy and signals therebetween). Additionally or alternatively, the electrical interface may be a contactless electrical interface for wirelessly transmitting (e.g., inductively transmitting) energy and signals therebetween. It is also contemplated that the adapter identification device 684 and the controller 698 may communicate wirelessly with each other via a wireless connection that is separate from the electrical interface.

[0115] The handle 697 may include a transceiver 683 configured to transmit instrument data from the controller 698 to other components of the system 680 (e.g., the LAN 20292, the cloud 693, the console 694, or the portable device 696). The controller 698 may also transmit instrument data and / or measurement data associated with one or more sensors 686 to the surgical hub. The transceiver 683 may receive data (e.g., cartridge data, loading unit data, adapter data, or other notifications) from the surgical hub 670. The transceiver 683 may also receive data (e.g., cartridge data, loading unit data, or adapter data) from other components of the system 680. For example, the controller 698 may transmit instrument data to the console 694, including the serial number of an attached adapter (e.g., adapter 685) attached to the handle 697, the serial number of a loading unit (e.g., loading unit 687) attached to the adapter 685, and the serial number of a multi-fire fastener cartridge loaded in the loading unit. The console 694 may then return data associated with the attached cartridge, loading unit, and adapter (e.g., cartridge data, loading unit data, or adapter data), respectively, to the controller 698. The controller 698 may display a message on a local instrument display or may send a message via the transceiver 683 to the console 694 or portable device 696, respectively, to display the message on the display 695 or portable device screen.

[0116] 7A illustrates a surgical system 700 that may include a matrix of surgical information. This surgical information may include any discrete atoms of information related to a surgical procedure. Generally described, such surgical information may include information related to the context and scope of the surgical procedure itself (e.g., healthcare information 728). Such information may include data such as, for example, procedure data and patient record data. The procedure data and / or patient record data may be associated with an associated healthcare data system 716 that is in communication with the surgical computing device 704.

[0117] The procedure data may include information related to the instruments and / or replaceable instrument parts used in a given procedure, such as, for example, a master list. The surgical computing device 704 may record (e.g., capture barcode scans) the instruments and / or replaceable instrument components used in the procedure. Such surgical information may be used by an algorithm to verify that the proper configuration of surgical instruments and / or replaceable components is being used. See U.S. Patent Application Publication No. 2020-0405296(A1), entitled "PACKAGING FOR A REPLACEABLE COMPONENT OF A SURGICAL STAPLING SYSTEM," filed June 30, 2019 (U.S. Patent Application No. 16 / 458,103), the contents of which are incorporated herein by reference in their entirety.

[0118] For example, patient record data may be suitable for use in modifying the configuration of a particular surgical device. For example, patient data may be used to understand and improve algorithmic behavior of a surgical device. In one example, a surgical stapler can adjust operating parameters related to compression, speed of operation, position of use, and feedback based on information in the patient record (e.g., information indicative of a particular patient's tissue and / or tissue characteristics). See U.S. Patent Application Publication No. 2019-0200981(A1), filed December 4, 2018, entitled "METHOD OF COMPRESSING TISSUE WITHIN A STAPLING DEVICE AND SIMULTANEOUSLY DISPLAYING THE LOCATION OF THE TISSUE WITHIN THE JAWS" (U.S. Patent Application No. 16 / 209,423), the contents of which are incorporated herein by reference in their entirety.

[0119] The surgical information may include information related to the configuration and / or control of devices being used in the procedure (e.g., device operation information 729). Such device operation information 729 may include information regarding the initial configuration of a surgical device. Device operation information 729 may include information regarding changes to the configuration of a surgical device. Device operation information 729 may include information regarding controls sent to the device from the surgical computing device 704 and the information flow associated with such controls.

[0120] Surgical information may include information generated during the surgery itself (e.g., surgical information 727). Such surgical information 727 may include any information generated by a surgical data source 726. The data source 726 may include any device within the surgical context that may generate useful surgical information 727. This surgical information 727 may present itself as an observable quality of the data source 726. The observable quality may include static qualities such as a device's model number, serial number, etc. The observable quality may include dynamic qualities such as the state of a device's configurable settings. Surgical information 727 may present itself, for example, as the result of sensor observations. Sensor observations may include observations from specific sensors in the operating room, sensors for monitoring conditions such as the patient's condition, sensors embedded in surgical devices, etc. Sensor observations may include information used during the surgery, such as video, audio, etc. Surgical information 727 may present itself as device event data. The surgical device may generate notifications and / or log events, and such events may be included in the surgical information 727 for communication to the surgical computing device 704. The surgical information 727 may present itself, for example, as a result of manual recording. A medical professional may record during the procedure by asking the patient to take notes, capturing still images from the display, etc.

[0121] The surgical data sources 726 may include modular devices (e.g., which may include sensors configured to detect parameters associated with the patient, HCP, and environment, and / or the modular devices themselves), local databases (e.g., a local EMR database containing patient records), patient monitoring devices (e.g., blood pressure (BP) monitors and electrocardiography (EKG) monitors), HCP monitoring devices, environmental monitoring devices, surgical instruments, surgical procedure support equipment, etc.

[0122] Intelligent surgical instruments can sense and measure certain operating parameters during the course of their operation. For example, intelligent surgical instruments, such as surgical robots, digital laparoscopic devices, and the like, may use such measurements to improve their operation, for example, limiting excessive compression, reducing collateral damage, minimizing tissue tension, optimizing application position, etc. See U.S. Patent Application Publication No. 2018-0049822(A1), filed August 16, 2016, entitled "CONTROL OF ADVANCEMENT RATE AND APPLICATION FORCE BASED ON MEASURED FORCES" (U.S. Patent Application No. 15 / 237,753), the contents of which are incorporated herein by reference in their entirety. Such surgical information can be communicated to the surgical computing device 704.

[0123] The surgical computing device 704 may be configured to derive contextual information regarding the surgical procedure from the data based, for example, on a particular combination of received data or a particular order in which data is received from the data sources 726. Contextual information inferred from the received data may include, for example, the type of surgical procedure being performed, the particular step of the surgical procedure the surgeon is performing, the type of tissue being operated on, or the body cavity that is the target of the procedure. This ability, according to some aspects of the surgical computing device 704, to derive or infer information regarding the surgical procedure from the received data may be referred to as “situational awareness.” For example, the surgical computing device 704 may incorporate a situational awareness system, which is hardware and / or programming associated with the surgical computing device 704 that derives contextual information regarding the surgical procedure from the received data and / or surgical planning information received from the edge computing system 714 or healthcare data system 716 (e.g., an enterprise cloud server). Such situational awareness capabilities may be used to generate surgical information (such as control and / or configuration information) based on sensed conditions and / or use. See U.S. Patent Application Publication No. 2019-0104919(A1) (U.S. Patent Application No. 16 / 209,478), filed December 4, 2018, entitled "METHOD FOR SITUATIONAL AWARENESS FOR SURGICAL NETWORK OR SURGICAL NETWORK CONNECTED DEVICE CAPABLE OF ADJUSTING FUNCTION BASED ON A SENSED SITUATION OR USAGE," the contents of which are incorporated herein by reference in their entirety.

[0124] During operation, this matrix of surgical information may exist as one or more information flows. For example, surgical information may flow from a surgical data source 726 to the surgical computing device 704. Surgical information may flow from the surgical computing device 704 to a surgical data source 726 (e.g., a surgical device). Surgical information may flow between the surgical computing device 704 and one or more healthcare data systems 716. Surgical information may flow between the surgical computing device 704 and one or more edge computing devices 714. Aspects of the information flows, including, for example, information flow endpoints, information storage, data interpretation, etc., may be managed for the surgical system 700 (e.g., for the medical facility). See U.S. Patent Application Publication No. 2019-0206564(A1), entitled METHOD FOR FACILITY DATA COLLECTION AND INTERPRETATION, filed December 4, 2018 (U.S. Patent Application No. 16 / 209,490), the entire contents of which are incorporated herein by reference.

[0125] The surgical information as presented in one or more information flows may be used in conjunction with one or more artificial intelligence (AI) systems to further enhance the operation of the surgical system 700. For example, a machine learning system, such as those described herein, may operate on one or more of the information flows to further enhance the operation of the surgical system 700.

[0126] 7B shows an exemplary computer-implemented surgical system 730 having multiple information flows 732. The surgical computing device 704 may communicate with and / or incorporate one or more surgical data sources. For example, the imaging module 733 (and endoscope) may exchange surgical information with the surgical computing device 704. Such information may include information from the imaging module 733 (and endoscope), such as video information, current settings, system status information, etc. The imaging module 733 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (software / firmware, etc.).

[0127] For example, the generator module 734 (and corresponding energy devices) may exchange surgical information with the surgical computing device 704. Such information may include information from the generator module 734 (and corresponding energy devices), such as electrical information (e.g., current, voltage, impedance, frequency, wattage), activity state information, sensor information such as temperature, current settings, system events, active duration, and startup timestamps. The generator module 734 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to the medical professional (e.g., changes in the pitch, duration, and melody of an audible tone), electrical application profiles and / or application logic that may instruct the generator module to provide energy having a defined characteristic curve over the application time, operational updates (e.g., software / firmware), and the like.

[0128] For example, the smoke evacuator 735 may exchange surgical information with the surgical computing device 704. Such information may include information from the smoke evacuator 735 such as operational information (e.g., revolutions per minute), activity status information, sensor information such as temperature, current settings, system events, active duration, and boot timestamps. The smoke evacuator 735 may receive information from the surgical computing device 704 such as control information, configuration information, operational updates (software / firmware, etc.).

[0129] For example, the aspirate / irrigate module 736 may exchange surgical information with the surgical computing device 704. Such information may include information from the aspirate / irrigate module 736, such as operational information (e.g., liters per minute), activity status information, internal sensor information, current settings, system events, active duration, and startup timestamps. The aspirate / irrigate module 736 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (software / firmware, etc.).

[0130] For example, the communications module 739, the processor module 737, and / or the storage array 738 may exchange surgical information with the surgical computing device 704. In one example, the communications module 739, the processor module 737, and / or the storage array 738 may comprise all or part of the computing platform on which the surgical computing device 704 operates. In one example, the communications module 739, the processor module 737, and / or the storage array 738 may provide local computing resources to other devices in the surgical system 730. Information from the communications module 739, the processor module 737, and / or the storage array 738 to the surgical computing device 704 may include logical computing related reports such as processing load, processing power, process identification, CPU %, CPU time, threads, GPU %, GPU time, memory utilization, memory threads, memory ports, energy usage, bandwidth related information, packets in, packets out, data rates, channel utilization, buffer status, packet loss information, system events, and other status information. The communications module 739, processor module 737, and / or storage array 738 may receive information, such as control information, configuration information, operational updates (software / firmware, etc.), etc., from the surgical computing device 704. The communications module 739, processor module 737, and / or storage array 738 may also receive information from the surgical computing device 704 generated by another element or device of the surgical system 730. For example, data source information may be transmitted to and stored in the storage array. For example, the data source information may be processed by the processor module 737.

[0131] For example, the intelligent instrument 740 (with or without a corresponding display) may exchange surgical information with the surgical computing device 704. Such information may include information from the intelligent instrument 740 regarding the operation of the instrument, such as device electrical and / or mechanical information (e.g., current, voltage, impedance, frequency, wattage, torque, force, pressure, etc.), load status information (e.g., information regarding the identity, type, and / or status of reusables such as staple cartridges), clamping force, tissue compression pressure, and / or internal sensor information such as time, system events, active duration, and activation timestamp. The intelligent instrument 740 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to the medical professional (e.g., changes in the pitch, duration, and melody of an audible tone), mechanical application profiles and / or application logic that may instruct the instrument's mechanical components to operate with defined characteristics (e.g., blade / anvil advancement speed, mechanical advantage, firing time, etc.), operational updates (software / firmware, etc.), and the like.

[0132] For example, in a surgical stapling and severing instrument, control and configuration information can be used to modify operating parameters such as, for example, motor speed. Data collection of surgical information can be used to define the power, force, and / or other functional operations and / or behavior of the intelligent surgical stapling and severing instrument. See U.S. Patent No. 10,881,399(B2) (U.S. Patent Application No. 15 / 628,175), filed June 20, 2017, entitled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," the contents of which are incorporated herein by reference in their entirety.

[0133] For example, energy devices can use control and configuration information (e.g., control and configuration information based on the situational awareness of the surgical computing device 704) to adapt their functionality and / or behavior for improved results. See U.S. Patent Application Publication No. 2019-0201047(A1), entitled "METHOD FOR SMART ENERGY DEVICE INFRASTRUCTURE," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,458), the entire contents of which are incorporated herein by reference. Similarly, combo-energy devices (e.g., devices capable of using two or more energy modalities) can use such control and / or configuration information to select an appropriate operating mode. For example, the surgical computing device 704 can use surgical information, including information received from patient monitoring, to send control and / or configuration information to the combo-energy device. See U.S. Patent Application Publication No. 2017-0202605(A1), filed December 16, 2016, entitled "MODULAR BATTERY POWERED HANDHELD SURGICAL INSTRUMENT AND METHODS THEREFOR" (U.S. Patent Application No. 15 / 382,515), the entire contents of which are incorporated herein by reference.

[0134] For example, the sensor module 741 may exchange surgical information with the surgical computing device 704. Such information may include information from the sensor module 741 about its sensor capabilities, such as the sensor results themselves, observation frequency and / or resolution, observation type, device alerts such as alerts for sensor failure, observations exceeding a defined range, observations exceeding an observable range, etc. The sensor module 741 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of the observations (e.g., frequency, resolution, observation type, etc.), triggers defining specific events for observations, on controls, off controls, data buffering, data pre-processing algorithms, operational updates (software / firmware, etc.), etc.

[0135] For example, the visualization system 742 may exchange surgical information with the surgical computing device 704. Such information may include information from the visualization system 742, such visualization data itself (e.g., still images, video, advanced spectral visualization, etc.), visualization metadata (e.g., visualization type, resolution, frame rate, encoding, bandwidth, etc.), etc. The visualization system 742 may receive information from the surgical computing device 704, such as control information, configuration information, changes in video settings (e.g., visualization type, resolution, frame rate, encoding, etc.), visual display overlay data, data buffering sizes, data pre-processing algorithms, operational updates (software / firmware, etc.), etc.

[0136] Surgical information may be exchanged and / or used with advanced imaging systems. For example, surgical information may be exchanged and / or used to provide context for imaging data streams. For example, surgical information may be exchanged and / or used to expand conditional understanding of such imaging data streams. See U.S. Patent Application No. 17 / 493,904, entitled "SURGICAL METHODS USING MULTI-SOURCE IMAGING," filed October 5, 2021, the contents of which are incorporated herein by reference in their entirety. See U.S. Patent Application No. 17 / 493,913, entitled "SURGICAL METHODS USING FIDUCIAL IDENTIFICATION AND TRACKING," filed October 5, 2021, the contents of which are incorporated herein by reference in their entirety.

[0137] For example, the surgical robot 743 may exchange surgical information with the surgical computing device 704. In one example, the surgical information may include information related to collaborative registration and interaction of surgical robotic systems. See U.S. Patent Application No. 17 / 449,765, filed October 1, 2021, entitled "COOPERATIVE ACCESS HYBRID PROCEDURES," the entire contents of which are incorporated herein by reference. Information from the surgical robot 743 may include any of the aforementioned information as it applies to robotic instruments, sensors, and devices. Information from the surgical robot 743 may also include information related to the robotic operation or control of such instruments, such as electrical / mechanical feedback of the robotic articulator, system events, system settings, mechanical resolution, control operation logs, articulator path information, etc. The surgical robot 743 may receive information from the surgical computing device 704, such as control information, configuration information, operation updates (software / firmware, etc.), etc.

[0138] Surgical devices in communication with the surgical computing device 704 may exchange surgical information to support coordinated operations between the devices. For example, the surgical robot 743 and the energy generator 734 may exchange surgical information with each other and / or with the surgical computing device 704 for cooperative operation. Cooperative operation between the surgical robot 743 and the energy generator 734 acting cooperatively can be used to minimize undesirable side effects, such as tissue sticking. Cooperative operation between the surgical robot 743 and the energy generator 734 acting cooperatively can be used to improve tissue welding. See U.S. Patent Application Publication No. 2019-0059929(A1), entitled "METHODS, SYSTEMS, AND DEVICES FOR CONTROLLING ELECTROSURGICAL TOOLS," filed August 29, 2017 (U.S. Patent Application No. 15 / 689,072), the contents of which are incorporated herein by reference in their entirety. The surgical information may be generated by the collaborating devices and / or the surgical computing device 704 in connection with their collaborative operation.

[0139] The surgical computing system 704 can record, analyze, and / or act on surgical information flows, such as those disclosed above. The surgical computing system 704 can aggregate such data for analysis. For example, the surgical computing system 704 can perform operations such as defining device relationships, establishing device collaboration behaviors, monitoring procedure details, and / or storing procedure details. Surgical information associated with such operations may be further analyzed to refine algorithms, identify trends, and / or adapt surgical procedures. For example, the surgical information may be further analyzed compared to patient outcomes responsive to such operations. See U.S. Patent Application Publication No. 2019-0206562(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, DISPLAY, AND CLOUD ANALYTICS," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,416), the contents of which are incorporated herein by reference in their entirety.

[0140] 7C illustrates an exemplary information flow associated with multiple surgical computing systems 704a, 704b within a common environment. As the overall configuration of a computer-implemented surgical system (e.g., computer-implemented surgical system 750) changes (e.g., as data sources are added and / or removed from the surgical computing system), additional surgical information may be generated to reflect the changes. In this example, a second surgical computing system 704b (e.g., a surgical hub) may be added to surgical system 750 (along with a corresponding surgical robot) comprising existing surgical computing system 704a. The messaging flows described herein represent additional surgical information flow 755 (e.g., further integrated, analyzed, and / or processed according to algorithms such as machine learning algorithms) used as disclosed herein.

[0141] Here, two surgical computing systems 704a, 704b request permission from the surgeon for the second surgical computing system 704b (with a corresponding surgical robot 756) to take control of the operating room from the existing surgical computing system 704a. The second surgical computing system 704b presents control of the corresponding surgical robot 756, robotic visualization tower 758, Monohat tool 759, and robotic stapler 749 in the operating room. Permission may be requested through the surgeon interface or console 751. Once permission is granted, the second surgical computing system 704b sends a message to the existing surgical computing system 704a requesting transfer of control of the operating room.

[0142] In one example, the surgical computing systems 704a, 704b can negotiate the nature of their interaction without external input based on previously collected data. For example, the surgical computing systems 704a, 704b may collectively determine that an upcoming surgical task requires the use of a robotic system. Such a determination may cause the existing surgical computing system 704a to autonomously hand over control of the operating room to a second surgical computing system 704b. Upon completion of the surgical task, the second surgical computing system 704b may then autonomously return control of the operating room to the existing surgical computing system 704a.

[0143] As illustrated in Figure 7C, the existing surgical computing system 704a has transferred control to a second surgical computing system 704b, which also assumes control of the surgeon interface 751 and secondary display 752. The second surgical computing system 704b assigns new identification numbers to the newly transferred devices. The existing surgical computing system 704a retains control of the handheld stapler 753, handheld powered dissector 754, and visualization tower 757. In addition, the existing surgical computing system 704a may perform a support role, with the processing and storage capabilities of the existing surgical computing system 704a now available to the second surgical computing system 704b.

[0144] 7D illustrates an exemplary surgical information flow in the context of a surgical procedure and corresponding exemplary uses of the surgical information for predictive modeling. The surgical information disclosed herein may provide data regarding one or more surgical procedures, including surgical tasks, instruments, instrument settings, motion information, procedural variations, and corresponding desirable metrics such as improved patient outcomes, lower costs (e.g., fewer resources utilized, shorter surgical time, etc.). The surgical information disclosed herein (e.g., that disclosed with respect to FIGS. 7A-7C ), in the context of one or more surgical systems and devices disclosed herein, provides a platform upon which certain machine learning algorithms and techniques disclosed herein may be used.

[0145] Surgical information 762 from multiple surgical procedures 764 (e.g., a subset of surgical information from each procedure) may be collected. Surgical information 762 may be collected from multiple surgical procedures 764, for example, by collecting data represented by one or more information flows disclosed herein.

[0146] To illustrate, an exemplary instance of surgical information 766 may be generated from an exemplary procedure 768 (e.g., a lung segmentectomy procedure as shown on timeline 769). Surgical information 766 may be generated during preoperative planning and may include patient record information. Surgical information 766 may be generated from data sources (e.g., data sources 726) during the course of a surgical procedure, including data generated each time medical personnel utilize a modular device paired with the surgical computing system (e.g., surgical computing system 704). The surgical computing system may receive this data from the paired modular device and other data sources. The surgical computing system itself may generate surgical information as part of its operation during a procedure. For example, the surgical computing system may record information related to configuration and control operations. The surgical computing system may record information related to situational awareness activities. For example, the surgical computing system may record recommendations, prompts, and / or other information provided to the medical team (e.g., provided via a display screen) that may be related to the next procedure step. For example, the surgical computing system may record configuration and control changes (e.g., adjustments to modular devices based on context) that may include activating a monitor, adjusting the field of view (FOV) of a medical imaging device, changing the energy level of an ultrasonic surgical instrument or an RF electrosurgical instrument, etc.

[0147] Hospital personnel retrieve the patient's EMR from the hospital's EMR database at 770. Based on the patient data selected in the EMR, the surgical computing system determines that the procedure to be performed is a thoracic procedure.

[0148] At 771, personnel scan incoming medical supplies for a procedure. The surgical computing system may cross-reference the scanned supplies with a list of supplies utilized in various types of procedures. The surgical computing system may verify that the mix of supplies corresponds to a thoracic procedure. Additionally, the surgical computing system may determine that the procedure is not a wedge resection (because the incoming supplies either do not include certain supplies needed for a thoracic wedge resection or are otherwise not compatible with a thoracic wedge resection). The medical personnel may scan a patient band via a scanner communicatively connected to the surgical computing system. The surgical computing system may verify the patient's identity based on the scanned data.

[0149] At 774, medical personnel turn on auxiliary equipment. The auxiliary equipment utilized may vary according to the type of surgical procedure and the technology used by the surgeon. In this example, the auxiliary equipment may include a smoke evacuator, an aspirator, and a medical imaging device. Once activated, the auxiliary equipment may pair with the surgical computing system. The surgical computing system may derive contextual information regarding the surgical procedure based on the paired type. In this example, the surgical computing system determines that the surgical procedure is a VATS procedure based on this particular combination of paired devices. The contextual information regarding the surgical procedure may be ascertained by the surgical computing system via information from the patient's EMR.

[0150] The surgical computing system may retrieve the steps of the procedure to be performed. For example, the steps may be associated with a treatment plan (e.g., a treatment plan specific to this patient's surgery, a treatment plan associated with a particular surgeon, a treatment plan template for the procedure in general, etc.).

[0151] At 775, staff attach EKG electrodes and other patient monitoring devices to the patient. The EKG electrodes and other patient monitoring devices pair with the surgical computing system. The surgical computing system may receive data from the patient monitoring devices.

[0152] At 776, medical personnel induce anesthesia in the patient. The surgical computing system may record information related to this procedure step, such as data from the modular devices and / or patient monitoring devices, including, for example, EKG data, blood pressure data, ventilator data, or a combination thereof.

[0153] At 777, the lung of the patient undergoing surgery is collapsed (and ventilation may be switched to the contralateral lung). The surgical computing system may determine that this procedure step has begun and may collect surgical information accordingly, including, for example, ventilator data, one or more timestamps, etc.

[0154] At 778, a medical imaging device (e.g., a scope) is inserted and video from the medical imaging device is initiated. The surgical computing system may receive medical imaging device data (i.e., video or image data) through a connection to the medical imaging device. The data from the medical imaging device may include imaging data and / or imaging metadata, such as the angle at which the medical imaging device is oriented relative to visualization of the patient's anatomy, the number of medical imaging devices currently active, etc. The surgical computing system may record positioning information for the medical imaging device. For example, one technique for performing a VATS lobectomy places the camera in the anterior-inferior corner of the patient's thoracic cavity above the diaphragm. Another technique for performing a VATS segmentectomy places the camera in an anterior intercostal position relative to the segmental fissure.

[0155] For example, using pattern recognition or machine learning techniques, the surgical computing system may be trained to recognize the positioning of a medical imaging device according to visualization of the patient's anatomy. For example, one technique for performing a VATS lobectomy utilizes a single medical imaging device. Another technique for performing a VATS segmentectomy uses multiple cameras. Yet another technique for performing a VATS segmentectomy uses an infrared light source (which may be communicatively coupled to the surgical computing system as part of the visualization system).

[0156] At 779, the surgical team begins the incision step of the procedure. The surgical computing system may collect data from the RF or ultrasonic generator indicating that the energy instrument is being fired. The surgical computing system may cross-reference the received data with the retrieved steps of the surgical procedure to determine that the energy instrument being fired at this point in the process (i.e., after previously discussed steps of the procedure have been completed) corresponds to the incision step. In one example, the energy instrument may be an energy tool mounted on a robotic arm of a robotic surgical system.

[0157] At 780, the surgical team proceeds to the ligation step of the procedure. The surgical computing system may collect surgical information 766 related to the surgeon ligating the arteries and veins based on receiving data from the surgical stapling and severing instrument indicating that such instrument is being fired. Next, the segmentectomy portion of the procedure is performed. The surgical computing system may collect information related to the surgeon transecting the parenchyma. For example, the surgical computing system may receive surgical information 766 from the surgical stapling and severing instrument, including data related to its cartridge, settings, firing details, etc.

[0158] At 782, the node dissection step is then performed. The surgical computing system may collect surgical information 766 related to the surgical team dissecting the node and performing the leak test. For example, the surgical computing system may collect data received from the generator indicating that an RF or ultrasonic instrument is being fired, including electrical and status information associated with the firing. The surgeon periodically alternates between the surgical stapling / cutting instrument and the surgical energy (i.e., RF or ultrasonic) instrument depending on the particular step in the procedure. The surgical computing system may collect surgical information 766 taking into account the particular sequence in which the stapling / cutting instrument and the surgical energy instrument are used. In one example, a robotic tool may be used for one or more steps in the surgical procedure. The surgeon may, for example, alternate between using the robotic tool and a handheld surgical instrument and / or use the devices simultaneously.

[0159] The incisions are then closed and the post-operative portion of the procedure begins. The patient is de-anesthetized at 784. The surgical computing system may collect surgical information regarding the patient emerging from anesthesia, for example, based on ventilator data (e.g., the patient's breathing rate begins to increase).

[0160] At 785, medical personnel remove various patient monitoring devices from the patient. The surgical computing system may collect information regarding the outcome of the procedure. For example, the surgical computing system may collect information related to the loss of EKG, BP, and other data from the patient monitoring devices.

[0161] The surgical information 762 (including the surgical information 766) may be structured and / or labeled. The surgical computing system may inherently provide such structure and / or labeling in the data collection. For example, the surgical information 762 may be labeled according to particular characteristics, desired results (e.g., efficiency, patient outcome, cost, and / or combinations thereof, etc.), particular surgical techniques, aspects of instrumentation (e.g., surgical instrument selection, timing, and activation, instrument settings, nature of instrument use, etc.), identities of medical professionals involved, particular patient characteristics, etc., each of which may be present in the data collection.

[0162] The surgical information (e.g., surgical information 762 collected over the procedure 764) may be used in connection with one or more artificial intelligence (AI) systems. AI may be used to perform computer cognitive tasks. For example, AI may be used to perform complex tasks based on observation of data. AI may be used to enable computing systems to perform cognitive tasks and solve complex tasks. AI may include using machine learning and machine learning techniques. ML techniques may include, for example, performing complex tasks without being programmed (e.g., explicitly programmed). For example, ML techniques may improve over time based on completing tasks with different inputs. An ML process may train itself, for example, using input data and / or a training dataset.

[0163] Machine learning (ML) techniques may be used, for example, in the medical field. For example, ML may be used on a set of data (e.g., a set of surgical data) to generate output (e.g., reduced surgical data, processed surgical data). In an example, the output of the ML process may include identified trends or relationships in the data input for processing. The output may include verifying results and / or outcomes associated with the input data. In an example, the input to the ML process may include medical data such as surgical images and patient scans. The ML process may output a determined medical condition based on the input surgical images and patient scans. The ML process may be used to diagnose a medical condition, for example, based on the surgical scans.

[0164] An ML process may improve itself, for example, using historical data and / or input data that trained the ML process. Thus, an ML process may continually improve with added inputs and processing. The ML process may update based on the input data. For example, over time, an ML process that generates medical outcomes based on medical data may improve and become more accurate and consistent in medical diagnoses.

[0165] ML processes may be used to solve different complex tasks (e.g., medical tasks). For example, ML processes may be used for data reduction, data preparation, data processing, trend identification, outcome determination, medical diagnosis, and / or the like. For example, an ML process may take surgical data as input and process the data for use in medical analysis. The processed data may be used to determine a medical diagnosis. Finally, an ML process may take raw surgical data and generate useful medical information (e.g., medical trends and / or diagnoses) associated with the raw surgical data.

[0166] ML processes may be combined to perform different discrete tasks on an input data set. For example, an ML process may include testing different combinations of ML subprocesses performing discrete tasks to determine which combination performs best (e.g., competitive use of different process / algorithm types and training to determine the best combination for a data set). For example, an ML process may include subprocess (e.g., algorithm) control and monitoring to refine and / or validate results and / or outcomes (e.g., error bounds).

[0167] An ML process may be initialized and / or set up to perform a task. For example, the ML process may be initialized based on initialization configuration information. The initialized ML process may be an untrained ML process and / or a base ML process for performing the task. An untrained ML process may be inaccurate in performing a specified task. As the ML process is trained, the task may be performed more accurately.

[0168] The initialization configuration information for the ML process may include initial settings and / or parameters. For example, the initial settings and / or parameters may include defined ranges for use by the ML process. The ranges may include manually entered and / or received data ranges. The ranges may include default ranges and / or randomized ranges for unreceived variables that may be used, for example, to complete the dataset for processing. For example, if a dataset lacks a data range, the default data range may be used as a substitute to run the ML process.

[0169] The initialization configuration information for the ML process may include data storage locations. For example, locations or data storage and / or databases associated with data interactions may be included. The databases associated with data interactions may be used to identify trends in the dataset. The databases associated with data interactions may include mappings of data to medical conditions. For example, the databases associated with data interactions may include mappings of heart rate data to arrhythmias, etc.

[0170] The initialization configuration information may include parameters associated with defining the system. The initialization configuration information may include instructions (e.g., methods) associated with displaying, verifying, and / or providing information to a user. For example, the initialization configuration may include instructions for an ML process to output data in a particular format for visualization to a user.

[0171] ML techniques may be used, for example, to perform data reduction. ML techniques for data reduction may include using multiple different data reduction techniques. For example, ML techniques for data reduction may include using one or more of the following: CUR matrix decomposition; decision trees; expectation-maximization (EM) processes (e.g., algorithms); explicit semantic analysis (ESA); exponential smoothing forecasting; generalized linear models; k-means clustering (e.g., nearest neighbor); naive Bayes; neural network processes; multivariate analysis; o-cluster; singular value decomposition; Q-learning; temporal difference (TD); deep adversarial networks; support vector machines (SVM); linear regression; dimensionality reduction; linear discriminant analysis (LDA); adaptive boosting (e.g., AdaBoost); gradient descent (e.g., stochastic gradient descent (SGD)); outlier detection; and / or others.

[0172] ML techniques may be used to perform data reduction, for example, using CUR matrix decomposition. CUR matrix decomposition may include using a matrix decomposition model (e.g., process, algorithm), such as a low-rank matrix decomposition model. For example, CUR matrix decomposition may include a low-rank matrix decomposition process that is expressed (e.g., explicitly expressed) in several (e.g., a small number) columns and / or rows of a data matrix (e.g., the CUR matrix decomposition may be interpretable). CUR matrix decomposition may include selecting columns and / or rows associated with statistical leverage and / or large influence in the data matrix. Using CUR matrix decomposition may enable identifying attributes and / or rows within the data matrix. Simplification of larger datasets (e.g., using CUR matrix decomposition) may enable users to review and interact (e.g., with the data). CUR matrix decomposition may facilitate regression, classification, clustering, and / or other processes.

[0173] ML techniques may be used to perform data reduction using, for example, decision trees (e.g., decision tree models). Decision trees may be used, for example, as a framework for quantifying outcome values ​​and / or the probability of an outcome occurring. Decision trees may be used, for example, to calculate values ​​for uncertain outcome nodes (e.g., in a decision tree). Decision trees may be used, for example, to calculate values ​​for decision nodes (e.g., in a decision tree). Decision trees may be models that enable classification and / or regression (e.g., applicable to classification and / or regression problems). Decision trees may be used to analyze numerical (e.g., continuous) and / or categorical data. Decision trees may be more successful and / or more efficient with large datasets (e.g., compared to other data reduction techniques).

[0174] Decision trees may be used in combination with other decision trees. For example, a random forest may refer to a collection of decision trees (e.g., an ensemble of decision trees). A random forest may include a collection of decision trees whose results may be aggregated into a result. A random forest may be a supervised learning algorithm. A random forest may be trained, for example, using a bagging training process.

[0175] A random decision forest (e.g., random forest) may add randomness (e.g., additional randomness) to a model, for example, while growing a tree. A random forest may be used, for example, to search for the best feature among a random subset of features, rather than searching for the most important feature (e.g., while splitting a node). Searching for the best feature among a random subset of features may result in a wide variety, which may result in a better (e.g., more efficient and / or accurate) model.

[0176] Random forests may include using parallel ensembles. Parallel ensembles may include, for example, fitting (e.g., several) decision tree classifiers in parallel on different dataset subsamples. Parallel ensembles may include using majority voting or averaging on the results or final outcome. Parallel ensembles may be used to minimize overfitting and / or increase prediction accuracy and control. Random forests with multiple decision trees may (e.g., generally) be more accurate than single decision tree-based models. A set of decision trees with controlled variation may be constructed, for example, by combining bootstrap aggregation (e.g., bagging) with random feature selection.

[0177] ML techniques may be used to perform data reduction, for example, using an expectation-maximization (EM) model (e.g., process, algorithm). For example, an EM model may be used to find likelihood (e.g., local maximum likelihood) parameters of a statistical model. An EM model may be used when equations cannot be solved directly. An EM model may consider latent variables and / or unknown parameters and known data observations. For example, an EM model may determine that missing values ​​are present in a dataset. An EM model receives configuration information indicating to assume the presence of missing (e.g., unobserved) data points in the dataset.

[0178] The EM model may use component clustering. For example, component clustering may allow EM components to be grouped into high-level clusters. For example, if component clustering is disabled (e.g., in the EM model), the components may be treated as clustered.

[0179] ML techniques may be used to perform data reduction, for example, using Explicit Semantic Analysis (ESA). ESA may be used at the level of semantics (e.g., meaning) rather than the vocabulary (e.g., surface form vocabulary) of words or documents. ESA may focus on the meaning of a set of text, for example, as a combination of concepts found within the text. ESA may be used for document classification. ESA may be used for semantic relevance computation (e.g., how similar words or fragments of text are to each other). ESA may be used for information retrieval.

[0180] ESAs may be used, for example, for document classification. Document classification may include tagging documents for management and sorting. Tagging documents (e.g., with keywords) may make them easier to search. Keyword tagging (e.g., using only keyword tagging) may limit the accuracy and / or efficiency of document classification. For example, using keyword tagging may reveal (e.g., only) documents that have the keyword, but not documents that have words with similar meanings to the keyword. Semantically classifying text (e.g., using ESAs) may improve a model's understanding of the text. Semantically classifying text may include representing documents as concepts and reducing reliance on specific keywords.

[0181] ML techniques may be used to perform data reduction, for example, using an exponential smoothing forecasting model. Exponential smoothing may be used to smooth time series data, for example, using an exponential window function. For example, in a moving average, past observations may be weighted equally, but using an exponential function, weights may be assigned that decrease exponentially over time.

[0182] ML techniques may be used to perform data reduction, for example, using linear regression. Linear regression may be used to predict continuous outcomes. For example, linear regression may be used to predict the value of a variable (e.g., a dependent variable) based on the values ​​of different variables (e.g., independent variables). Linear regression may apply a linear approach to model the relationship between a scalar response and one or more explanatory variables (e.g., a dependent variable and / or independent variables). Simple linear regression may refer to a linear regression use case associated with one explanatory variable. Multiple linear regression may refer to a linear regression use case associated with two or more explanatory variables. Linear regression may model the relationship, for example, using a linear prediction function. The linear prediction function may estimate unknown model parameters from a dataset.

[0183] For example, linear regression may be used to identify patterns within a training dataset. The identified patterns may relate to groupings of values ​​and / or labels. The model may learn the relationship between (e.g., each) label and expected outcomes. After training, the model may be used on raw data outside the training dataset (e.g., data that does not have a mapped and / or known output). A trained model using linear regression may determine a calculated prediction associated with the raw data, such as identifying seasonal changes in sales data.

[0184] ML techniques may be used to perform data reduction, e.g., generalized linear models (GLMs). GLMs may be used as flexible generalizations of linear regression. GLMs may generalize linear regression, for example, by allowing linear models to relate response variables.

[0185] ML techniques may be used to perform data reduction, for example, using k-means clustering (e.g., nearest neighbor models). K-means clustering may be used in vector quantization. K-means clustering may be used in signal processing. K-means clustering may, for example, aim to divide n observations into k clusters, with each observation falling into the cluster with the closest mean.

[0186] K-means clustering may include K-Nearest Neighbor (KNN) learning. KNN may be instance-based learning (e.g., non-generalized learning, lazy learning). KNN may refrain from building a general internal model. KNN may include storing instances corresponding to training data in an n-dimensional space. KNN may use the data to classify data points, for example, based on a similarity measure (e.g., a Euclidean distance function). Classification may be calculated, for example, based on a majority vote of the k neighbors of a point (e.g., each point). KNN may be robust to noisy training data. Accuracy may depend on data quality (e.g., for KNN). KNN may include selecting the number of neighbors to consider (e.g., an optimal number of neighbors to consider). KNN may be used for classification and / or regression.

[0187] ML techniques may be used to perform data reduction, for example, using a naive Bayes model (e.g., process). For example, a naive Bayes model may be used to build a classifier. Using the naive Bayes model, a class label may be assigned to a problem instance (e.g., represented as a vector of feature values). The class label may be drawn from a set (e.g., a finite set). Different processes (e.g., algorithms) may be used to train the classifier. A family of processes (e.g., a family of algorithms) may be used. A family of processes may be based on the principle that a naive Bayes classifier (e.g., all naive Bayes) classifier assumes that feature values ​​are independent of different feature values ​​(e.g., given a class variable).

[0188] ML techniques may be used to perform data reduction, for example, using neural networks. The neural network may learn (e.g., be trained) by processing examples, for example, to perform other tasks (e.g., similar tasks). The processed examples may include inputs and results (e.g., inputs mapped to results). The neural network may learn by forming probability-weighted associations between inputs and results. The probability-weighted associations may be stored within the neural network's data structure. Training the neural network from a given example may be performed by determining the difference between the network's processed output (e.g., prediction) and a target output. This difference may be an error. The neural network may adjust the weighted associations (e.g., stored weighted associations), for example, according to a learning rule and an error value.

[0189] ML techniques may be used to perform data reduction, for example, using multivariate analysis, which may include performing multivariate state estimation and / or non-negative matrix factorization.

[0190] ML techniques may be used to perform data reduction using, for example, a support vector machine (SVM). SVMs may be used in multidimensional spaces (e.g., high-dimensional spaces, infinite-dimensional spaces). SVMs may be used to construct hyperplanes (e.g., a set of hyperplanes). A hyperplane with the greatest distance (e.g., compared to other constructed hyperplanes) from the nearest training data point within a class (e.g., any class) may achieve strong separation (e.g., generally, the larger the margin, the lower the generalization error of the classifier). SVMs may be effective in high-dimensional spaces. SVMs may behave differently, for example, based on different mathematical functions (e.g., kernels, kernel functions). For example, kernel functions may include one or more of linear, polynomial, radial basis function (RBF), sigmoid, etc. Kernel functions may be used as SVM classifiers. SVMs may be limited, for example, in use cases where the dataset contains a large amount of noise (e.g., overlapping target classes).

[0191] ML techniques may be used to perform data reduction, such as, for example, dimensionality reduction. Reducing the dimensionality of a sample of data (e.g., unlabeled data) may help refine groups and / or clusters. Reducing the number of variables in a model may simplify the data's trends. Simplified data trends may allow for more efficient processing. Dimensionality reduction may be used, for example, when many (e.g., too many) dimensions obscure (e.g., adversely affect) insights, trends, patterns, outcomes, and / or the like.

[0192] Reducing dimensionality may include using principal component analysis (PCA). PCA may be used to establish principal components that govern the relationships between data points. PCA may focus on simplifying (e.g., simplifying only) the principal components. Dimensionality reduction (e.g., PCA) may be used to maintain the diversity of data groupings within a dataset, but reduce the number of distinct groups.

[0193] ML techniques may be used to perform data reduction, e.g., linear discriminant analysis (LDA). LDA may refer to a linear decision boundary classifier, which may be created, for example, by fitting class conditional densities to data (e.g., and applying Bayes' rule). LDA may include a generalization of Fisher's Linear Discriminant (e.g., projecting a given dataset into a lower-dimensional space to reduce dimensionality, minimize model complexity, and reduce computational cost). An LDA model (e.g., a standard LDA model) may fit classes with Gaussian densities. An LDA model may assume that classes (e.g., all classes) share a covariance matrix. LDA may be similar to an analysis of variance (ANOVA) process and / or regression analysis. For example, LDA may be used to express a dependent variable as a linear combination of other features and / or measurements.

[0194] ML techniques may be used to perform data reduction, such as, for example, adaptive boosting (e.g., AdaBoost). Adaptive boosting may include creating a classifier (e.g., a powerful classifier). Adaptive boosting may include creating a classifier by combining multiple classifiers (e.g., poorly performing classifiers), for example, to obtain a resulting classifier with high accuracy. AdaBoost may be an adaptive classifier that improves the efficiency of a classifier. AdaBoost may trigger overfitting. AdaBoost may be used (e.g., most commonly used) to improve the performance of decision trees, base estimators, binary classification problems, and / or the like. AdaBoost may be sensitive to noisy data and / or outliers.

[0195] ML techniques may be used to perform data reduction, such as stochastic gradient descent (SGD). SGD may include an iterative process used to optimize a function (e.g., an objective function). SGD may be used, for example, to optimize an objective function with specific smoothness properties. Stochastic may refer to random probability. SGD may be used, for example, to reduce computational load in high-dimensional optimization problems. SGD may be used, for example, to enable faster iterations while trading off a slower convergence rate. Gradient may refer, for example, to the slope of a function that calculates the degree of change of a variable in response to a change in another variable. Gradient descent may refer to a convex function that outputs the partial derivative of a set of its input parameters. For example, α may be a learning rate, and J may be the cost of training examples for the i-th iteration. This formula may represent a stochastic gradient descent weight update method for the j-th iteration. In large-scale ML and sparse ML, SGD may be applied to problems in text classification and / or natural language processing (NLP). SGD can be sensitive to feature scaling (e.g., it may be necessary to use a range of hyperparameters, such as the regularization parameter and number of iterations).

[0196] ML techniques may be used to perform data reduction, such as using outlier detection. An outlier may be a data point that contains information (e.g., useful information) about abnormal behavior of the system described by the data. Outlier detection processes may include univariate and multivariate processes.

[0197] The ML process may be trained, for example, using one or more training methods. For example, the ML process may be trained using one or more of the following training techniques: supervised learning; unsupervised learning; semi-supervised learning; reinforcement learning; and / or others.

[0198] Machine learning can be supervised (e.g., supervised learning). Supervised learning algorithms can create a mathematical model from training data sets (e.g., training data). FIG. 8A illustrates an exemplary supervised learning framework 800. Training data (e.g., training examples 802, e.g., as shown in FIG. 8) may consist of a series of training examples (e.g., input data mapped to labeled outputs, e.g., as shown in FIG. 8A). Training examples 802 may include one or more inputs and one or more labeled outputs. The labeled outputs may serve as supervisory feedback. In the mathematical model, training examples 802 may be represented by an array or vector, sometimes called a feature vector. The training data may be represented by rows of the feature vector that form a matrix. Through iterative optimization of an objective function (e.g., a cost function), supervised learning algorithms can learn a function (e.g., a prediction function) that can be used to predict outputs associated with one or more new inputs. A properly trained predictive function (e.g., a trained ML model 808) can determine outputs 804 (e.g., labeled outputs) for one or more inputs 806 that may not be part of the training data (e.g., input data without mapped labeled outputs, as shown in FIG. 8A). Exemplary algorithms may include linear regression, logistic regression, neural networks, nearest neighbors, naive Bayes, decision trees, SVMs, and / or others. Exemplary problems that can be solved by supervised learning algorithms may include classification, regression problems, etc.

[0199] Machine learning can be unsupervised (e.g., unsupervised learning). FIG. 8B illustrates an exemplary unsupervised learning framework 810. An unsupervised learning algorithm 814 may train on a dataset that may include input 811 and may find structure 812 in the data (e.g., pattern detection and / or descriptive modeling). The structure 812 in the data may resemble groupings or clusterings of data points. Thus, the algorithm 814 may learn from training data that may be unlabeled. Instead of responding to supervised feedback, the unsupervised learning algorithm may identify commonalities in the training data and may react based on the presence or absence of such commonalities in each training data. For example, training may include operating on training input data to generate a model and / or output with a particular energy (e.g., cost function, etc.), and such energy may be used to further refine the model (e.g., to define a model that minimizes the cost function given the training input data). Exemplary algorithms may include the Apriori algorithm, K-means, K-nearest neighbors (KNN), K-medians, etc. Representative problems that can be solved by unsupervised learning algorithms may include clustering problems, anomaly / outlier detection problems, etc.

[0200] Machine learning may be semi-supervised (e.g., semi-supervised learning). Semi-supervised learning algorithms may be used in scenarios where labeling data is costly (e.g., because a skilled expert is required to label the data) and labels for the data are limited. Semi-supervised learning models may take advantage of the idea that while the group membership of unlabeled data is unknown, the data still holds important information about group parameters.

[0201] Machine learning may include reinforcement learning, which may be an area of ​​machine learning that may concern how a software agent can take actions in an environment to maximize some notion of cumulative reward. Reinforcement learning algorithms may not assume knowledge of an exact mathematical model of the environment (e.g., represented by a Markov decision process (MDP)) and may be used when an exact model may not be feasible. Reinforcement learning algorithms may be used in autonomous vehicles or in learning to play games against human opponents. Exemplary algorithms may include Q-learning, temporal difference (TD), deep adversarial networks, and / or others.

[0202] Reinforcement learning may involve an algorithm (e.g., an agent) that continuously learns from an environment in an iterative manner. During the training process, the agent may learn from experience with the environment until the agent has explored the full range of states (e.g., possible states). Reinforcement learning may be defined by the type of problem. Reinforcement learning solutions may be classified as reinforcement learning algorithms. In a problem, an agent may determine an action to select (e.g., a best action) based on the agent's current state. When steps are repeated, the problem may be referred to as an MDP.

[0203] For example, reinforcement learning may include an action step. The action step in reinforcement learning may include an agent observing an input state. The action step in reinforcement learning may include causing an agent to perform an action using a decision-making function. The action step may include an agent receiving a reward and / or reinforcement from the environment (e.g., after an action is performed). The action step in reinforcement learning may include storing state-action pair information related to the reward.

[0204] Machine learning may be part of a technology platform called cognitive computing (CC), which may comprise various fields such as computer science and cognitive science. CC systems may be able to learn at scale, reason purposefully, and interact naturally with humans. Self-teaching algorithms, which may use data mining, visual recognition, and / or natural language processing, may enable CC systems to solve problems and optimize human processes.

[0205] The output of a machine learning training process may be a model for predicting outcomes for new data sets. For example, a linear regression learning algorithm may have a cost function that can minimize the prediction error of a linear prediction function during the training process by adjusting the coefficients and constants of the linear prediction function. If a minimum value can be reached, the linear prediction function with the adjusted coefficients may be considered trained and constitute the model produced by the training process. For example, a neural network (NN) algorithm for classification (e.g., a multilayer perceptron (MLP)) may include a hypothesis function represented by a network of layers of nodes that are assigned biases and interconnected with weighted connections. The hypothesis function may also be a nonlinear function (e.g., a highly nonlinear function) that may include linear and logistic functions nested together, with an outermost layer consisting of one or more logistic functions. The NN algorithm may include a cost function for minimizing classification error by adjusting biases and weights through a process of feedforward propagation and backpropagation. If a global minimum can be reached, the optimized hypothesis function with its adjusted bias and weight layers may be considered trained and constitute the model that the training process produced.

[0206] Data aggregation may be performed for machine learning as a first stage in a machine learning lifecycle. Data aggregation may include steps such as identifying various data sources, collecting data from the data sources, and integrating the data. For example, to train a machine learning model for predicting surgical complications and / or post-surgical recovery rates, data sources including pre-surgical data such as a patient's medical condition and biomarker measurement data may be identified. Such data sources may be a patient's electronic medical record (EMR), a computing system that stores the patient's pre-surgical biomarker measurement data, and / or other similar data stores. Data from such data sources may be retrieved and stored in a central location for further processing in the machine learning lifecycle. Data from such data sources may be linked (e.g., logically linked) and accessed as if they were centrally stored. Surgical data and / or post-surgical data may be similarly identified and collected. Furthermore, the collected data may be integrated. In examples, a patient's pre-surgical medical record data, pre-surgical biomarker measurement data, pre-surgical data, surgical data, and / or post-surgical data may be combined into a patient record, which may be an EMR.

[0207] Data preparation can be performed for machine learning as another stage of the machine learning lifecycle. Data preparation can include data preprocessing steps such as data formatting, data cleaning, and data sampling. For example, collected data may not be in a data format suitable for training a model. Such data records can be converted into a flat file format for model training. Such data can be mapped to numerical values ​​for model training. Such identifying data can be removed before model training. For example, identifying data can be removed for privacy reasons. As another example, data can be removed because there may be more available data than can be used for model training. In such cases, a subset of the available data can be randomly sampled and selected for model training, and the remainder can be discarded.

[0208] Data preparation may include data transformation operations (e.g., after preprocessing), such as scaling and aggregation. For example, the preprocessed data may include data values ​​at various scales. These values ​​may be scaled up or down, e.g., to be between 0 and 1, for model training. For example, the preprocessed data may include data values ​​that become more meaningful when aggregated.

[0209] Model training may be another aspect of the machine learning life cycle. The model training process described herein may depend on the machine learning algorithm used. A model may be considered suitably trained after it has been trained, cross-validated, and tested. Thus, a dataset from the data preparation stage (e.g., an input dataset) may be divided into a training dataset (e.g., 60% of the input dataset), a validation dataset (e.g., 20% of the input dataset), and a test dataset (e.g., 20% of the input dataset). After a model is trained on the training dataset, it may be run on the validation dataset to reduce overfitting. If the model's accuracy is increasing, but decreases when run on the validation dataset, this may indicate an overfitting problem. The test dataset may be used to test the accuracy of the final model to determine whether it is ready for deployment or whether more training may be required.

[0210] Model deployment can be another aspect of the machine learning lifecycle. Models may be deployed as part of a standalone computer program. Models may be deployed as part of a larger computing system. Models may be deployed with model performance parameters. Such performance parameters may monitor model accuracy as it is used to make predictions on a running dataset. For example, such parameters may track false positives and false positives of a classification model. Such parameters may further store false positives and false positives for further processing to improve the accuracy of the model.

[0211] Model updates after deployment may be another aspect of the machine learning cycle. For example, a deployed model may be updated as false positives and / or as false positives are predicted on the production data. In one example, for an MLP model deployed for classification, when a false positive occurs, the deployed MLP model may be updated to increase the probability cutoff for predicting a positive to reduce the false positives. In one example, for an MLP model deployed for classification, when a false negative occurs, the deployed MLP model may be updated to decrease the probability cutoff for predicting a positive to reduce the false negatives. In one example, for an MLP model deployed for classification of surgical complications, when both false positives and false negatives occur, the deployed MLP model may be updated to decrease the probability cutoff for predicting a positive to reduce the false negatives, as predicting a false positive may be less serious than a false negative.

[0212] For example, the deployed model may be updated as more live production data becomes available as training data. In such cases, the deployed model may be further trained, validated, and tested using such additional live production data. In one example, the updated biases and weights of the further trained MLP model may update the biases and weights of the deployed MLP model. Those skilled in the art will recognize that post-deployment model updates may not be a one-time occurrence, but may occur as frequently as is suitable to improve the accuracy of the deployed model.

[0213] ML techniques may be used independently of each other or in combination. Different problems and / or datasets may benefit from using different ML techniques (e.g., combinations of ML techniques). Different training types for models may be more suitable for particular problems and / or datasets. The optimal algorithm (e.g., combinations of ML techniques) and / or training type may be determined for a particular use, problem, and / or dataset. For example, processes may be performed to select one or more of the following: select a data reduction type, select a model and / or algorithm configuration, determine the location of data reduction, determine the efficiency of the reduction and / or results, and / or other.

[0214] For example, an ML technique and / or combination of ML techniques may be determined for a particular problem and / or use case. Multiple data reduction and / or data analysis processes may be performed to determine accuracy, efficiency, and / or compatibility associated with a dataset. For example, a first ML technique (e.g., a first set of combined ML techniques) may be used on a dataset to perform data reduction and / or data analysis. The first ML technique may generate a first output. Similarly, a second ML technique (e.g., a second set of combined ML techniques) may be used on a dataset (e.g., the same dataset) to perform data reduction and / or data analysis. The second ML technique may generate a second output. The first output may be compared to the second output to determine which ML technique produced a more desirable result (e.g., a more efficient result, a more accurate result). Multiple ML techniques may be compared on the same dataset to determine the optimal ML technique to use on future similar datasets and / or problems.

[0215] In an example, in a medical context, a surgeon or medical professional may provide feedback to the ML technique and / or model used on the dataset. The surgeon may input the feedback into the weighted results of the ML model. The feedback may be used as input by the model to determine reduction methods for future analysis.

[0216] In examples, a data analysis method (e.g., an ML technique to be used in the data analysis method) may be determined based on the dataset itself. For example, the origin of the data may influence the type of data analysis method to be used for the dataset. Available system resources may be used to determine the data analysis method to be used for a given dataset. The magnitude of the data may be considered, for example, in determining the data analysis method. For example, the need for an external dataset for local processing levels or magnitude of operational response may be considered (e.g., small device changes may be made using local data, while large device operational changes may require global compilation and validation).

[0217] Such ML techniques may be applied to surgical information (e.g., combining the information flows of the surgical information in Figures 7A-D) to generate useful ML models.

[0218] A computing device, such as a surgical hub, may configure an operating range (e.g., an acceptable operating range) associated with a surgical device. The operating range may be or may include an upper safe operating range and a lower safe operating range of an acceptable input for controlling the surgical device. The computing device may determine an operating range for controlling the surgical device for a surgical step associated with a surgical procedure. The computing device may analyze data (e.g., collected data associated with operating ranges for surgical steps performed by one or more healthcare professionals (HCPs)). The computing device may use the data to train machine learning and can provide a suitable operating range for the surgical step. The computing device can provide an operating range, e.g., an acceptable operating range, to an HCP about to perform a surgical step.

[0219] A device, such as a surgical device, may receive and / or be configured with an operating range, e.g., an acceptable operating range, to control the surgical device during a surgical step. For example, in the case of a surgical procedure in an operating room, the surgical device being used may be configured with an operating range. The operating range may have an upper range and a lower range for controlling the surgical device. As described herein, the configured operating range may be or include a predefined safe operating range.

[0220] The configured operating range and / or predefined safe operating range may be associated with the magnitude of the functional adaptation. For example, the device may be a motor-controlled surgical device. The motor-controlled surgical device may have a predetermined operating program and may have the ability to modify the operation of the motor-controlled surgical device based on the current surgical step and / or current circumstances during the surgical procedure. The motor-controlled surgical device may be bounded by an operating range, a predefined safe operating range, and / or a window of adjustment. For example, the motor-controlled surgical device may allow a change in device operation that is within the operating range, the predefined safe operating range, and / or the window of adjustment. If the motor-controlled surgical device determines that the change in device operation is outside the operating range, the predefined safe operating range, and / or the window of adjustment, the motor-controlled surgical device may prevent the change in operation. The motor-controlled surgical device may send an alert (e.g., an alert message) to the HCP.

[0221] The device may have different operating ranges, predefined safe operating ranges, and / or adjustment windows. For example, the device may have a larger range adjustment for the operating range, predefined safe operating range, and / or adjustment window for an HCP-initiated update and / or after receiving an acknowledgment from the HCP. If the adjustment is generated based on an ML model (e.g., using an ML algorithm), the device may have a smaller range adjustment (e.g., smaller than HCP-involved control).

[0222] In an example, a device may collect operational data. The device can use the collected operational data to train an ML model. The device can generate configuration information associated with an operating range, a predefined safe operating range, and / or a window of adjustment based on the ML training model. Additionally and / or alternatively, one or more other devices may receive aggregate data associated with the surgical device. The one or more other devices can use the aggregate data to train an ML model. The device can use the ML model to generate configuration information associated with an allowable operating range, an allowable predefined safe operating range, and / or a window of adjustment.

[0223] In an example, a device can receive configuration information from a computing device, such as a surgical hub. For example, the computing device can send configuration information to the device, including an operating range, a predefined safe operating range, and / or a window of adjustment information. Based on the configuration information, the device can operate within the operating range, the predefined safe operating range, and / or the window of adjustment.

[0224] In an example, the device may determine and / or receive a determination from a computing device that an increased risk of resulting in secondary damage is imminent. In an example, the device may determine that a patient's condition has changed (e.g., suddenly changed) and / or whether an emergency has occurred. Based on the determination, the device may have adaptive configuration information associated with an operating range, a predefined safe operating range, and / or a window of adjustment. For example, the device may enable a larger window of operating range, a predefined safe operating range, and / or a window of adjustment if the patient's condition has changed and / or an emergency has occurred.

[0225] A device, such as a surgical device, can receive and / or be configured with an allowable operating range for a surgical procedure, for example, based on a trained ML model. For example, a surgical device can receive and / or be configured with an allowable operating range to control the device based on a trained ML model. The allowable operating range may be generated based on and / or using data from the trained ML model. The allowable operating range can be changed (e.g., adaptively changed) based on data available to the device (e.g., a computing device) and / or information available from the trained ML model (e.g., generated using an ML process and / or ML algorithm). The allowable operating range may be used to achieve a predetermined and / or estimated allowable control range for controlling the device during a surgical procedure, for example, based on information from the trained ML model.

[0226] The data used for the acceptable operating range may be from an ML training model. For example, the data from the ML training model may be based on data associated with at least one of the patient, the HCP, the surgical device, the current surgical procedure, the risks involved in the surgical procedure, user input, the magnitude of risk of failure, the risk of predicted and / or unexpected outcomes, etc.

[0227] In examples, the data associated with the patient may be or may include body mass index (BMI), height, weight, medical history, etc. In examples, the data associated with the HCP may be or may include experience, such as number of times performing a surgical procedure, success and / or failure rates, preferred settings for using the device, tendency to adjust device configurations, and / or data associated with success rates, data of other HCPs performing the same surgical procedure, etc.

[0228] ML processes and / or algorithms can be used to analyze data such as those described herein. For example, a device may use the data to train an ML model and provide acceptable operating ranges. The acceptable operating ranges generated by the trained ML model and / or algorithm can limit control of the device based on, for example, the frequency, success rate, magnitude of changes made, etc. For example, the acceptable operating ranges can be used to prevent cascading effects (e.g., unintentionally causing large effects and / or self-propagating problems) due to changes in controlling the device.

[0229] In an example, a device may be configured with an estimated allowable operating range for controlling the device. For example, a surgical device may receive an estimated allowable operating range for controlling the surgical device for a surgical procedure externally (e.g., from a computing device, a surgical hub, a cloud network, etc.). In an example, the device may configure and / or use an ML algorithm, for example, as described herein, to determine the estimated allowable operating range for controlling the device.

[0230] In an example, a device may configure an ML algorithm to analyze data, such as operational data. Based on the analysis of the data, the device may use the data to train an ML model. The device may use data from the ML model to adjust (e.g., automatically adjust) the behavior of a control algorithm operation. The control algorithm operation may be used and / or configured to achieve an acceptable operating range as described herein. For example, a device may configure an ML algorithm to adjust the behavior of future control algorithm operation to achieve an acceptable operating range based on patterns determined from the data. A device may configure an ML algorithm to have magnitude and / or frequency limits for adjustments. In an example, a device may configure an ML algorithm to have fixed limits for adjustments. For example, a device may configure an ML algorithm to allow no more than two adjustments per week, no more than a 5% adjustment up, no more than a 10% adjustment slow down, etc. A device may configure an ML algorithm to limit adjustments based on aspects of the surgical procedure, such as risk, overall benefit, or issues with user interface operation. In an example, a device may configure an ML algorithm to have adjustable limits (e.g., adaptive limits) based on aspects of the surgical procedure, as described herein.

[0231] In an example, a device may configure an ML algorithm to limit adjustments based on the effectiveness and / or frequency of previous adjustments. For example, if the device determines that one or more large magnitude adjustments have been made in the past (e.g., recently), the device may configure the ML algorithm to limit adjustments (e.g., one or more future adjustments). The device may reduce the potential impact of a previous adjustment for a predetermined amount of time based on the adjustment limit. For example, the adjustment limit may limit the harmful magnitude of a previous adjustment, the type and / or timing of future adjustments, etc. The adjustment limit may enable improvements and / or make larger directional adjustments, for example, using previous adjustments.

[0232] In an example, a device may configure an ML algorithm to limit adjustments based on historical adaptations. For example, the device may be configured to limit adjustments based on the user, the surgical procedure, historical device usage data, etc. The device may configure an ML algorithm to compare an output (e.g., the device's actual performance using a configured acceptable operating range) with one or more similar previous outputs. Based on the comparison, the device determines that the current output is within normal limits (e.g., thresholds and / or acceptable operating limits). For example, for a surgical knife device, the device may be configured (e.g., initially configured) so that the acceptable operating range may be 30 mm / s. The device may compare the acceptable operating range of 30 mm / s with one or more previous power change recommendations for the same surgical knife device used in the same surgical procedure. For example, the device may compare the acceptable operating range before displaying the acceptable operating range to the HCP. The device may compare previous power change recommendations from a local database, an edge and / or fog network, a cloud network, etc. The device may perform a check (e.g., additional check) against the acceptable operating range based on the comparison. The device can adjust and / or compensate the allowable range of motion based on the comparison and present the adjusted / compensated allowable range of motion to the HCP. For example, as described herein, a device can be configured (e.g., initially configured) with an allowable range of motion of 30 mm / s to control a surgical knife. Based on a comparison with historical data associated with the device, the patient, and the target action, the device can determine that the allowable range of motion may be adjusted to, for example, 18 mm / s, to have a higher success rate.

[0233] FIG. 9 illustrates a flow diagram 50900 of a device, such as a computing device, determining an allowable operating range for controlling a surgical device. As illustrated at 50902, a computing device, such as a surgical hub, can receive surgical data. The surgical data may be or include data and / or information associated with a surgical procedure. For example, the surgical data may be associated with and / or include surgical information (e.g., with respect to FIGS. 7A-7D ). In an example, the surgical data may be or include at least one of patient information, HCP information, surgical procedure information, information associated with a surgical device used in a surgical procedure, etc.

[0234] As described herein, patient information may be or may include BMI, weight, height, blood type, medical history, scans, test results, etc. HCP information may be or may include experience associated with the HCP, expertise associated with the HCP, the number of times the HCP has performed the current surgical procedure, the HCP's preferred settings, etc. Surgical procedure information may be or may include one or more surgical procedures associated with the surgery, one or more surgical devices associated with the surgery, patient information associated with the surgery, one or more HCPs associated with the surgery, etc. Information associated with a surgical device may be or may include the manufacturer, usage history (e.g., number of failures associated with the device), service history of the surgical device, battery level, etc.

[0235] As illustrated in 50904, the computing device can identify surgical devices used in the surgical procedure and / or surgical steps performed in the surgical procedure. For example, the computing device can identify surgical devices used in the surgical procedure and / or surgical steps performed in the surgical procedure based on the surgical procedure data.

[0236] The computing device can determine an allowable operating range associated with a surgical device used in a surgical procedure. For example, as illustrated at 50906, the computing device can determine the allowable operating range based on at least one of an identified surgical device (e.g., as illustrated at 50904), an identified surgical step (e.g., as illustrated at 50904), and / or received surgical data (e.g., as illustrated at 50902). As described herein, the computing device can train an ML model (e.g., using an ML algorithm and / or ML process) using the surgical data, the identified surgical device, and / or the identified surgical step. The computing device can determine the allowable operating range based on data associated with the ML model. For example, the computing device can analyze usage history associated with a surgical device for a surgical step performed by the HCP based on data from the ML training model. The trained ML model data can provide a range of control inputs that have a high success rate for the current surgical step. The computing device can provide an allowable operating range for controlling the surgical device for the surgical step based on the analysis.Providing an acceptable operating range for controlling the surgical devices disclosed herein is in accordance with U.S. Patent Application No. 16 / 209,423, filed December 4, 2018, entitled "Method Of Compressing Tissue Within A Stapling Device And Simultaneously Displaying The Location Of The Tissue Within The Jaws," U.S. Patent No. 10,881,399, filed January 5, 2021, entitled "Techniques For Adaptive Control Of Motor Velocity Of A Surgical Stapling And Cutting Instrument," U.S. Patent Application No. 16 / 458,103, filed June 30, 2019, entitled "Packaging For A Replaceable Component Of A Surgical Stapling System," and U.S. Patent Application No. 16 / 458,103, filed August 27, 2019, entitled "Control Of Advancement Rate And Application Force Based On Measured Force," and ... August 27, 2019, entitled "Control Of Advancement Rate And Application Force Based On Measured Force," and U.S. Patent Application No. 16 / 458,103, filed December 4, 2018, entitled "Method Of Compressing Tissue Within A Stapling Device And Simultaneously Displaying The Location Of The Tissue Within The Jaws," and U.S. Patent Application No. 16 / 458,103, filed December 4, 2018, entitled "Control No. 10,390,895, entitled "Electrosurgical Forces," issued on March 2, 2021; U.S. Patent Application No. 16 / 209,458, entitled "Method For Smart Energy Device Infrastructure," filed on December 4, 2018; and U.S. Patent No. 10,842,523, entitled "Modular Battery Powered Handheld Surgical Instrument And Methods Therefor," filed on November 24, 2020, each of which is incorporated by reference in its entirety.

[0237] As described herein, the computing device can use and / or be configured to use data to train an ML model, and the computing device can utilize data from the trained ML model to determine an acceptable operating range. Surgical information associated with the same surgical procedure using the same surgical device performed by other HCPs (e.g., 726, 727, 762, 766 as described herein with respect to FIGS. 7A-7D ) can be configured as one or more inputs to the ML model. The inputs can be used to train the ML model, for example, using one or more training methods suitable for training the surgical information. For example, the computing device can use the surgical information to train the ML model using supervised learning, such as a supervised learning algorithm as described herein (e.g., with respect to FIGS. 8A-8B ). The output of the ML training model (e.g., a supervised learning algorithm) can be or include information suitable for the computing device to determine an acceptable operating range for the surgical procedure using the surgical device, as described herein. For example, the output of the ML training model may be or may include a labeled output that provides supervisory feedback that provides an acceptable operating range for a surgical procedure using a surgical device.

[0238] As shown in 50908, the computing device may receive an adjustment input configuration. The adjustment input configuration may be configured to control a surgical device for a surgical step. In an example, the adjustment input configuration may be an input for increasing / decreasing a motor associated with a surgical stapler. In an example, the adjustment input configuration may be an input for increasing / decreasing a current associated with a surgical cutter and / or a cauterization device. The adjustment input configuration may be generated by an ML training model. As described herein, the computing device may use and / or be configured to use data from the ML training model to generate / receive the adjustment input configuration. The computing device may use appropriate data and / or surgical information (e.g., with respect to FIGS. 7A-7D ) as input to train the ML model. For example, the computing device may train the ML model using surgical data associated with surgical devices used by other HCPs for the same surgical step. The computing device may use the input surgical data associated with the surgical device to train the ML model. As described herein, the computing device may use one or more suitable training methods to train the ML model. For example, the computing device can use surgical data associated with the surgical device to train an ML model using supervised learning, such as a supervised learning algorithm as described herein (e.g., with respect to FIGS. 8A-8B). The output of the ML training model (e.g., supervised learning) can be or include an adjustment input configuration appropriate for the current surgical step. For example, the output of the ML data can be configured to provide an adjustment input configuration for controlling the surgical device for the current surgical step. The output of the ML model can provide an adjustment input configuration for increasing or decreasing the control input of the surgical device.

[0239] The computing device may determine that the adjusted input configuration is outside the determined acceptable operating range, as shown at 50910. The computing device may determine that the adjusted input configuration is within the determined acceptable operating range.

[0240] As illustrated in 50912, if the computing device determines that the adjustment input configuration is outside the determined acceptable operating range, the computing device can prevent the adjustment input configuration from being adjusted to control the surgical device. The computing device may send an alert (e.g., an alert message) to the HCP. In an example, the computing device can send an alert message indicating that the adjustment input configuration is outside the acceptable operating range. In an example, the computing device can send an alert message indicating that the adjustment input configuration is outside the acceptable operating range. The computing device can send an alert message indicating a risk associated with adjusting the input to an adjustment input configuration that is outside the acceptable operating range. In an example, the computing device can send a message indicating that the adjustment input configuration is within the acceptable operating range.

[0241] The computing device may determine the origin of the adjusted input configuration. For example, the computing device can determine whether the adjusted input configuration is from an HCP, e.g., a surgeon using a surgical device, or whether it is generated by ML data, e.g., by another computing device, a remote server, the cloud, etc.

[0242] In an example, if the computing device determines that the adjustment input configuration is from an HCP, the computing device can send a message to the HCP. The message can include, for example, whether to adjust the input for controlling the surgical device using the adjustment input configuration that is outside the acceptable operating range. The computing device can receive a feedback message and / or response from the HCP. For example, the feedback message and / or response can confirm that the adjustment input configuration should be used (e.g., despite being outside the acceptable operating range). As described herein, the computing device may request HCP confirmation justification (e.g., to use the adjustment input configuration that is outside the acceptable operating range). For example, the computing device can ask / request additional information about the adjustment input configuration (e.g., a change in patient condition, a malfunction of the surgical device, switched and / or incorrect scan data, incorrect patient information, etc.).

[0243] The computing device may enable the adjustment input configuration as an input for controlling the surgical device for the current surgical step, for example, based on a feedback message, a response from the HCP, and / or justification. The computing device may send a request message to the HCP. The request message may indicate, for example, whether the allowable operating range needs to be modified based on the adjustment input configuration. In an example, the HCP may indicate that the adjustment input configuration is temporary (e.g., one time) and that the allowable operating range does not require modification. In an example, the HCP may indicate that the adjustment input configuration is permanent and that the allowable operating range requires modification, for example, based on the adjustment input configuration and / or current operating data.

[0244] In an example, if the computing device determines that an adjusted input configuration was generated by an ML model, the computing device can send ML data to the HCP. The ML data may be or include the information and / or analysis that caused the adjusted input configuration. For example, the ML data may be or include at least one of: frequency information of other HCPs using the adjusted input configuration for the current surgical step, or a success rate of a surgical procedure using the adjusted input configuration.

[0245] If the computing device determines that the adjusted input configuration is within an acceptable operating range, the computing device may, for example, use the adjusted input configuration to adjust an input controlling a surgical device for a surgical step.

[0246] 10 illustrates a flow diagram 50920 of a device, such as a surgical device, determining an allowable operating range for controlling a surgical device. As illustrated in 50922, the device, such as a surgical device, can receive surgical data. The surgical data may be or include data and / or information associated with a surgical procedure. In an example, the surgical data may be or include at least one of patient information, HCP information, surgical procedure information, etc.

[0247] As described herein, patient information may be or may include BMI, weight, height, blood type, medical history, scans, test results, etc. HCP information may be or may include experience associated with the HCP, expertise associated with the HCP, the number of times the HCP has performed the current surgical procedure, the HCP's preferred settings, etc. Surgical procedure information may be or may include one or more surgical procedures associated with the surgery, one or more surgical devices associated with the surgery, patient information associated with the surgery, one or more HCPs associated with the surgery, etc.

[0248] As illustrated in 50924, the surgical device can identify the surgical steps to be performed in the surgical procedure. For example, the surgical device can identify the surgical steps to be performed in the surgical procedure based on the surgical procedure data.

[0249] The surgical device can determine an allowable operating range for controlling a surgical device used in a surgical procedure. For example, as illustrated at 50926, the surgical device can determine the allowable operating range based on an identified surgical step (e.g., as illustrated at 50924) and / or received surgical data (e.g., as illustrated at 50922). As described herein, the surgical device can use data (e.g., the surgical step and / or the surgical data) to train an ML model. The surgical device can determine the allowable operating range using data from the ML training model. For example, the surgical device can analyze a usage history associated with the surgical device for surgical steps performed by the HCP based on data from the ML training model. The data from the ML model may be configured to provide a range of control inputs that have a high success rate for the current surgical step. The surgical device can provide an allowable operating range for controlling the surgical device for the surgical step based on the analysis.

[0250] As shown in 50928, the surgical device can receive an adjustment input configuration. The adjustment input configuration can be configured to control the surgical device for a surgical step. In an example, the adjustment input configuration can be an input to increase / decrease a motor associated with a surgical stapler. In an example, the adjustment input configuration can be an input to increase / decrease a current associated with a surgical cutter and / or a cauterization device.

[0251] The surgical device can determine that the adjustment input configuration is outside the determined allowable operating range, as shown at 50930. The surgical device can determine that the adjustment input configuration is within the determined allowable operating range.

[0252] As illustrated in 50932, if the surgical device determines that the adjustment input configuration is outside the determined acceptable operating range, the surgical device can prevent the adjustment input configuration in order to control the surgical device. The surgical device can send an alert (e.g., an alert message) to the HCP. In an example, the surgical device can send an alert message indicating that the adjustment input configuration is outside the acceptable operating range. In an example, the surgical device can send an alert message indicating that the adjustment input configuration is outside the acceptable operating range. The surgical device can send an alert message indicating the risks associated with adjusting the input to the adjustment input configuration that is outside the acceptable operating range. In an example, the surgical device can send a message indicating that the adjustment input configuration is within the acceptable operating range.

[0253] The surgical device may determine the origin of the adjusted input configuration. For example, the surgical device can determine whether the adjusted input configuration is from an HCP, e.g., a surgeon using the surgical device, or whether it is generated by an ML model, e.g., by a computing device, a remote server, the cloud, etc.

[0254] In an example, if the surgical device determines that the adjusted input configuration is from an HCP, the surgical device can send a message to the HCP. The message can include, for example, whether to adjust the input for controlling the surgical device using the adjusted input configuration that is outside of the acceptable operating range. The surgical device can receive a feedback message and / or response from the HCP. For example, the feedback message and / or response can confirm that the adjusted input configuration should be used (e.g., despite being outside of the acceptable operating range). As described herein, the surgical device may request HCP confirmation justification (e.g., to use the adjusted input configuration that is outside of the acceptable operating range). For example, the surgical device can ask / request additional information about the adjusted input configuration (e.g., a change in patient condition, a malfunction of the surgical device, switched and / or erroneous scan data, incorrect patient information, etc.).

[0255] The surgical device may enable an adjustment input configuration as an input for controlling the surgical device for the current surgical step, for example, based on a feedback message, a response from the HCP, and / or justification. The surgical device may send a request message to the HCP. The request message may indicate, for example, whether the allowable operating range needs to be modified based on the adjustment input configuration. In an example, the HCP may indicate that the adjustment input configuration is temporary (e.g., one time) and that the allowable operating range does not require modification. In an example, the HCP may indicate that the adjustment input configuration is permanent and that the allowable operating range requires modification, for example, based on the adjustment input configuration and / or current operating data.

[0256] In examples, if the surgical device determines that the adjusted input configuration was generated by an ML model, the surgical device can transmit ML data to the HCP. The ML data may be or include the information and / or analysis that caused the adjusted input configuration. For example, the ML data may be or include at least one of: frequency information of other HCPs using the adjusted input configuration for the current surgical step, or a success rate of a surgical procedure using the adjusted input configuration.

[0257] If the surgical device determines that the adjustment input configuration is within an acceptable operating range, the surgical device can, for example, use the adjustment input configuration to adjust the input controlling the surgical device for the surgical step.

[0258] 11 illustrates a computing device that determines an allowable operating range associated with a surgical device. For example, as described herein, a surgical hub 50944 can determine an allowable operating range 50942 associated with a surgical stapler 50940. The allowable operating range 50942 can be an allowable input range for controlling the surgical stapler 50940. As described herein, a computing device such as the surgical hub 50944 can configure data described herein to train an ML model and use data from the ML model to determine the allowable operating range 50942, for example, based on the surgical stapler 50942, the surgical step, and / or surgical procedure data.

[0259] 12 illustrates a computing device adjusting an allowable operating range associated with a surgical device based on an adjustment input configuration from a medical professional. For example, a surgical hub 50954 can receive an adjustment input configuration 50952 from a surgical stapler 50950. As described herein, the surgical hub 50954 can determine whether the adjustment input configuration 50952 is outside of an allowable operating range 50956. If the surgical hub 50954 determines that the adjustment input configuration 50952 is outside of the allowable operating range 50956, the surgical hub 50954 can determine whether the adjustment input configuration 50952 can be initiated by an HCP, such as a surgeon using the device. If the surgical hub 50954 determines that the adjustment input configuration 50952 was initiated by a surgeon, the surgical hub 50954 can adjust the allowable operating range 50956. For example, the surgical hub 50954 can configure a modified allowable operating range 50958 that extends from the allowable operating range 50956 to take into account the adjusted input configuration 50952 from the surgeon.

[0260] 13 illustrates a computing device receiving an adjustment input configuration that is outside of an allowable operating range, the adjustment input configuration being from an ML model (e.g., an ML training model using an ML process and / or ML algorithm as described herein). For example, a surgical hub 50964 may receive an adjustment input configuration 50962 from a surgical stapler 50960. As described herein, the surgical hub 50964 may determine whether the adjustment input configuration 50962 is outside of an allowable operating range 50966. If the surgical hub 50964 determines that the adjustment input configuration 50962 is outside of an allowable operating range 50966, the surgical hub 50964 may determine whether the adjustment input configuration 50962 can be initiated by the ML training model. As described herein, if the surgical hub 50964 determines that the adjustment input configuration 50962 is outside the allowable operating range 50966 based on the ML training model, the surgical hub 50964 can prevent the adjustment input configuration 50962. For example, the surgical hub 50964 can configure the surgical stapler 50960 to maintain the allowable operating range 50966 and prevent the adjustment input configuration 50962. In an example, the surgical hub 50964 can verify that the allowable operating range 50966 has not changed based on the adjustment input configuration 50962, for example, and resend the allowable operating range 50966 to the surgical stapler 50960.

[0261] In an example, a device can receive and / or be configured with an allowable operating range as described herein. In an example, a device can determine an allowable operating range using, for example, data from an ML training model as described herein. A device, such as a surgical device, can receive control input from an HCP and / or an intermediate device of an original equipment manufacturer (OEM) to control the surgical device in a surgical procedure. The device can include a process that determines the received / configured and / or determined allowable operating range and / or a user can allow, deny, limit, and / or adjust.

[0262] The device may receive a control input to control the device. For example, a surgical device may receive a control input to control the device from an HCP performing a current surgical step. As described herein, the surgical device may be configured with and / or determined to have an acceptable operating range for controlling the surgical device. The surgical device can determine whether the control input from the HCP is within or outside the acceptable operating range. If the surgical device determines the control input is within the acceptable operating range, the surgical device can allow the control input to control the device. If the surgical device determines the control input is outside the acceptable operating range, the surgical device can prevent the control input from controlling the device. The surgical device can send an alert (e.g., an alert message) to the HCP indicating that the provided control input is outside the acceptable operating range.

[0263] The surgical device may receive feedback from the HCP. The feedback from the HCP may indicate an acknowledgment from the HCP that the control input is outside of an acceptable operating range. The feedback may include an acknowledgment that the control input (e.g., is outside of an acceptable operating range) is acceptable and / or the acceptable operating range may be modified based on the HCP's acknowledgment.

[0264] In an example, the HCP may receive an acceptable operating range, for example, before providing a control input. As described herein, the HCP may accept the configured acceptable operating range. In an example, the HCP may reject the configured acceptable operating range. The HCP may provide an updated operating range. The device may be configured to use the updated operating range, configure data to train an ML model, and provide the updated acceptable operating range.

[0265] In an example, a device may use third-party validation. For example, the device may use third-party validation to enable one or more successive algorithm adjustments. The device may display data from an ML training model, results (e.g., relationships, recommendations, control system changes, etc.), etc. Additionally and / or alternatively, the device may display the data (e.g., a reduced configuration of the data) and allow a third party (e.g., a third-party device) to determine whether use of the results is valid and / or warranted.

[0266] In an example, the device may show results (e.g., data) of output parameters recommended by the ML training model to complete a task, such as an acceptable operating range. For example, the device may show a minimum drive time, a step shift in the output parameters, one or more adjustments made by the HCP based on the HCP's experience, visual, visual and / or device feedback, sensory feedback, etc. This example may enable a third party (e.g., a third-party device) and / or the HCP to override, reduce, or eliminate suggested adjustment data (e.g., an acceptable operating range) from the ML model. Feedback from the third party and / or user may be provided to the device via a display, such as a screen, associated with the device and / or via a computing device, such as a hub and / or a display associated with the hub.

[0267] The device may exchange information with the HCP if the device determines that an adjustment from the HCP exceeds a preconfigured adjustment (e.g., a large adjustment). The device may query the HCP, supervisor, etc. for confirmation of the adjustment, as described herein. The device may request justification for such an adjustment. In an example, during a surgical procedure, the device may provide an acceptable operating range for the device based on the device moving to the right. The device may receive an adjustment (e.g., a large adjustment) from the HCP that exceeds a preconfigured threshold. The device may request justification from the HCP for the adjustment. The device may ask the HCP whether the device is moving to the left (e.g., instead of the recommended right). If the device receives a positive response from the HCP, the device may provide an updated acceptable operating range (e.g., based on the entire position being flipped from right to left). In an example, the device may query the HCP whether a switch in operation (e.g., from right to left) is a one-time occurrence, or whether the procedure and / or acceptable operating range needs to be updated (e.g., before proceeding to the next step).

[0268] In an example, the device may request justification from the HCP if the HCP makes an adjustment greater than a preconfigured threshold, as described herein. The device may ask if scan data was improperly tagged and / or entered into the device. The device may inquire if right and left were switched and the change was not made in time (e.g., before the data was entered into the device) as being mislabeled. The device may ask to confirm if the patient had a previous procedure that was recorded, entered, and / or forgotten. The device may ask if one or more markers that should be present are missing for the patient. For example, the device may ask to confirm that the patient is missing a kidney and / or that the kidney is being used as a reference for another procedure. The device may ask if the wrong patient is on the operating table. The device may indicate that the surgical plan and / or scan do not match the input to the device. The device may request confirmation from the HCP and / or ask whether to continue or adjust the surgical plan based on the input.

[0269] The device may determine that a potential input from the HCP may have catastrophic consequences. The device may notify the HCP about the possibility of a catastrophic consequence and / or indicate that the potential input may be outside of a normal operating range. The device may query whether the device determined that a change in the patient's condition occurred during the scan and / or assessment and when a surgical procedure is being performed.

[0270] The device may request HCP feedback if the surgical treatment plan needs to be updated. In an example, the device may determine, based on an MRI scan, that a patient has a meniscus tear and provide a surgical plan and / or allowable range of motion for meniscus tear repair. After the scan and / or during surgery, the device may determine that the remaining portion of the meniscus is completely torn. The device may confirm to the HCP that the surgical plan and / or allowable range of motion needs to be updated.

[0271] In an example, the device may provide a surgical plan and / or acceptable operating ranges for a patient's gallbladder surgery. During the surgery, based on input, the device may determine that the patient has cancer. The device may ask the HCP to confirm that the surgical plan and / or acceptable operating ranges need to be updated.

[0272] The device may configure data from the ML model to determine adjustment weightings (e.g., adjustments to an acceptable operating range). For example, the device may configure data from the ML model to determine adjustment weightings based on one or more feedback from an HCP and / or a third party, as described herein. The adjustment weightings may be based on temporal aspects and / or frequency. In an example, if the device determines that one or more adjustments result in improvement, the device may increase the frequency of the adjustments and / or allow less time for the adjustments. In an example, if the device determines that one or more adjustments result in adverse and / or unsuccessful outcomes, the device may decrease the frequency of the adjustments and / or allow more time for the adjustments.

[0273] The device can determine the weighting of the adjustment (e.g., adjustment to the acceptable operating range) based on the type of change. In an example, the device can determine that an acceptable operating range for a treatment step (e.g., less critical and / or less life-threatening) may be needed. The device may provide an adjusted acceptable operating range (e.g., more frequent and / or fewer checklists). In an example, the device can determine an acceptable operating range for a critical step and / or a critical treatment (e.g., a treatment that may include risk and / or non-treatment steps). The device may require more information, one or more confirmation steps, and / or more user confirmation before providing the adjusted acceptable operating range.

[0274] The device may utilize weighted responses, for example, to control the magnitude of algorithm adaptation. For example, the device may compile and / or aggregate one or more results (e.g., ideal results). The device may use the compiled and / or aggregated results to have a weighted and / or predefined aggregate list. The device can combine the weighted and / or predefined aggregate list with current data (e.g., a portion of the current data). The device can request verification (e.g., remote verification) and / or validation. The device may require verification and / or validation and can upload data to a cloud and / or remote server for review and / or combination with other system results (e.g., results from other locations and / or facilities).

[0275] A device, such as a computing device, can collect data in a remote server and / or cloud. The computing device can use the ML model to provide conclusions and implement global device operation changes (e.g., global acceptable operating ranges). The global device operation changes (e.g., global acceptable operating ranges) can control the device (e.g., one or more facilities using the device). The global device operation changes (e.g., global acceptable operating ranges) can validate the acceptable operating ranges for device recommendations and / or implement one or more proposed changes in a controlled and / or global manner. The global device operation changes can prevent inadvertent and / or uncontrolled changes to local devices (e.g., local operation and / or local environment).

[0276] The computing device may compare a proposed change (e.g., a global device operation change) with conflicting changes proposed for, e.g., related devices, related steps, related techniques, etc. The computing device may, for example, prevent a constant cycle of changes based on the comparison and / or changes from related devices.

[0277] In an example, the device can configure data from an ML training model and process the output based on the collected parameters. When the device determines that the output is complete, the device may have one or more parameters (e.g., one or more additional parameters). The device can go back and weight the one or more parameters (e.g., including one or more additional parameters) and / or the defined parameters. The device may modify the output based on the weighting coefficients. For example, the device can include a disease state of the tissue and / or an identification of a blood vessel and / or artery. The disease state and / or identification can modify and / or weight the output to compensate for the parameters, for example.

[0278] The device may determine thresholds (e.g., threshold functions) that limit feasible adjustment boundaries. The device may determine threshold boundaries for the magnitude of algorithmic changes. For example, the device may configure a functional algorithm to determine one or more boundaries of a functional range. The device may determine whether an adjustment is out of bounds or within bounds. For example, the device may determine whether an adjustment is out of bounds based on data (e.g., data entering and leaving the ML model). The ML model may use patient information such as BMI, height, and weight. The ML model may use the patient information to generate a predictive model configuration. For example, the ML model may be configured based on patient information before a surgical procedure to predict what tissue characteristics are, what functional range the device is suitable for operating in, etc. During the surgical procedure, the device may compare device performance data with the predictive modeling configuration. The device may determine whether drift and / or error exists in the predictive model and / or procedure. Based on a determination that drift and / or error exists, the device may adjust the predictive model to acceptable limits.

[0279] In an example, the device may determine whether the patient's tissue is more difficult to traverse even once or more difficult to traverse consistently. In an example, the device may determine whether a problem exists with the device and / or sensors. In an example, the device may determine whether a problem exists with a predictive model of the patient's tissue composition. Based on this determination, the device may provide an alert to the HCP. For example, the device may present an alert to the HCP that a different cartridge is recommended for the current surgical procedure (e.g., relative to the cartridge being used in the current surgery).

[0280] The device may use historical data to predict one or more devices that will be most effective for one or more surgical steps. The device may provide and / or integrate product recommendations to purchasing and hospital inventory management systems based on the historical data. For example, the device may ensure that sufficient cartridges (e.g., blue, white, gold, and / or similar cartridges) are in stock for the hospital. The device may send an alert (e.g., an alert message) if one or more cartridges are low in stock. The device may make adjustments if a supply chain interruption occurs. The device may also modify one or more recommendations, for example, to more effectively allocate resources. For example, if procedure A uses blue and / or gold cartridges and procedure B is more effective with gold cartridges, the device may recommend using blue cartridges for procedure A (e.g., instead of using gold cartridges). The device may integrate and / or provide product recommendations to higher levels of management, for example, for an entire hospital network, to more effectively deploy resources and / or supplies, if necessary.

[0281] When using modern surgical devices to perform surgical steps in a surgical procedure, there are typically many possible operating configurations of the surgical device relative to its various operating inputs. For example, a surgical device may have an output power supply of 0-50 W, a power supply time of 10-15 seconds, and a maximum temperature setting of 45° C. Determining how to adjust the operating inputs for a surgical step is not easy because it may vary depending on the patient undergoing the surgical procedure, the operator performing the surgical procedure, and the surgical device being used to perform the surgical procedure.

[0282] Typically, operators use their experience from previous surgical procedures to tailor operational inputs for a particular patient using a particular device. However, reliance on operator experience leaves open the possibility that operational inputs may not be appropriate for the patient or surgical device, potentially leading to poor surgical outcomes for the patient. Furthermore, operational inputs may be accidentally set to unsafe levels by the operator. However, no mechanism exists to prevent inappropriate or unsafe operational inputs from being used in a surgical device for a surgical step.

[0283] As used herein, the terms "operator" and "healthcare professional (HCP)" are interchangeable.

[0284] As used herein, the terms "machine learning process" and "machine learning model" are interchangeable. As used herein, the terms "machine learning technique" and "machine learning algorithm" are interchangeable. When the term "machine learning" is used herein, it refers to either a machine learning model or a machine learning algorithm, as will be clear from the context in which the term is used.

[0285] As used herein, the term "machine learning data" refers to data output from a machine learning model. As used herein, the terms "machine learning model metadata" and "machine learning data information" are interchangeable.

[0286] As used herein, the term "operating range" is defined as the range of motion inputs used to control a surgical device, preferably to perform a surgical step. The range may correspond to all possible operating configurations of the device for a particular motion input. For example, a surgical device may have an output power supply between 0 and 50 W, and therefore 0 to 50 W is the operating range of the surgical device for that output power. The operating range may have an upper limit and no lower limit. Alternatively, the operating range may have a lower limit and no upper limit. Alternatively, the operating range may have upper and lower limits that define a safe operating range. In some embodiments, the operating range may include an upper safe operating range and a lower safe operating range. The terms "predefined safe operating range" and "window of adjustment" may also be used herein to refer to the "operating range."

[0287] As used herein, the term "acceptable operating range" is defined as the range of operational inputs used to control a surgical device, preferably to perform a surgical step. The allowable operating range is a subset of all possible operational configurations of a device for a particular operational input. The allowable operating range may be based on a safe operating range for performing a particular surgical step. For example, a power supply may have an allowable operating range of 45-50 W (out of a 0-50 W operating range). The allowable operating range may be based on previous operating ranges used to perform a particular surgical step, i.e., from historical surgical data. In embodiments where the allowable operating range is based on previous operating ranges used to perform a particular surgical step, machine learning may be used to train a model for determining the allowable operating range. Similar to an operating range, the allowable operating range may have an upper limit and no lower limit. Alternatively, the allowable operating range may have a lower limit and no upper limit. Alternatively, the allowable operating range may have upper and lower limits that define a safe operating range. In some embodiments, the allowable operating range may include an upper safe operating range and a lower safe operating range.

[0288] The term “adjustment input configuration” refers to a change in the operating configuration (i.e., an adjustment to the operating configuration) of a surgical device for a particular operational input. The adjustment input configuration is within the operating range of the surgical device, i.e., one of the possible operating configurations of the device for a particular operational input. However, the adjustment input configuration may not be within the allowable operating range of the surgical device, i.e., within a subset of all possible operating configurations of the device for a particular operational input. The adjustment may be a manual adjustment (i.e., initiated by the operator) or an automatic adjustment (i.e., initiated by the device). The automatic adjustment may have one or more of: a fixed limit for the adjustment; a magnitude and / or frequency limit for the adjustment; and an adjustment based on aspects of the surgical procedure, such as surgical risk, overall benefit to the surgical outcome, or issues with user interface operation. An ML model can be trained to determine the adjustment input configuration as an input for controlling the surgical device for a surgical step using previous adjustment input configurations, i.e., from historical surgical data.

[0289] The following is a non-exhaustive list of embodiments that may or may not be claimed.

[0290] 1. A device, a processor, receiving surgical data associated with the surgical procedure, the surgical data including at least one of patient information, healthcare professional (HCP) information, or information associated with a surgical device used in the surgical procedure; Identifying a surgical device used in the surgical procedure and a surgical step associated with the surgical procedure based on the surgical procedure data; determining an allowable operating range associated with the surgical device based on the surgical device, the surgical step, and the surgical procedure data, the allowable operating range being an operating range for controlling the surgical device for the surgical step; receiving an adjustment input configuration, the adjustment input configuration configured to control a surgical device for a surgical step; Determining that the regulated input configuration is within a determined allowable operating range; Interfering with the adjustment input configuration that controls the surgical device based on a determination that the adjustment input configuration is outside of an acceptable operating range; A device that is configured to:

[0291] 2. The processor: determining whether the adjustment input configuration is from a HCP or generated by a machine learning (ML) model; sending an alert to the HCP, the alert indicating that the adjustment input configuration is outside of an acceptable operating range, the alert further indicating a risk associated with adjusting the input to the adjustment input configuration that is outside of the acceptable operating range; The device of embodiment 1, configured to perform the following:

[0292] 3. The processor: sending a message to the HCP based on a determination that the adjustment input configuration is from the HCP, the message including whether to adjust an input for controlling the surgical device using the adjustment input configuration that is outside of an acceptable operating range; receiving a feedback message from the HCP, the feedback message confirming use of the adjusted input configuration as an input for controlling the surgical device; accepting the adjusted input configuration as an input for controlling the surgical device for the surgical step based on the received feedback message; The device of embodiment 2, configured to perform the following:

[0293] 4. The processor: sending a request message to the HCP, the request message including determining a modified allowable operating range based on the adjustment input configuration; The device of embodiment 3, configured to perform the following:

[0294] 5. The processor: and transmitting ML data based on a determination that the adjusted input configuration was generated by the ML model, the ML data including at least one of frequency information of other HCPs using the adjusted input configuration for the surgical step or a success rate of the surgical procedure using the adjusted input configuration. The device of embodiment 2, configured to perform the following:

[0295] 6. The processor: Using the adjusted input configuration to adjust an input for controlling a surgical device for a surgical step based on determining that the adjusted input configuration is within an acceptable operating range. The device of embodiment 1, configured as follows:

[0296] 7. A method comprising: receiving surgical data associated with a surgical procedure, the surgical data including information associated with at least one of a patient, a healthcare professional (HCP), or a surgical device used in the surgical procedure; Identifying a surgical device used in the surgical procedure and a surgical step associated with the surgical procedure based on the surgical procedure data; determining an allowable operating range associated with the surgical device based on the surgical device, the surgical step, and the surgical procedure data, the allowable operating range being an operating range for controlling the surgical device for the surgical step; receiving an adjustment input configuration, the adjustment input configuration configured to control a surgical device for a surgical step; Determining that the regulated input configuration is within a determined allowable operating range; Interfering with the adjustment input configuration that controls the surgical device based on a determination that the adjustment input configuration is outside of an acceptable operating range; A method comprising:

[0297] 8. How to determining whether the adjustment input configuration is from a HCP or generated by a machine learning (ML) model; sending an alert to the HCP, the alert indicating that the adjustment input configuration is outside of an acceptable operating range, the alert further indicating a risk associated with adjusting the input to the adjustment input configuration that is outside of the acceptable operating range; 8. The method of embodiment 7, comprising:

[0298] 9. How to sending a message to the HCP based on a determination that the adjustment input configuration is from the HCP, the message including whether to adjust an input for controlling the surgical device using the adjustment input configuration that is outside of an acceptable operating range; receiving a feedback message from the HCP, the feedback message confirming use of the adjusted input configuration as an input for controlling the surgical device; accepting the adjustment input configuration as an input for controlling a surgical device for a surgical step; 9. The method of embodiment 8, comprising:

[0299] 10. How to sending a request message to the HCP, the request message including determining a modified allowable operating range based on the adjustment input configuration; 10. The method of embodiment 9, comprising:

[0300] 11. How to and transmitting ML data based on a determination that the adjusted input configuration was generated by the ML model, the ML data including at least one of frequency information of other HCPs using the adjusted input configuration for the surgical step or a success rate of the surgical procedure using the adjusted input configuration. 9. The method of embodiment 8, comprising:

[0301] 12. How to adjusting an input for controlling a surgical device for the surgical step using the adjustment input configuration based on determining that the adjustment input configuration is within an acceptable operating range. 8. The method of embodiment 7, comprising:

[0302] 13. A surgical device comprising: a processor, the processor comprising: receiving surgical data associated with the surgical procedure, the surgical data including at least one of patient information, healthcare professional (HCP) information, or information associated with a surgical device used in the surgical procedure; Identifying surgical steps associated with the surgical procedure based on the surgical procedure data; determining an allowable operating range associated with the surgical device based on the identified surgical step and surgical procedure data, the allowable operating range being an operating range for controlling the surgical device for the surgical step; receiving an adjustment input configuration, the adjustment input configuration configured to control a surgical device for a surgical step; Determining that the regulated input configuration is within a determined allowable operating range; Interfering with the adjustment input configuration controlling the surgical device based on a determination that the adjustment input configuration is outside of an acceptable operating range; A surgical device configured to:

[0303] 14. The processor: Transmitting an alert to the HCP, the alert indicating that the adjustment input configuration is outside of an acceptable operating range, the alert further indicating a risk associated with adjusting an input for controlling the surgical device to an adjustment input configuration that is outside of the acceptable operating range. 14. The surgical device of embodiment 13, configured to:

[0304] 15. The processor: determining that the adjustment input configuration is from an HCP; sending a message to the HCP based on a determination that the adjustment input configuration is from the HCP, the message including whether to adjust an input for controlling the surgical device using the adjustment input configuration that is outside of an acceptable operating range; 14. The surgical device of embodiment 13, configured to:

[0305] 16. The processor: receiving a feedback message from the HCP, the feedback message confirming use of the adjusted input configuration as an input for controlling the surgical device; accepting the adjusted input configuration as an input for controlling the surgical device for the surgical step based on the received feedback message; 16. The surgical device of embodiment 15, configured to:

[0306] 17. The processor: sending a request message to the HCP, the request message including determining a modified allowable operating range based on the adjustment input configuration; 16. The surgical device of embodiment 15, configured to:

[0307] 18. The processor: determining that the adjustment input configuration was generated by a machine learning (ML) model; transmitting the ML model metadata to the HCP based on a determination that the tuned input configuration was generated by the ML model; 14. The surgical device of embodiment 13, configured to:

[0308] 19. The surgical device of embodiment 18, wherein the ML data includes at least one of frequency information of other HCPs using the adjusted input configuration for the surgical step, or a success rate of a surgical procedure using the adjusted input configuration.

[0309] 20. The processor: Using the adjusted input configuration to adjust an input for controlling a surgical device for a surgical step based on determining that the adjusted input configuration is within an acceptable operating range. 14. The surgical device of embodiment 13, configured as follows:

[0310] [Embodiment] (1) A processor-implemented method, the method comprising: receiving surgical data associated with a surgical procedure, the surgical data including data associated with at least one of a patient, an operator, or a surgical device used in the surgical procedure; identifying surgical steps associated with the surgical procedure based on the surgical procedure data; determining an allowable operating range associated with the surgical device based on the surgical step and the surgical data, the allowable operating range being an operating range for controlling the surgical device for the surgical step; receiving an adjustment input configuration, the adjustment input configuration configured to control the surgical device for the surgical step; determining whether the adjustment input configuration is within the determined allowable operating range; preventing the adjustment input configuration from controlling the surgical device based on a determination that the adjustment input configuration is not within the allowable operating range; and A method comprising: (2) The method of embodiment 1, wherein determining an acceptable operating range associated with the surgical device includes inputting the surgical data into a machine learning (ML) model, wherein the ML model is trained to determine the acceptable operating range of the surgical device for performing the surgical step using past surgical data. (3) The method of embodiment 1, wherein determining the allowable operating range associated with the surgical device includes retrieving the allowable operating range of the surgical device for performing the surgical step from past surgical data. (4) identifying the surgical device to be used in the surgical procedure based on the surgical procedure data; 2. The method of claim 1, wherein determining an allowable operating range associated with the surgical device is also based on the surgical device. (5) A method according to any one of claims 1 to 4, further comprising sending an alert to the operator, the alert indicating that the adjustment input configuration is not within the acceptable operating range, and the alert further indicating a surgical risk associated with adjusting an input to the adjustment input configuration that is not within the acceptable operating range.

[0311] (6) A method according to any one of embodiments 1 to 5, wherein the adjustment input configuration is received from the operator or generated by an ML model, the ML model being trained to determine the adjustment input configuration as the input for controlling the surgical device for the surgical step using past surgical data. (7) determining whether the adjustment input configuration is from the operator; based on the determination that the adjustment input configuration is from the operator, sending a message to the operator to confirm whether to adjust the input for controlling the surgical device using the adjustment input configuration that is not within the acceptable operating range; 7. The method of embodiment 6, further comprising: (8) receiving a feedback message from the operator confirming use of the adjusted input configuration as the input for controlling the surgical device; accepting the adjustment input configuration as the input for controlling the surgical device for the surgical step; 8. The method of embodiment 7, further comprising: (9) The method of embodiment 8, further comprising sending a request message to the operator to confirm whether to modify the allowable operating range based on the adjustment input configuration. (10) determining whether the adjustment input configuration was generated by a machine learning (ML) model; and sending ML model metadata to the operator based on the determination that the tuning input configuration was generated by the ML model; and 7. The method of embodiment 6, further comprising:

[0312] (11) The method of claim 10, wherein the ML model metadata includes at least one of frequency information of other operators using the adjusted input configuration for the surgical step, or a surgical success rate of the surgical procedure using the adjusted input configuration. (12) A method according to any one of claims 1 to 11, further comprising using the adjustment input configuration to adjust an input for controlling the surgical device for the surgical step based on the determination that the adjustment input configuration is within the allowable operating range. (13) A surgical device, A surgical device comprising a processor configured to execute a method according to any one of embodiments 1 to 12. (14) A device, A device comprising a processor configured to execute a method according to any one of embodiments 1 to 12. (15) A computing program that, when executed by a processor, causes the processor to perform the method described in any one of embodiments 1 to 12.

Claims

1. 1. A processor-implemented method, the method comprising: receiving surgical data associated with a surgical procedure, the surgical data including data associated with at least one of a patient, an operator, or a surgical device used in the surgical procedure; identifying surgical steps associated with the surgical procedure based on the surgical procedure data; determining an allowable operating range associated with the surgical device based on the surgical step and the surgical data, the allowable operating range being an operating range for controlling the surgical device for the surgical step; receiving an adjustment input configuration, the adjustment input configuration configured to control the surgical device for the surgical step; determining whether the adjustment input configuration is within the determined allowable operating range; preventing the adjustment input configuration from controlling the surgical device based on a determination that the adjustment input configuration is not within the allowable operating range; and A method comprising:

2. 10. The method of claim 1, wherein determining an allowable operating range associated with the surgical device includes inputting the surgical data into a machine learning (ML) model, the ML model being trained to determine the allowable operating range of the surgical device for performing the surgical step using historical surgical data.

3. The method of claim 1 , wherein determining an allowable operating range associated with the surgical device includes retrieving an allowable operating range of the surgical device for performing the surgical step from historical surgical data.

4. further comprising identifying the surgical device to be used in the surgical procedure based on the surgical procedure data; The method of claim 1 , wherein determining an allowable operating range associated with the surgical device is also based on the surgical device.

5. 10. The method of claim 1, further comprising sending an alert to the operator, the alert indicating that the adjustment input configuration is not within the acceptable operating range, the alert further indicating a surgical risk associated with adjusting an input to the adjustment input configuration that is not within the acceptable operating range.

6. 2. The method of claim 1, wherein the adjustment input configuration is received from the operator or generated by an ML model, the ML model being trained using past surgical data to determine the adjustment input configuration as the input for controlling the surgical device for the surgical step.

7. determining whether the adjustment input configuration is from the operator; based on the determination that the adjustment input configuration is from the operator, sending a message to the operator to confirm whether to adjust the input for controlling the surgical device using the adjustment input configuration that is not within the acceptable operating range; The method of claim 6 further comprising:

8. receiving a feedback message from the operator confirming use of the adjusted input configuration as the input for controlling the surgical device; accepting the adjustment input configuration as the input for controlling the surgical device for the surgical step; The method of claim 7 further comprising:

9. The method of claim 8 , further comprising: sending a request message to the operator to confirm whether to modify the allowable operating range based on the adjustment input configuration.

10. determining whether the tuning input configuration was generated by a machine learning (ML) model; sending ML model metadata to the operator based on the determination that the adjustment input configuration was generated by the ML model; and The method of claim 6 further comprising:

11. 11. The method of claim 10, wherein the ML model metadata includes at least one of: frequency information of other operators using the adjusted input configuration for the surgical step; or a surgical success rate of the surgical procedure using the adjusted input configuration.

12. 10. The method of claim 1, further comprising using the adjustment input configuration to adjust an input for controlling the surgical device for the surgical step based on the determination that the adjustment input configuration is within the allowable operating range.

13. 1. A surgical device comprising: A surgical device comprising a processor configured to perform the method of any one of claims 1 to 12.

14. A device, A device comprising a processor configured to perform the method of any one of claims 1 to 12.

15. A computing program which, when executed by a processor, causes the processor to carry out the method of any one of claims 1 to 12.

Images