High bipolar sealing quality prediction

Abstract

A system is provided for assessing the quality of a seal applied to tissue by an electrosurgical instrument. The system can include a surgical device having a control circuit for determining that the end effector of the surgical device is in a closed configuration, determining the presence of tissue disposed between a first jaw and a second jaw of the end effector, monitoring the movement of the end effector in the closed configuration in which tissue is present between the first jaw and the second jaw, detecting movement of the end effector outside of a predetermined range based on the movement data, and providing feedback data based on the detected movement of the end effector.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 63 / 357,177, entitled "ADVANCED BIPOLAR SEAL QUALITY PREDICTION," filed June 30, 2022, the disclosure of which is incorporated herein by reference in its entirety. [Background technology]

[0002] FIELD OF THE DISCLOSURE The present disclosure relates to electrosurgical instruments designed to seal and cut tissue. Summary of the Invention [Means for solving the problem]

[0003] The following summary is provided to facilitate an understanding of some of the innovative features unique to the aspects disclosed herein and is not intended to be an exhaustive description, A complete understanding of the various aspects can be gained by taking the entire specification, claims, and abstract as a whole.

[0004] In one general aspect, the present disclosure is directed to a surgical device. The surgical device includes an end effector having a first jaw and a second jaw. The surgical device further includes a first sensor for detecting tissue disposed between the first jaw and the second jaw, a second sensor for detecting the end effector in a closed configuration, and a third sensor for detecting movement of the end effector. The surgical device further includes a control circuit communicatively coupled to the first sensor, the second sensor, and the third sensor. The control circuit includes a processor and a memory, and when the memory is executed by the processor, causes the control circuit to determine, based on the first sensor data, that the end effector is in a closed configuration, determine, based on the second sensor data, the presence of tissue disposed between the first jaw and the second jaw, and monitor, based on the third sensor data, the movement of the end effector in a closed configuration with tissue present between the first jaw and the second jaw. The memory stores further instructions that, when executed by the processor, cause the control circuit to detect movement of the end effector outside a predetermined range based on the movement and provide feedback data based on the detected movement of the end effector.

[0005] In at least one aspect, the memory stores further instructions that, when executed by the processor, cause the control circuit to calculate the tension on the tissue based on the movement.

[0006] In at least one aspect, the feedback is visual. In at least one aspect, the visual feedback is overlaid on a display image of the surgical site.

[0007] In at least one aspect, the feedback is auditory.

[0008] In at least one aspect, the first jaw includes a clamp arm and the second jaw includes an ultrasonic blade.

[0009] In at least one aspect, the first jaw comprises an anvil and the second jaw comprises a staple cartridge.

[0010] In another general aspect, the present disclosure is directed to a surgical device. The surgical device comprises an end effector having a first jaw and a second jaw. The surgical device further comprises a first sensor for detecting tissue disposed between the first jaw and the second jaw, a second sensor for detecting that the end effector is in a closed configuration, a first reference mark, and a second reference mark. The surgical device further comprises a control circuit communicatively coupled to the first sensor, the second sensor, and a camera. The control circuit comprises a processor and a memory. When executed by the processor, the memory causes the control circuit to receive video data of the surgical site from the camera, determine on the first sensor data that the end effector is in a closed configuration, determine the presence of tissue disposed between the first jaw and the second jaw based on the second sensor data, store instructions that cause the control circuit to determine the position of the device tip within the video data based on the first reference mark and the second reference mark, determine a region of interest within the video data based on the position of the device tip within the video data, and analyze the region of interest of the end effector in the closed configuration where tissue is present between the first jaw and the second jaw. The memory stores further instructions that, when executed by the processor, cause the control circuit to determine tissue tension based on the analysis and provide feedback based on the tension.

[0011] In at least one aspect, the memory stores further instructions that, when executed by the processor, cause the control circuit to detect movement of the end effector outside a predetermined range based on the analysis.

[0012] In at least one aspect, the memory stores further instructions that, when executed by the processor, cause the control circuit to determine the device type based on the first reference mark and the second reference mark.

[0013] In at least one aspect, the feedback is visual. In at least one aspect, the visual feedback is overlaid on a display image of the surgical site.

[0014] In at least one aspect, the feedback is auditory.

[0015] In at least one aspect, the first jaw comprises a clamping arm and the second jaw comprises an ultrasonic blade.

[0016] In at least one aspect, the first jaw comprises an anvil and the second jaw comprises a staple cartridge.

[0017] In yet another general aspect, the present disclosure is directed to a surgical system including a surgical instrument, an RF energy source, and a control circuit. The surgical instrument includes an end effector for capturing tissue. The end effector includes an electrode for applying radio-frequency (RF) energy to the tissue captured by the end effector. The RF energy source provides RF energy to the electrode. The control circuit transmits a control signal to the RF energy source. The control signal causes the RF energy source to provide RF energy to the electrode to apply a seal to the tissue captured by the end effector. The control circuit further predicts the quality of the seal and provides feedback to the user based on the prediction.

[0018] In at least one aspect, the control circuit generates a value associated with the seal and compares the value to a seal threshold. To provide feedback to the user based on the prediction, the control circuit provides feedback to the user based on the result of the comparison.

[0019] In at least one aspect, the control circuit refrains from providing feedback based on the value reaching or exceeding the seal threshold.

[0020] In at least one aspect, the surgical system further comprises a display. The control circuit transmits a signal to the display based on the value falling below a sealing threshold. The feedback includes visual feedback on the display. The visual feedback is based on the signal.

[0021] In at least one aspect, the surgical system further comprises an audio feedback module. The control circuit transmits a signal to the audio feedback module based on the value falling below a sealing threshold. The feedback includes audio feedback via the audio feedback module. The audio feedback is based on the signal.

[0022] In at least one aspect, the surgical system further comprises a tactile feedback module. The control circuit transmits a signal to the tactile feedback module based on the value falling below a sealing threshold. The feedback includes tactile feedback via the tactile feedback module. The tactile feedback is based on the signal.

[0023] In at least one aspect, the RF energy source comprises a control circuit.

[0024] In at least one aspect, the surgical system further comprises a processing unit comprising a control circuit.

Brief Description of the Drawings

[0025] The various features of the embodiments described herein, together with their advantages, can be understood by practicing the following invention in conjunction with the accompanying drawings below.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19A

Figure 19B

Figure 19C

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32A

Figure 32B

Figure 33

Figure 34

Figure 35

Figure 36A

Figure 36B

Figure 36C

Figure 36D

Figure 37

Figure 38

Figure 39

Figure 40

[0026] Throughout the several views, corresponding reference numerals indicate corresponding parts. The examples described herein are illustrative of various embodiments of the invention in one form, and such examples should not be construed as limiting the scope of the invention in any way.

DETAILED DESCRIPTION OF THE INVENTION

[0027] To provide a complete understanding of the overall structure, function, manufacture, and use of the embodiments, numerous specific details are set forth as described in the specification and shown in the accompanying drawings. Well-known operations, components, and elements are not described in detail so as not to obscure the embodiments described herein. It will be understood by those of ordinary skill in the art that the embodiments described and illustrated herein are non-limiting examples, and thus the specific structural and functional details disclosed herein may be representative and exemplary. Modifications and variations can be made thereto without departing from the scope of the claims.

[0028] The terms "comprise", "comprises" and any other forms of the word "comprise" such as "comprising", "have", "has" and any other forms of the word "have" such as "having", "include", "includes" and any other forms of the word "include" such as "including", and "contain", "contains" and any other forms of the word "contain" such as "containing" are open-ended conjunctive verbs. As a result, a surgical system, device or apparatus that "comprises", "has", "includes" or "contains" one or more elements has those one or more elements, but is not limited to having only those one or more elements. Similarly, an element of a system, device or apparatus that "comprises", "has", "includes" or "contains" one or more features has those one or more features, but is not limited to having only those one or more features.

[0029] The terms "proximal" and "distal" are used herein with reference to a clinician who operates the handle portion of a surgical instrument. The term "proximal" refers to the portion closest to the clinician, and the term "distal" refers to the portion located farther from the clinician. For convenience and clarity, it will be further understood that spatial terms such as "vertical", "horizontal", "up" and "down" may be used herein with respect to the drawings. However, surgical instruments are used in many orientations and positions, and these terms are not intended to be limiting and / or absolute.

[0030] Various exemplary devices and methods are provided for performing laparoscopic and minimally invasive surgical procedures. However, it will be readily understood by the reader that the various methods and devices disclosed herein can be used in many surgical procedures and applications, including those related to, for example, open surgical procedures. By reading on in the "DETAILED DESCRIPTION" of this specification, the reader will further understand that the various instruments disclosed herein can be inserted into the body in any manner, such as through an existing opening or through an incision or puncture formed in the tissue. The working part of these instruments, i.e., the end effector part, can be inserted directly into the patient's body or also through an access device having an operating passage through which the end effector and the elongate shaft of the surgical instrument can be advanced.

[0031] Referring to FIG. 1, hub 106 is shown in communication with visualization system 108, robotic system 110, and hand-held intelligent surgical instrument 112. In some aspects, visualization system 108 may be a separable device. In an alternative aspect, visualization system 108 may be included within hub 106 as a functional module. Hub 106 includes hub display 135, imaging module 138, generator module 140, communication module 130, processor module 132, storage array 134, and operating room mapping module 133. In certain aspects, as shown in FIG. 1, hub 106 further includes a smoke evacuation module 126, a suction / irrigation module 128, and / or an insufflation module 129. In certain aspects, any of the modules within hub 106 can be combined with each other to form a single module.

[0032] During a surgical procedure, applying energy to tissue for sealing and / or cutting is generally associated with smoke evacuation, aspiration of excess fluid, and / or perfusion of tissue. Fluid lines, power lines, and / or data lines from different sources often become entangled during a surgical procedure. Valuable time may be lost in addressing this issue during a surgical procedure. To untangle the lines, it may be necessary to unplug the lines from their corresponding modules, and for that, it may be necessary to reset the modules. The hub module type enclosure 136 provides an integrated environment for managing power, data, and fluid lines and reduces the frequency of entanglement between such lines.

[0033] Aspects of the present disclosure present a surgical hub for use in a surgical procedure involving the application of energy to tissue at a surgical site. The surgical hub includes a hub enclosure and a combined generator module slidably receivable within a docking station of the hub enclosure. The docking station includes data contacts and power contacts. The combined generator module includes one or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component housed within a single unit. In one aspect, the combined generator module also includes a smoke evacuation component, at least one energy delivery cable for connecting the combined generator module to a surgical instrument, at least one smoke evacuation component for discharging smoke, fluid, and / or particulates generated by the application of therapeutic energy to tissue, and a fluid line extending from a remote surgical site to the smoke evacuation component.

[0034] In one aspect, the fluid line is a first fluid line, and a second fluid line extends from a remote surgical site to an aspiration and perfusion module slidably received within the hub enclosure. In one aspect, the hub enclosure includes a fluid interface.

[0035] Certain surgical procedures may require the application of two or more energy types to tissue. One energy type may be more beneficial for cutting tissue, while another, different energy type may be more beneficial for sealing tissue. For example, a bipolar generator may be used to seal tissue, while an ultrasonic generator may be used to cut the sealed tissue. Aspects of the present disclosure present a solution in which the hub modular enclosure 136 houses different generators and facilitates bidirectional communication between them. One advantage of the hub modular enclosure 136 is that it allows for quick removal and / or replacement of various modules.

[0036] Aspects of the present disclosure provide a modular surgical enclosure for use in a surgical procedure involving the application of energy to tissue. The modular surgical enclosure includes a first energy generator module for generating a first energy for application to tissue and a first docking station including a first docking port including first data and power contacts. In one aspect, the first energy generator module is slidably movable into electrical engagement with the power and data contacts, and the first energy generator module is slidably movable out of electrical engagement with the first power and data contacts. In an alternative aspect, the first energy generator module is stackably movable into electrical engagement with the power and data contacts, and the first energy generator module is stackably movable out of electrical engagement with the first power and data contacts.

[0037] In addition to the above, the modular surgical enclosure also includes a second energy generator module for generating a second energy, the same as or different from the first energy, for application to tissue, and a second docking station having a second docking port that includes second data and power contacts. In one aspect, the second energy generator module is slidably movable to engage electrically with the power and data contacts, and the second energy generator module is slidably movable to disengage from the electrical engagement with the second power and data contacts. In an alternative aspect, the second energy generator module is stackably movable to engage electrically with the power and data contacts, and the second energy generator module is stackably movable to disengage from the electrical engagement with the second power and data contacts.

[0038] In addition, the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port to facilitate communication between the first energy generator module and the second energy generator module.

[0039] Referring to FIG. 1, an aspect of the present disclosure presents a hub module type enclosure 136 that enables modular integration of a generator module 140, a smoke exhaust module 126, a suction / irrigation module 128, and an air supply module 129. The hub module type enclosure 136 further facilitates interactive communication between the module 140, the module 126, the module 128, and the module 129. The generator module 140 may be a generator module comprising an integrated single-pole component, bipolar component, and ultrasonic component, supported within a single housing unit slidably insertable into the hub's modular enclosure 136. The generator module 140 can be connected to a monopolar device 142, a bipolar device 144, and an ultrasonic device 148. Alternatively, the generator module 140 may comprise a series of monopolar, bipolar, and / or ultrasonic generator modules that interact via the hub module type enclosure 136. The hub module type enclosure 136 can facilitate the insertion of multiple generators and two-way communication between the generators docked to the hub module type enclosure 136 such that the generators function as a single generator.

[0040] In one aspect, the hub's modular enclosure 136 comprises a modular power and communication backplane 149 with external and wireless communication headers to enable removable attachment of the modules 140, 126, 128, 129 and interactive communication therebetween.

[0041] Further information regarding the hub can be found in U.S. Patent Application Publication Nos. 2019 / 0201136 and 2020 / 0078106, which are hereby incorporated by reference in their entirety.

[0042] Figure 2 shows a control circuit 500 for controlling aspects of a surgical instrument or tool, according to one aspect of the present disclosure. The control circuit 500 can implement various processes described herein. The control circuit 500 may comprise a microcontroller comprising one or more processors 502 (e.g., a microprocessor, a microcontroller) coupled to at least one memory circuit 504. The memory circuit 504 stores machine-executable instructions that, when executed by the processor 502, cause the processor 502 to execute machine instructions for implementing the various processes described herein. The processor 502 may be any one of a number of single-core or multi-core processors well known in the art. The memory circuit 504 may comprise volatile and non-volatile storage media. The processor 502 may include an instruction processing unit 506 and an arithmetic unit 508. The instruction processing unit can receive instructions from the memory circuit 504 of the present disclosure.

[0043] Figure 3 shows a combinational logic circuit 510 for controlling aspects of a surgical instrument or tool, according to one aspect of the present disclosure. The combinational logic circuit 510 can implement various processes described herein. The combinational logic circuit 510 may comprise a finite state machine comprising combinational logic 512 for receiving data associated with a surgical instrument or tool at an input 514, processing the data by the combinational logic 512, and providing an output 516.

[0044] Figure 4 shows a sequential logic circuit 520 for controlling aspects of a surgical instrument or tool, according to one aspect of the present disclosure. The sequential logic circuit 520 or combinatorial logic 522 can implement various processes described herein. The sequential logic circuit 520 may comprise a finite state machine. The sequential logic circuit 520 can comprise, for example, combinatorial logic 522, at least one memory circuit 524, and a clock 529. The at least one memory circuit 524 can store the current state of the finite state machine. In certain examples, the sequential logic circuit 520 can be synchronous or asynchronous. The combinatorial logic 522 can receive data associated with the surgical instrument or tool from an input 526, process the data by the combinatorial logic 522, and provide an output 528. In other aspects, the circuit can comprise a combination of a processor (e.g., the processor 502 of FIG. 2) and a finite state machine to implement various processes herein. In other aspects, the finite state machine can comprise a combination of a combinatorial logic circuit (e.g., the combinatorial logic circuit 510 of FIG. 3) and the sequential logic circuit 520.

[0045] Figure 5 is a schematic view of a surgical instrument 790 for controlling various functions, according to one aspect of the present disclosure. In one aspect, the surgical instrument 790 is programmed to control the distal translation of a displacement member, such as a closure member 764. The surgical instrument 790 comprises an end effector 792 that can comprise a clamp arm 766, a closure member 764, a blade 768 that can be exchanged with or function with one or more RF electrodes 796 (shown in dashed lines). In various embodiments, the blade can include an I-beam, such as those described elsewhere herein. In various embodiments, the blade can include an ultrasonic blade coupled to an ultrasonic transducer driven by an ultrasonic generator.

[0046] In one aspect, the sensor 788 may be implemented, among other things, as a limit switch, an electromechanical device, a solid state switch, a Hall effect device, an MR device, a GMR device, or a magnetometer. In other implementations, the sensor 638 may be, among other things, a solid state switch that operates under the influence of light, such as an optical sensor, an IR sensor, or an ultraviolet sensor. Further, the switch may be a solid state device such as a transistor (e.g., FET, junction FET, MOSFET, bipolar). In other implementations, the sensor 788 may include, among other things, a non-electric conductor-containing switch, an ultrasonic switch, an accelerometer, and an inertial sensor.

[0047] In one aspect, the position sensor 784 may be implemented as an absolute positioning system, including a magnetic rotary absolute positioning system implemented as the AS5055EQFT single-chip magnetic rotary position sensor available from austriamicrosystems AG. The position sensor 784 can be interfaced with the control circuit 760 to provide an absolute positioning system. The position is located above the magnet and is provided to implement a concise and efficient algorithm for calculating hyperbolic and trigonometric functions that require only addition, subtraction, bit shifting, and table reference operations, and may include a plurality of Hall effect elements coupled to a CORDIC processor, also known as the digit-by-digit method and the border algorithm.

[0048] In some examples, the position sensor 784 may be omitted. If the motor 754 is a stepper motor, the control circuit 760 may track the position of the closure member 764 by summing the number and direction of steps that the motor has been instructed to execute. The position sensor 784 may be located within the end effector 792 or at any other part of the instrument.

[0049] The control circuit 760 may communicate with one or more sensors 788. The sensors 788 may be positioned on the end effector 792 and adapted to operate with the surgical instrument 790 to measure various derived parameters such as gap distance versus time, tissue compression versus time, and anvil strain versus time. The sensors 788 may include magnetic sensors, magnetic field sensors, strain gauges, pressure sensors, force sensors, inductive sensors such as eddy current sensors, resistive sensors, capacitive sensors, optical sensors, and / or any other suitable sensors for measuring one or more parameters of the end effector 792. The sensors 788 may include one or more sensors.

[0050] The RF energy source 794 is coupled to the end effector 792 and is applied to the RF electrode 796 when the RF electrode 796 is provided within the end effector 792 in place of the blade 768 or to operate with the blade 768. For example, the blade may be made of a conductive metal and used as a return path for electrosurgical RF current. The control circuit 760 controls the delivery of RF energy to the RF electrode 796.

[0051] The control circuit 760 may communicate with a tactile feedback module 870. In some embodiments, the tactile feedback module 870 is positioned within a handpiece, such as within the handpieces 1205, 1207, 1209 of the surgical instruments 1204, 1206, 1208, as will be described in more detail below. In some embodiments, the tactile feedback module 870 is positioned within an input interface with which a user interacts to control various surgical instruments described elsewhere herein. The control circuit 760 can transmit a control signal to the tactile feedback module 870 to activate the tactile feedback module 870 and provide tactile feedback, as will be described in more detail elsewhere herein.

[0052] The control circuit 760 may communicate with the audio feedback module 871. The control circuit 760 may send a control signal to the audio feedback module 871 to activate the audio feedback module 871 and provide audio feedback to the user, as described in more detail elsewhere in this specification.

[0053] Additional details are disclosed in U.S. Patent Application No. 15 / 636,096, filed Jun. 28, 2017, entitled "SURGICAL SYSTEM COUPLABLE WITH STAPLE CARTRIDGE AND RADIO FREQUENCY CARTRIDGE, AND METHOD OF USING SAME", which is hereby incorporated by reference in its entirety.

[0054] Generator hardware As used throughout this specification, the term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that can communicate data through the use of modulated electromagnetic radiation via a non-solid medium. This term does not mean that the associated devices do not include any wired components, but in some aspects, they may not be present. The communication module may implement any one of a number of wireless or wired communication standards or protocols, including but not limited to Wi-Fi (IEEE802.11 family), WiMAX (IEEE802.16 family), IEEE802.20, Long-Term Evolution (LTE), Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, Bluetooth, and only derivatives of these Ethernet, as well as any other wireless and wired protocols designated as 3G, 4G, 5G, and beyond. The computing module may include a plurality of communication modules. For example, the first communication module may be dedicated to short-range wireless communication such as Wi-Fi and Bluetooth, and the second communication module may be dedicated to long-range wireless communication such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, etc.

[0055] As used in this specification, a processor or processing unit is an electronic circuit that performs operations on some external data source (usually memory) or some other data stream. In this specification, this term is used to refer to the central processor (central processing unit) within a system or computer system (especially a system on a chip (SoC)) that combines many specialized "processors".

[0056] As used herein, a system-on-chip (SoC or SOC) is an integrated circuit (also known as an "IC" or "chip") that integrates all the components of a computer or other electronic system. This can include digital, analog, mixed-signal, and in many cases high-frequency functions, all on a single substrate. An SoC integrates a microcontroller (or microprocessor) with up-to-date peripheral devices such as a graphics processing unit (GPU), Wi-Fi module, or coprocessor. An SoC may or may not include on-chip memory.

[0057] As used herein, a microcontroller or controller is a system that integrates a microprocessor with peripheral circuits and memory. A microcontroller (or MCU of a microcontroller unit) may be implemented as a small computer on a single integrated circuit. This may be similar to an SoC, which may include a microcontroller as one of its components. A microcontroller may house memory and programmable input / output peripherals along with one or more core processing units (CPUs). Program memory in the form of ferroelectric RAM, NOR flash, or OTP ROM and a small amount of RAM are also often included on the chip. A microcontroller can be used for embedded applications, as opposed to microprocessors used in personal computers or other general-purpose applications composed of various discrete chips.

[0058] As used herein, the term controller or microcontroller may be a stand-alone IC or chip device that interfaces with peripheral devices. This may also be a connection between two parts of a computer or controller on an external device that manages the operation of the device (and the connection to the device).

[0059] Any of the processors or microcontrollers described in this specification may be implemented by any single-core or multi-core processor, such as those known by the trade name of ARM Cortex made by Texas Instruments. In one aspect, the processor may be, for example, the LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments. This processor core includes on-chip memory of 256KB single-cycle flash memory or other non-volatile memory with a maximum of 40MHz, a prefetch buffer for improving performance beyond 40MHz, 32KB single-cycle serial random access memory (SRAM), internal read-only memory (ROM) with StellarisWare (registered trademark) software, 2KB electrically erasable programmable read-only memory (EEPROM), one or more pulse width modulation (PWM) modules, one or more quadrature encoder input (QEI) analogs, and one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels. The details are available in the product datasheet.

[0060] In one aspect, the processor may include a safety controller that includes two controller-based families such as TMS570 and RM4x, also known by the trade name of Hercules ARM Cortex R4 made by Texas Instruments. The safety controller may be configured specifically for safety-critical applications of IEC61508 and ISO26262, among others, to provide a highly integrated safety mechanism while offering scalable performance, connectivity, and memory options.

[0061] A modular device includes a module (such as described in connection with FIG. 1) that is receivable within a surgical hub, and a surgical device or instrument that can be connected to various modules to connect or pair with a corresponding surgical hub. Examples of modular devices include, for example, intelligent surgical instruments, medical imaging devices, aspiration / irrigation devices, smoke evacuators, energy generators, ventilators, inhalers, and displays. The modular devices described herein can be controlled by a control algorithm. The control algorithm can be executed on the modular device itself, on the surgical hub to which a particular modular device is paired, or on both the modular device and the surgical hub (e.g., via a distributed computing architecture). In some examples, the control algorithm of the modular device controls the device based on data sensed by the modular device itself (i.e., by sensors within, on, or connected to the modular device). This data can be related to the patient during surgery (e.g., tissue characteristics or insufflation pressure), or it can be related to the modular device itself (e.g., the speed of a advancing knife, motor current, or energy level). For example, the control algorithm of a surgical stapling and cutting instrument can control the speed at which the motor of the instrument drives the knife through tissue based on the resistance generated by the knife as it advances.

[0062] FIG. 6 shows one form of a surgical system 1200 comprising a modular energy system 1000 and various surgical instruments 1204, 1206, 1208 that can be used therewith. The surgical instrument 1204 is an ultrasonic surgical instrument, the surgical instrument 1206 is an RF electrosurgical instrument, and the multifunctional surgical instrument 1208 is a combined ultrasonic / RF electrosurgical instrument. The modular energy system 1000 is configurable to be used with various surgical instruments. According to various forms, the modular energy system 1000 can be configured to be used with different types of different surgical instruments, including, for example, an ultrasonic surgical instrument 1204, an RF electrosurgical instrument 1206, and a multifunctional surgical instrument 1208 that integrates RF energy and ultrasonic energy delivered individually or simultaneously from the modular energy system 1000. In the form of FIG. 6, the modular energy system 1000 is shown separately from the surgical instruments 1204, 1206, 1208 in one form, but the modular energy system 1000 may be integrally formed with any of the surgical instruments 1204, 1206, 1208 to form an integrated surgical system. The modular energy system 1000 can be configured for wired or wireless communication.

[0063] The modular energy system 1000 is for driving a plurality of surgical instruments 1204, 1206, 1208. The first surgical instrument is the ultrasonic surgical instrument 1204, which includes a hand-piece 1205, an ultrasonic transducer 1220, a shaft 1226, and an end effector 1222. The end effector 1222 includes an ultrasonic blade 1228 acoustically coupled to the ultrasonic transducer 1220 and a clamp arm 1240. The hand-piece 1205 includes a trigger 1243 for operating the clamp arm 1240 and a combination of toggle buttons 1234a, 1234b, 1234c for energizing and driving the ultrasonic blade 1228 or other functions. The toggle buttons 1234a, 1234b, 1234c can be for energizing the ultrasonic transducer 1220 in the modular energy system 1000.

[0064] The modular energy system 1000 is also for driving a second surgical instrument 1206. The second surgical instrument 1206 is an RF electrosurgical instrument and includes a hand-piece 1207, a shaft 1227, and an end effector 1224. The end effector 1224 includes electrodes within clamp arms 1242a, 1242b and returns through the conductive portion of the shaft 1227. The electrodes are coupled to a bipolar energy source within the modular energy system 1000 and are thereby energized. The hand-piece 1207 includes a trigger 1245 for operating the clamp arms 1242a, 1242b and an energy button 1235 for activating an energy switch to energize the electrodes within the end effector 1224.

[0065] The modular energy system 1000 is also for driving a multi-functional surgical instrument 1208. The multi-functional surgical instrument 1208 includes a handpiece 1209 (HP), a shaft 1229, and an end effector 1225. The end effector 1225 includes an ultrasonic blade 1249 and a clamp arm 1246. The ultrasonic blade 1249 is acoustically coupled to an ultrasonic transducer 1220. The ultrasonic transducer 1220 may be separable from or integral with the handpiece 1209. The handpiece 1209 includes a trigger 1247 for operating the clamp arm 1246 and a combination of toggle buttons 1237a, 1237b, 1237c for energizing and driving the ultrasonic blade 1249 or other functions. The toggle buttons 1237a, 1237b, 1237c can energize the ultrasonic transducer 1220 using the modular energy system 1000 and energize the ultrasonic blade 1249 using a bipolar energy source similarly housed within the modular energy system 1000.

[0066] The modular energy system 1000 can be configured for use with various surgical instruments. According to various forms, the modular energy system 1000 can be configured for use with different types of different surgical instruments, including, for example, an ultrasonic surgical instrument 1204, an RF electrosurgical instrument 1206, and a multi-functional surgical instrument 1208 that integrates RF energy and ultrasonic energy delivered individually or simultaneously from the modular energy system 1000. In the form of FIG. 6, the modular energy system 1000 is shown separately from the surgical instruments 1204, 1206, 1208, but in another form, the modular energy system 1000 may be integrally formed with any one of the surgical instruments 1204, 1206, 1208 to form a single surgical system. A generator for digitally generating an electrical signal waveform and further aspects of the surgical instrument are described in U.S. Patent Application Publication No. 2017-0086914 (A1), which is hereby incorporated by reference in its entirety.

[0067] Additional information regarding the modular energy system can be found in U.S. Patent Application No. 17 / 217,394, entitled "METHOD FOR MECHANICAL PACKAGING FOR MODULAR ENERGY SYSTEM," filed on March 30, 2021, which is hereby incorporated by reference in its entirety.

[0068] Tissue tension Although the general implementation of the modular energy system 1000 and several surgical instruments that can be coupled thereto has been described, the present disclosure now turns to the issue of tissue tension and the ability to calculate tissue tension using surgical devices. Tissue tension is an important parameter for both the performance of energy and in-body devices, and if used inappropriately, it can have an adverse effect on the quality of sealing or stapling. Put another way, tissue tension affects all surgical devices that clamp tissue, such as staplers, clip appliers, ultrasonic devices, suture devices, bipolar devices, monopolar devices, and graspers. Having insufficient control over the device can result in more tissue tension, which can damage the tissue being treated and have an adverse effect on the surgical outcome. Currently, many surgical devices, such as handheld surgical devices, do not have the ability to determine tissue tension.

[0069] The present disclosure provides various solutions for calculating tissue tension and warning the user of the current tissue tension. This enables the surgeon to maintain low tissue tension and can assist the surgeon in generating beneficial surgical outcomes. The control circuit of the surgical device can be coupled to sensors that provide it with information. This information can enable the control circuit to determine tissue tension and warn the user. The control circuit can calculate tissue tension quantitatively or qualitatively. Some qualitative examples can be "Low", "Medium", "High", "Lower" / "Falling", or "Higher" / "Rising". The user can be warned of tissue tension in real time through visual or auditory feedback. In some examples, the user can also be provided with feedback regarding postoperative tissue tension.

[0070] FIG. 7 shows a postoperative surgical video evaluation system that enables a fair observer to use the evaluation system 1600 to evaluate how a surgeon performed in a surgical video recording. The evaluation system 1600 covers bilateral dexterity 1610, efficiency 1620, force sensitivity 1630, and robot control 1640. The goal of the evaluation system 1600 is to provide the surgeon with fair feedback regarding their performance so that the surgeon can improve surgical outcomes. Providing the surgeon with real-time feedback regarding tissue tension during surgery can help the surgeon improve the evaluation of force sensitivity 1630. Having high tissue tension can lead to tissue tearing and / or damage to neighboring structures, which can, inter alia, reduce the surgeon's force sensitivity evaluation 1630. When the surgeon is suturing, high tissue tension can lead to frequent suture breakage and also reduce the force sensitivity evaluation 1630.

[0071] Figures 8 - 10 show view 1390 provided to a surgeon when performing a surgical procedure. Surgical device 1300 has a shaft 1310 inserted into a patient. Shaft 1310 terminates in an end effector 1320. End effector 1320 has a first jaw 1322 and a second jaw 1324. First jaw 1322 rotates relative to second jaw 1324 from an open configuration to a closed configuration. Shaft 1310 of surgical device 1300 has thereon a label 1312 that details the type of surgical device. For example, in Figures 8 - 10, the surgical device is an ultrasonic surgical device having label 1312 ultrasonic. In the case of surgical device 1300, first jaw 1322 may be a clamp arm and second jaw 1324 may be a blade. However, if the surgical device were a stapler, the first jaw could be an anvil and the second jaw could accommodate a staple cartridge. Surgical device 1300 can include various sensors that provide information to a control circuit that controls surgical device 1300. For example, the control circuit that controls the surgical device can be hub 106, control circuit 500, or control circuit 760. In one example, the control circuit can be part of the surgical device. In an alternative example, the control circuit is separate and can be communicatively coupled to the surgical device.

[0072] Surgical device 1300 can include a tissue sensor for detecting tissue disposed between first jaw 1322 and second jaw 1324. In one example, the tissue sensor can be a capacitance sensor that can detect tissue between jaws 1322, 1324. In an alternative example, the tissue sensor can detect the continuity between the two jaws 1322, 1324 and determine whether there is tissue between jaws 1322, 1324. For example, the surgical device can use a return electrode to calculate the continuity. If the surgical device is a bipolar electrosurgical device, the continuity can be calculated directly. The tissue sensor enables the control circuit to determine when there is tissue between jaws 1322, 1324.

[0073] Surgical device 1300 can include a jaw closure sensor for detecting an end effector in a closed configuration. In one example, the jaw closure sensor may be a jaw closure switch, which is activated when jaws 1322, 1324 are closed. In an alternative example, the jaw closure sensor may be a jaw closure mechanism position sensor, and the position sensor can be used to determine where the first jaw 1322 is in the closing process. For example, the position sensor can be used to determine when jaws 1322, 1324 are closed.

[0074] Surgical device 1300 can include a motion sensor for detecting the motion of end effector 1320 or the motion of device 1300. For example, the motion sensor can detect the motion of the end effector including orientation, position, velocity, and acceleration. In various examples, the motion sensor can include an accelerometer, a gyroscope, an inertial measurement unit, etc. The motion sensor can also be a combination of multiple sensors that provide the motion of the end effector. One or more motion sensors can be attached within or on surgical device 1300.

[0075] The control circuit can also determine when surgical device 1300 is in the surgeon's hand. In some examples, the control circuit can use the motion sensor to determine when surgical device 1300 is in use. In an alternative example, the control circuit can use video analysis from a camera viewing the surgical site to determine when surgical device 1300 enters the field of view. For example, the video analysis can detect a marker 1312 on device 1300 when device 1300 is within the field of view. As another example, the video analysis can detect the geometric shape of device 1300 to detect when device 1300 is within the field of view.

[0076] Referring to FIG. 8, the end effector 1320 of the surgical device 1300 is in an open configuration at a first time t1 in a surgical procedure. In FIG. 9, the end effector 1320 is closed on tissue 1330 at a second time t2 in the surgical procedure. The first time t1 occurs before the second time t2. The tip of the device 1300 is located at 1326. In FIG. 9, it can be seen that tension is applied to the tissue from the tissue tenting 1332. The point 1340 is a position on the tissue that can be monitored to check whether the tissue moves after tissue division. The point 1350 is a point on the surgical device 1300 that can be monitored to see how the device moves after tissue division. In FIG. 10, the tissue is cut and sealed at a third time t3 in the surgical procedure. The second time t2 occurs before the third time t3. It can be seen that there is a sudden movement of the device 1300 following tissue division. This is indicated by the movement of the point 1350 to the point 1352. Further, the point on the tissue moves from 1340 to 1342, indicating that the tissue has also moved. These movements can indicate the tissue tension while the tissue is clamped within the device 1300. The movement of the device 1300 is captured and / or measured, and the events can be tagged for procedure analysis.

[0077] The control circuit can monitor device motion as an indicator of tissue tension, according to FIG. 11. In various examples, the control circuit can conserve computational resources by monitoring the device for tissue tension when the device is clamped onto tissue. The control circuit process 1700 begins at 1702. The control circuit determines whether the surgical device is in use (1704). If the surgical device is in use, the control circuit proceeds to determine whether the end effector is in a closed configuration (1706). If the surgical device is in a closed configuration, the control circuit proceeds to determine whether the jaw is clamped onto tissue (1708). If the jaw is clamped onto tissue, the control circuit proceeds to monitor the motion of the device (1710). If the surgical device is not in use, the end effector is not in a closed configuration, or the jaw is not clamped onto tissue, the control circuit starts process 1700 again (1702). Thereby, the control circuit can monitor the motion of the device only when the surgical device is clamped onto tissue as shown in FIG. 9. The control circuit can monitor the movement of the device and look for any gross or sudden movement that may indicate tissue tension 1712. For example, the control circuit can detect movement of the end effector outside of a predetermined range. The motion of the device can be scaled based on a known distance from the motion sensor to the tip of the device end.

[0078] Monitoring this movement includes monitoring the movement of the device at the end of activation when the tissue is separated or released. The end of activation can be detected by a change in the tissue sensor. It can also be detected through deactivation of the user or a change in the frequency of the ultrasonic device. By monitoring the movement of the device, the control circuit can determine whether there was tension on the tissue 1714. For example, a sudden movement of the device after tissue dissection can indicate tissue tension. The control circuit then proceeds to provide the user with feedback on tissue tension and / or sudden or rough movement on the head-up display. For example, the feedback can be overlaid on the camera view 1390. The feedback can be auditory. In either case, the feedback can be provided to the surgeon in real time, enabling the surgeon to modify their movement to benefit the surgical outcome.

[0079] Feedback can also be provided to the surgeon after the surgery. For example, the movement of the device throughout the surgical procedure can be provided to the user. Referring to FIG. 12, the surgeon can be provided with a drawing 1400 showing the movement of the tip of the surgical device during the surgical procedure. This can provide the surgeon with a view of how the device was moved and whether there were any sudden or rough movements. Drawing 1400 shows the movement of a powered surgical stapler 1420 performing a surgical procedure and a non-powered surgical stapler 1410 performing a similar surgical procedure. It can be seen that the powered surgical stapler provides 37% less movement due to the firing of the push button, enabling the surgeon to be more consistent between surgical procedures.

[0080] In addition, sudden or rough movements can be captured, measured, and tagged for event processing and analysis. The evaluation system 1600 can also be used to provide surgeons with an overall assessment of their abilities after surgery. In various examples, motion feedback can be classified into categories such as motion during movement, motion while in contact with tissue, motion while clamped on tissue, and motion immediately after tissue dissection or release. These categories can enable surgeons to visualize and identify where device motion needs to be improved between different aspects of a surgical procedure. It can also indicate whether a surgeon has continuous sudden movements during a particular aspect of a surgical procedure. Overall, motion feedback can be used to guide and improve a surgeon's skills in real-time or in a post-operative report.

[0081] Referring to FIG. 13, the surgical device 1300 can include one or more reference marks 1362, 1364, and 1366. In various examples, the reference marks 1362, 1364, and 1366 can be rings that wrap around the shaft 1310. In FIG. 13, each reference mark is a black ring around the shaft 1310, and reference mark 1362 is distal to reference mark 1364 which is distal to reference mark 1366. In alternative examples, the one or more reference marks can be any marks on the device 1300 that can be used by a control circuit. The reference marks 1362, 1364, and 1366 can be used to provide device identification. For example, the size and spacing of the reference marks 1362, 1364, and 1366 can be used by a control circuit to identify the surgical device or the type of surgical device. In addition, device type identification information can be known via Bluetooth or direct wired communication to an OR system (e.g., an energy generator). However, one or more reference marks on the device can be useful for identifying the device when multiple surgical devices are present within the camera's field of view. The label 1312 can also be used to identify multiple devices within the field of view. It is also possible to use a data matrix, QR code (registered trademark), etc. to identify multiple devices within the field of view.

[0082] The control circuit can calculate tissue tension during a surgical procedure using video data from a camera. When performing real-time video analysis, there can be a problem with the computational load. However, this burden can be reduced by analyzing small locations within the video image and performing the analysis only over a specific time window based on events of interest. For example, the event of interest can be when tissue is clamped within jaws 1322, 1324 of end effector 1320.

[0083] Referring to FIG. 14, fiducial marks 1362, 1364, and 1364 can be used to define vectors (orientation and direction) in three-dimensional space that can be used to define the location of the tip 1326 of device 1300. In various examples, the maximum extent of each ring, e.g., ends 1362a, 1362b of fiducial mark 1362, ends 1364a, 1364b of fiducial mark 1364, and ends 1366a, 1366b of fiducial mark 1366, can be used to generate vectors in three-dimensional space. The location of tip 1326 of device 1300 within the video data can be calculated based on prior knowledge of the distances between fiducial marks 1362, 1364, and 1366 and tip 1326 of device 1300. In an alternative example, the control circuit can perform an analysis of the video data to localize the tip of the surgical instrument based on the color and geometric shape of the surgical device. Using the location of tip 1326 of surgical device 1300, a region of interest 1370 within the video image data can be calculated. Region of interest 1370 can significantly reduce the computational load by reducing the amount of space within the video that requires video analysis. Additionally, the computational load can be further reduced by performing video analysis only when the surgical device 1300 is clamped onto tissue.

[0084] The control circuit can perform video analysis and calculate tissue tension according to FIG. 15. In various cases, the control circuit can save computational resources by performing video analysis on a subset of video data defined by the region of interest around the device tip. Additionally, the control circuit can choose to perform video analysis only when the device is clamped onto tissue, which further reduces the computational load. The control circuit process 1800 begins by receiving current video data from the camera 1802 looking at the surgical site. The control circuit determines (1804) whether the end effector is in a closed configuration. If the end effector is in a closed configuration, the control circuit proceeds to determine (1806) whether the jaws are clamped onto tissue. If the jaws are clamped onto tissue, the control circuit proceeds to calculate the distal tip position within the video data (1808). If the end effector is not in a closed configuration or the jaws are not clamped onto tissue, the control circuit restarts process 1800 by receiving the next batch of video data from camera 1802. This enables the control circuit to perform video analysis only when the surgical device is clamped onto tissue as shown in FIG. 14.

[0085] The position of the distal tip within the video data can be calculated by using one or more of the fiducial marks of the fiducial marks 1362, 1364, and 1366 (FIG. 14) on the device. One or more fiducial marks can be used to generate a vector in three-dimensional space. The position of the device tip within the video data can be calculated based on prior knowledge of the distance between one or more fiducial marks and the tip of the device. In an alternative example, motion sensors can be used to calculate the tip of the device in three-dimensional space.

[0086] Once the device tip position is known, the control circuit can generate a region of interest around the device tip position within the video data (1810). An exemplary region of interest in the video data is region of interest 1370 shown in FIG. 14. This process reduces the amount of data that needs to be processed by video analysis since the region of interest is only a subset of the video data. Once the region of interest is known, the control circuit can perform video analysis on the video image data to calculate tissue tension. Once the tissue tension is known, the control circuit can provide tissue tension feedback to the user. For example, if tissue tension is detected, a color scale tension alert such as tissue tension alert 1380 can be overlaid on the video monitor. The color scale can be related to a quantitative or qualitative evaluation system. For example, the color scale can be related to a qualitative scale of "low", "medium", or "high" tissue tension. In additional or alternative examples, the tissue tension alert can be auditory.

[0087] Referring to FIG. 14, video analysis can calculate tissue tension by looking at tissue 1330 within region of interest 1370. Reference marks 1362, 1364, and 1366 provide video information regarding the location and orientation of device tip 1326 that is used to "look" for tension. Video analysis can look for tenting 1332 within tissue 1330 on one or both sides of the clamped end effector 1320. Tenting 1332 of tissue 1330 can indicate the presence and potentially the degree of tension. This information can then be provided to the surgeon via visual feedback and / or auditory feedback.

[0088] Figures 16 and 17 show two diagrams of tissue tension. Figure 16 shows the axial tissue tension, and Figure 17 shows the radial tissue tension. Video analysis can establish at least two points 1522, 1524 on the grasped tissue adjacent to the marker at a set relative distance using at least one fiducial marker such as fiducial marks 1362, 1364, or 1366 added to the device. Using these points 1522, 1524, the system can establish two lines 1526, 1528, one line 1526 extending towards the grasped tissue on the left side of the end effector and one line 1528 extending towards the grasped tissue on the right side. The two lines intersect at point 1512, and the angle between the two lines can be used to assess the degree to which the tissue is being pulled either axially or radially by the device. Figure 16 shows the axial tension, angle a1 indicates no tension, angle a2 indicates slight tension, and angle a3 indicates greater tension. Figure 17 shows the radial tension, angle b1 indicates no tension, angle b2 indicates slight tension, and angle b3 indicates more tension. As more tension is applied to the tissue, the degree to which angles "a" and "b" change increases. For example, the change from angle a1 to angle a2 is smaller than the change from angle a1 to a3. Similarly, the change from angle b1 to angle b2 is smaller than the change from angle b1 to b3.

[0089] Tissue tension information can be converted to the device user via visual (e.g., low - medium - high scale) or auditory (e.g., an alert tone is played if excessive tension is applied) indicators. Using relative changes across the extent of the video image, the progression of tension can be seen, such as how the angles and lengths of the edges change. This information can be provided to the surgeon in real - time for the surgeon to manage tissue tension with the aim of minimizing tissue tension during a surgical procedure.

[0090] Advanced bipolar sealing quality prediction Advanced Bipolar (ABP) tools are electrosurgical tools used in surgical procedures for soft tissue incision and vascular sealing, offering the major benefits of shortening OR time and minimizing blood loss. These tools use high-frequency current to generate the heat necessary for sealing, but the resulting temperature is generally not sufficient to cut tissue. These ABP tools further utilize an algorithm to ensure sealing integrity and a mechanically sharp knife, ultrasonic blade, or any other suitable blade for cutting tissue after sealing. Combining sealing and cutting into a single device enhances the versatility of ABP tools, thereby increasing their multi-usability, reducing the need for instrument exchange inside and outside the body, and enhancing surgical efficiency, ease of use, and workflow.

[0091] Incision and sealing most often result in a desirable outcome, but part of the activation of ABP tools can still potentially result in minor or major intraoperative bleeding. The surgeon can use information regarding the quality of the seal to help prevent or reduce bleeding and thus notify subsequent surgical procedures that reduce the patient's blood loss and improve surgical efficiency.

[0092] The present disclosure provides a solution that can predict in real time, before it occurs, poor seal quality that is likely to result in bleeding. These predictions can be made during or after energy activation using electrical and other measurements over the course of the activation, as well as additional device data. When insufficient seal quality is detected, the system can issue a warning (auditory, visual, or tactile, or a combination thereof) that enables the surgeon to determine what appropriate next step to take to avoid bleeding, such as not mechanically cutting the tissue with the blade. The surgeon may also choose to reapply energy before transection or perform other actions to prevent bleeding.

[0093] Sealing quality prediction is based on an inference model that predicts the likelihood of bleeding in near real-time, based on data from the generator. The inference module is based on a pattern recognition model trained on annotated real-world data from human surgeries. The module can be deployed on digital hardware in the operating room using direct access to device data.

[0094] In some embodiments, the system uses time-series electrical data (such as current, voltage, impedance, power, sealing cycle phase, total energy, etc.) over the entire activation of the instrument, as well as device manufacturing data (such as clamp force, multi-point jaw gap measurements, device electrical impedance, etc.) as a basis for making predictions regarding sealing quality. Information regarding sealing quality is communicated to the surgeon, who can select an appropriate response. The surgical workflow is described in the "Concept" section below.

[0095] Data is required to develop the algorithm, which is described in the "Data Collection and Annotation" section below. This data is pre-processed into features in real-time through several different mathematical transformations and scaling operations, which are described in the "Feature Generation" section below. The pre-processing can result in hundreds or thousands of features being fed into a machine learning model, which takes the input and generates a numerical prediction, such as a variable scale from 0 to 1 as an example. In one embodiment, a threshold value, such as >0.5 as an example, can be applied to this output to convert this output into a binary prediction for communication to the user. The machine learning model is described in the "Machine Learning ("ML") Model" section below.

[0096] In some embodiments, the threshold can be selected for a particular model in order to appropriately balance the sensitivity and specificity of the prediction. In order to optimize the effectiveness of the solution in the context of the surgical workflow, it is important to cancel out false positives (i.e., false alarms that lead to alarm fatigue) as well as false negatives (i.e., missed bleeding events that lead to bleeding). This will be described in the following "Metrics" section. Finally, the information must be communicated to the surgeon. The electrosurgical treatment system can provide its own interface or work with another system. This will be described in the following "Integration and Interface" section.

[0097] Concept As described above, in some embodiments, the system utilizes time-series electrical data (such as current, voltage, impedance, power, sealing cycle phase, total energy, etc.) over the entire activation of the instrument, as well as device manufacturing data (such as clamp force, multi-point jaw gap measurements, device electrical impedance, etc.) as a basis for making predictions regarding the sealing quality. Information regarding the sealing quality is communicated to the surgeon, and the surgeon can select an appropriate response.

[0098] Referring now to FIG. 18, a flowchart 2000 is provided in accordance with at least one aspect of the present disclosure. During a surgical procedure, the clamp jaws (plural) of electrosurgical tools such as surgical instruments 1204, 1206, 1208 can grip tissue (2002). The jaws (plural) and the gripped tissue can be viewed on a display such as display 135, display 711, or any other suitable display described elsewhere herein. Once the desired tissue is gripped, a generator such as generator module 140, RF energy source 794, or modular energy system 1000, or any other suitable energy generator described elsewhere herein, can provide RF energy to the electrode(s) of the electrosurgical tool, which can then apply the RF energy to the tissue to seal the tissue. As will be described in more detail below, based on the application, a control system such as processor module 132, processor 502, control circuit 760, or any other suitable control system described elsewhere herein, can predict the quality of the seal (2003) and determine an appropriate response. Based on the prediction, the control system can provide feedback to the surgeon, such as visual, audio, or tactile feedback, or a combination thereof, indicating the type quality of the seal.

[0099] In one aspect, when the control system predicts (2004) that the seal is a high-quality seal, the control system can provide feedback indicating this to the surgeon. Based on the high-quality seal metrics, the surgeon can confidently proceed with cutting (2006) the sealed tissue using a knife of an electrosurgical tool such as blade 768, ultrasonic blade 1249, ultrasonic blade 1228, or any other knife or blade described elsewhere in this specification. In various other embodiments, when the control system predicts (2004) that the seal is of high seal quality, the control system may not be able to provide feedback to the surgeon. The lack of feedback can inform the surgeon that the electrosurgical tool functioned as intended, i.e., that the applied seal was of high quality. The lack of feedback provides the advantage of preventing the surgeon from being distracted by feedback that would require corrective action. When the tissue is cut by the blade, the surgeon can proceed to the next surgical task (2008).

[0100] In one aspect, when the control system predicts (2010) that the seal is a low-quality seal, the control system can provide feedback indicating this to the surgeon. Based on the insufficient seal quality metrics, the surgeon can determine the appropriate course of action to take (2012). In one aspect, the surgeon can pause the operation of the electrosurgical tool (2014) and assess the sealed tissue before proceeding. In one aspect, the surgeon can release the tissue captured by the jaws of the electrosurgical tool (2016) and re-grasp the tissue or grasp a different tissue. In one aspect, the surgeon can apply additional treatment energy to the grasped tissue (2018) to further seal the tissue before proceeding. In one aspect, the surgeon can prepare an auxiliary tool such as a different energy instrument that can be used to effect hemostasis within the tissue (2020).

[0101] Data Collection and Annotation As described above, data is required to develop an algorithm that can implement the above-described predictor of encapsulation quality. To predict the quality of encapsulation, the control system can utilize a machine learning algorithm that is trained, optimized, and tested using data (both input and output data) obtained from various sources, such as clinical sources, pre-clinical sources, or bench-top sources, or combinations thereof. In one aspect, the more closely the data represents actual use, the greater the variability that can be introduced during training of the algorithm, and the more robustly the algorithm will perform during use in a surgical procedure.

[0102] In some embodiments, the data used to train the algorithm can be obtained from clinical, real-world sources. In one aspect, the data can be clinical data, real-world data (both input and output data) obtained from the same or similar hardware components used by humans. By utilizing clinical data, it becomes possible to train the algorithm using data that very closely matches what the algorithm will encounter during a surgical procedure. In one aspect, the input data can include, by way of example, time-series electrical data from an electrosurgical generator, internal system event data, and device manufacturing data.

[0103] In various embodiments, the output data includes outputs obtained from surgical videos that are recorded and then annotated in post-processing. Video annotation is the process of applying labels to a surgical video so that the algorithm can extract structured information from the video. In one aspect, labels such as those shown in Table 2050 of FIGS. 19A - 19C can be added to the video by an operator based on label definitions and the operator's experience. In various embodiments, the labels can be stored in memory within a storage array 134, a memory 504, or any other suitable memory described elsewhere in this specification. In various embodiments, the labels can be applied to the surgical video by an operator at an input interface such as a computer, a touch screen monitor, etc.

[0104] For predicting sealing quality, it is important to know whether the tissue bleeds after each activation, i.e., whether it is a "hemostatic outcome". Thus, in some embodiments, one of either a "bleeding" or "dryness" label is applied to each activation, which serves as the ground truth for the training algorithm. In some embodiments, the bleeding vs. dryness label is converted to a binary value for training the algorithm. In other embodiments, qualitative estimates of bleeding, such as dry, oozing, mild, severe, can be encoded as ordinal data for multi-class algorithm training. As an example, other labels obtained through video annotation, such as tissue thickness, tissue type, tissue adhesion, etc., can be used during training to assist multi-task learning. In one aspect, multi-task learning is a machine learning algorithm trained to predict multiple outputs and improves the accuracy of sealing quality prediction by leveraging additional tissue data not included in the bleeding / dryness label.

[0105] In some embodiments, the data used to train the algorithm can be obtained from bench-top tests. Augmenting real-world data with bench-top tissue data improves algorithm accuracy by expanding conditions outside of clinical use in humans. For example, creating bench-top data using devices with low and high tolerance components (at the edge or outside of tolerance) allows the algorithm to learn to better compensate for these factors and recognize patterns of out-of-tolerance devices in the electrical data, improving the robustness of the algorithm. This improves the accuracy of prediction even for devices with nominal component tolerances.

[0106] In some embodiments, the data used to train the algorithm can be obtained from preclinical live animal tests. Similar to adding the above-described bench-top test data, live animal data can further expand the training and test data sets. Adding animal data has the advantage of achieving good algorithm performance during development and then passing validation and verification activities.

[0107] As noted above, a machine learning model can be trained to predict multiple outputs using multi-task learning to improve the accuracy of seal quality prediction. Referring now to FIG. 20, a flowchart 2100 of multi-task learning during model training and validation is provided in accordance with at least one aspect of the present disclosure. A model 2102 can receive features (inputs) 2104 described in more detail below and output tissue hemostasis 2106, tissue type 2108, and whether there was tissue adhesion 2110. In various embodiments, these outputs can include outputs obtained from surgical videos that are recorded and then annotated in post-processing, as described above. In various embodiments, for tissue hemostasis 2106, a numerical value within a numerical range can be assigned according to the quality of the seal. For example, the numerical range can include a minimum value such as 0 and a maximum value such as 4. These outputs can be fed to a training module 2112 and then fed back to the model 2102 (i.e., backpropagation) to further train the model 2102 and improve the accuracy of seal quality prediction.

[0108] Referring now to FIG. 21, there is provided a flowchart 2200 of multi-task learning during validation, verification, and clinical use, according to at least one aspect of the present disclosure. The model 2202 can receive features (inputs) 2204 described in more detail below and can output hemostasis of tissue 2206, tissue type 2208, and whether there was tissue adhesion 2210. In various embodiments, for the hemostasis of tissue 2206, a numerical value within a numerical range can be assigned according to the quality of the seal. For example, the numerical range can include a minimum value such as 0 and a maximum value such as 4. According to the value, the system can perform a prediction 2212 of the seal quality.

[0109] Feature Generation As referenced above, the algorithm model can receive features (inputs) that can be used to train a machine learning algorithm.

[0110] In various embodiments, the features can include algorithm-based features. In one aspect, the ABP generator algorithm can be controlled using a composite load curve (CLC) look-up table 2300 shown in FIG. 22. In various embodiments, the look-up table 2300 can be stored in a memory such as the storage array 134, the memory 504, the memory 524, or any other suitable memory described herein. Given look-up table positions, CLC code (section) and index (subsection) values during activation are determined based on a combination of time and impedance. By segmenting the electrical data based on where in the control algorithm the activation is, it can be determined how well the algorithm is functioning.

[0111] For feature generation, the generator file is split into CLC codes and then further split into smaller chunks within each CLC code. This can be seen in FIG. 22, where each section 2302, 2304, 2306, 2308 is a new CLC code number and the vertical lines indicate how each CLC code is further segmented. Summary statistics (max, min, mean, std, max / min ratio) are then obtained from these smaller chunks and used as features for the machine learning model. This is done for combinations of the electrical parameters time, voltage, current, impedance, power, and energy (t, V, I, Z, P, E). In various embodiments, global (complete activation) summary statistics are also used as features. In various embodiments, the amount of time spent at each CLC index is also used as a feature. The resulting vectors are then combined and stacked to create one vector for each activation.

[0112] In various embodiments, the features can include wavelet-based features. In one aspect, wavelets (continuous and discrete) are a signal processing technique that obtains a "mother" wave, scales it, and transforms it over a given signal. Unlike sinusoidal waves that are not time-localized, wavelets are time-localized. This allows the wavelet transform to obtain time information in addition to frequency information. Since wavelets are time-localized, the signal can be multiplied by the wavelet at different positions in time, and this procedure is also known as convolution, as seen in the representation 2310 shown in FIG. 23. After this is done to the original (mother) wavelet, it can be scaled to be larger and the process can be repeated. The resulting transform 2320, shown in FIG. 24, is similar to the Fast Fourier Transform ("FFT"). However, it provides context for the signal in both the frequency domain and the time domain.

[0113] As described above, wavelets can include discrete wavelets and continuous wavelets. Discrete wavelet transform ("discrete wavelet transformation, DWT") and continuous wavelet transform ("continuous wavelet transformation, CWT") are taken from raw signals and used as features. In some embodiments, for both DWT and CWT, the mother wavelet used is "db4". In some embodiments, in the case of CWT, 128 scales of the mother wavelet are used, and the coefficients and frequencies resulting from the transformation are stacked, zero-padded, and an even-length feature array is created. In some embodiments, in the case of DWT, summary statistics and entropy calculations of the resulting coefficients are used instead of the raw coefficients. An example of the wavelet transform (power spectrum) of a signal is shown in graph 2330 of FIG. 25.

[0114] In various embodiments, the features can include time-based features. Referring to graph 2400 shown in FIG. 26, ABP electrical parameter voltage, current, impedance, power, and energy (V, I, Z, P, E) can be divided into "x" time-length segments 2402. Then, summary statistics (maximum, minimum, average, standard, maximum / minimum ratio) are taken from these segments. Then, as shown in graph 2400, the obtained vectors are stacked to create one vector.

[0115] In various embodiments, the features can include raw signal features. The raw signal from the generator can be analyzed into five vectors, namely voltage, current, impedance, power, and energy (V, I, Z, P, E), which can then be stacked on top of each other and zero-padded, with zeros added at the end of each signal to ensure that each vector for each activation is of the same length. In one aspect, there can be 540 numbers per signal, where 540 is the theoretical maximum length (5.4 seconds) of the activation.

[0116] Machine Learning (「ML」) Model As referred to above, preprocessing can result in hundreds or thousands of features that are fed into a machine learning model that takes in inputs and generates numerical predictions. In one aspect, using a machine learning (「ML」) model, the main idea of the sampling-based approach is to modify the distribution of events so that the rare class is well represented in the training samples. Given that bleeding cases represent only about 10% of all cases, various class balancing techniques are applied to the machine learning model.

[0117] In the case of undersampling, random samples are taken from the majority class, i.e., non-bleeding events. However, a potential problem associated with undersampling is that some useful non-bleeding instances may not be selected for training, resulting in a suboptimal classifier. In the case of oversampling, replications of the events are taken from the minority class, i.e., bleeding cases. However, a potential problem associated with oversampling is that this technique can lead to overfitting to noisy data that will be replicated multiple times. As a result, the generalization of the model becomes insufficient.

[0118] Accordingly, the present disclosure provides a hybrid approach of oversampling and undersampling, referred to herein as synthetic minority over-sampling technique ("SMOTE"), which creates artificial minority class data using feature space similarity. SMOTE is described in detail in a paper entitled "SMOTE: Synthetic Minority Over-sampling Technique" by Notsh Chawla et al., published in June 2002, which is hereby incorporated by reference in its entirety. The advantage of SMOTE is that it reduces overfitting caused by random oversampling because synthetic examples are generated instead of replicating instances. Further, there is no loss of information. In one aspect, referring to the representation 2430 shown in FIG. 29, SMOTE can be implemented by taking the difference between a sample point and one of its nearest neighbors, multiplying the difference by a random number between 0 and 1, and adding the multiplied number to the feature vector. This selects a random point along the line segment between two specific features.

[0119] Referring now to FIG. 27, a scatter plot 2410 of the dataset is provided, showing a large number of points (dark dots, "0") belonging to the majority class and a few points (light dots, "1") scattered for the minority class. Using SMOTE, the minority class is oversampled, the majority class is undersampled, and then plotted, resulting in the scatter plot 2420 shown in FIG. 28.

[0120] Using multiple approaches including time-based, algorithms, wavelets, and raw features, 224 summary statistics were obtained. To extract the hidden (potentially low-dimensional) structure from the high-dimensional feature space, principal component analysis ("Principal Component Analysis, PCA") was performed. PCA is an orthographic projection or transformation of the data into a (possibly lower-dimensional) subspace such that the variance of the projected data is maximized. Identifying the axes, known as principal component analysis, can be obtained by using classical matrix calculation tools (eigenvalue decomposition or singular value decomposition). Referring to FIG. 30, PCA is shown to transform the features from one state 2440 to a second state 2442 such that only the latent features need to be preserved.

[0121] In some embodiments, for the ML model, a prediction model including a mixture of linear and tree-based classifiers can be used. The model framework has an input of a data frame of size [n, 224], where 224 represents the summary statistics of the electrical signals obtained from feature engineering. In one embodiment, the preprocessing is achieved, by way of example, using SMOTE or PCA, both of which are described above. The tissue classification included bleeding or drying.

[0122] For logistic regression, a binary classification model that uses the logistic (sigmoid) function to model probabilities is used, which includes a regularization model with alpha equal to 0.5 and beta equal to 0.5. Hyperparameter tuning was utilized to obtain the optimal values of alpha and beta. The support vector machine ("SVM") algorithm was used to find a hyperplane in an N-dimensional space (N is the number of features) that clearly classifies the data points. The linear SVM was fitted using L2 regularization. The radial kernel function was utilized to model non-linearity. Referring to FIG. 31, Xgboost2450, a machine learning library constructed around an efficient implementation form of the boot for a tree model (such as GBM), was utilized. Xgboost allows the model to be tuned back to multiple parameters such as the number of trees (300), maximum depth (3, 6, 8), metric (binary Logloss), L1 regularization (0, 1, 2), L2 regularization (1, 2), number of iterations (100, 150, 250), and boosting type ("Dart", "gbtree"). Further information on XGBoost is described in the paper titled "Comparative Analysis of Artificial Neural Network and XGBoost Algorithm for PolSAR Image Classification" by Nimra Menon et al., published in November 2019, which is hereby incorporated by reference in its entirety into this specification.

[0123] Another model used is a CNN model, which is a deep learning classification model that can be trained to classify patterns by extracting useful aspects of signals through learnable weights and biases that describe multiple layers of convolution. In one aspect, the 1D CNN model can include an input of five [1×540] vectors that describe an electrical signal. The preprocessing can be a [1×7] median filter and standard scaling. Referring to FIGS. 32A and 32B, the model architecture 2460 can include Conv1D(8)>Dropout(0.2)>Max(2)>Conv1D(16)>Dropout(0.2)>Max(2)>Conv1D(32)>Dropout(0.2)>Max(0.2)>Conv1D(32)>Dropout(0.2)>Max(0.2)>Flattend>Dense(100)>Dropout(0.2)>Dense(2).

[0124] Another model used is a CNN+Dense multi-head architecture model, which is a deep learning classification model that can be trained to classify patterns by extracting useful aspects of signals through learnable weights and biases that describe multiple layers of convolution and combining this information with other inputs. These other inputs are processed by a neural network having several densely connected layers. The outputs from the multiple heads are combined and processed by a final stack of dense layers.

[0125] In one aspect, the model can include an input of two [1×540] vectors that describe an electrical signal and one [1×5] vector that describes a mechanical parameter. The preprocessing can be a [1×7] median filter, standard scaling for the electrical signal, and standard scaling for the mechanical parameter vector.

[0126] Metrics As described above, the threshold can be selected for a particular model in order to appropriately balance the sensitivity and specificity of the prediction. In order to optimize the effectiveness of the solution in the context of the surgical workflow, it is important to offset false positives (i.e., false alarms leading to alarm fatigue) as well as false negatives (i.e., missed bleeding events leading to bleeding).

[0127] Referring now to FIG. 33, a confusion matrix table 2470 according to at least one aspect of the present disclosure is provided that can be used to compare predicted outcomes to actual outcomes. In one aspect, the confusion matrix can be utilized when training an ML algorithm. For example, if it is predicted that the tissue is dry and it is actually dry, a true negative designation “TN” is provided. If the tissue is dry and it is predicted that it is actually bleeding, a false negative designation “FN” is provided. If the tissue is bleeding and it is predicted that it is actually dry, a false positive designation “FP” is provided. If the tissue is bleeding and it is predicted that it is actually bleeding, a true positive designation “TP” is provided.

[0128] The accuracy of the system is determined by the ratio of true results among the total number of cases examined. The formula for accuracy is provided as the number of TP and TN designations divided by the total number of actual cases. In this case, considering that the data is imbalanced (about 10% bleeding events), accuracy cannot correctly capture the accuracy and sensitivity values for bleeding events.

[0129] The ratio of predicted positives to those that are truly positive determines the precision of the system. The formula for precision is provided as the number of TP designations divided by the sum of TP and FP designations. In this case, the classifier showed a high accuracy value but was unable to predict all bleeding.

[0130] The ratio of actually positive cases that are correctly classified determines the recall of the system. The recall equals is provided as the number of TP designations divided by the sum of TP and FN designations. Optimizing the classifier model based on this metric resulted in a low accuracy score.

[0131] The F1 score helps to obtain a reproduction accuracy score within the clinical survival threshold. The F1 score is a number between 0 and 1 and is the harmonic mean of precision and recall. The formula for the F1 score is provided as 2 * recall * precision / (recall + precision).

[0132] The area under the curve ("Area under Curve, AUC") of the receiver operating characteristic ("ROC") curve indicates how well the probability from the positive class is separated from the negative class. This is plotted between the true positive rate value and the false positive rate value, as shown in the exemplary graph 2480 of FIG. 34. Considering that the dataset is skewed, the AUC ROC curve continues to have a high score even though the accuracy is low for the minority class. For example, if the model is overfitted to the majority class, it may not be able to predict the minority class, but still result in a high true positive rate ("TPR") and a low false positive rate ("FPR") value. This results in a high AUC score.

[0133] In conversations with various surgeons across North America and APAC, target metrics were developed. While discussing the acceptable false positive rate (alarm fatigue) and false negative rate (missed bleeding events) with the surgeons, the confusion matrix 2490 shown in FIG. 35 was used to facilitate the discussion and help reach numerical target values. After the overall concept of seal quality prediction was explained to the surgeons, it was explained that there was a trade-off between the false positive rate and the false negative rate for a given machine learning model. The surgeons were then walked through four confusion matrices 2500, 2510, 2520, 2530 shown in FIGS. 36A - 36D with various false positive rates and false negative rates and asked to give their opinions and which scenarios were preferred.

[0134] After all considerations were completed, responses were aggregated to determine the clinical survival threshold. It was determined that the "alarm fatigue", i.e., the tolerance for false positives, is approximately 2-4 times higher than the "missed bleeding events", i.e., false negatives.

[0135] Integration and Interface As referred to above, information must be communicated to the surgeon so that the surgeon can determine an appropriate course of action once the quality of the seal is predicted. Communicating the prediction of seal quality can be done in many ways, and a balance must be struck between distracting or interfering with the surgeon, while minimizing the potential for information loss.

[0136] In one aspect, it is advantageous to display an unobtrusive message on a monitor such as monitor 14, the primary wrap monitor, or any other monitor or display described elsewhere in this specification when a poor seal is predicted, and to display nothing when a good seal is predicted. Considering that good seals occur most of the time, not displaying seal quality information in these cases helps to minimize information overload and distraction.

[0137] In various embodiments, the visual methods of displaying poor seal quality (on-screen displays 2540, 2550 as shown in FIGS. 37 and 38) may comprise some or all of binary feedback (good vs. bad, or sealed vs. unsealed), continuous feedback (seal quality percentage, 0-100% that can be visually indicated by a color change with a recommended threshold), and / or discrete feedback (red, yellow, green indicator icons).

[0138] A first embodiment of a sealing quality prediction system for a high bipolar device 2600 is provided in FIG. 39, and the sealing quality inference occurs within a high bipolar generator 2610, similar to the generator module 140, energy source 794, or modular energy system 1000, or any other generator described elsewhere herein. The inference output is communicated to a real-time image processing unit 2620, which is similar to the processor module 132, processor 502, control circuit 760, or any other suitable image processing unit described elsewhere herein, and overlays the sealing quality information on the laparoscopic video image 2650 captured using the laparoscopic camera 2630 and the video camera processing unit 2640. The advantage of this configuration is that it is compatible with any third-party laparoscopic video system.

[0139] A second embodiment of a sealing quality prediction system for a high bipolar device 2600 is provided in FIG. 40. The second embodiment is similar to the first embodiment described above, except that the high bipolar generator 2610 transmits the electrophysiological data and activation metadata to a real-time inference and image processing unit (RTIIP) 2700 instead of the real-time image processing unit 2620. The sealing quality inference is performed within the RTIIP 2700, and then the RTIIP overlays the sealing quality information on the laparoscopic video image. The advantage of this configuration is that the RTIIP 2700 can be provided as a stand-alone product compatible with the energy system (generator and device). The inference function can be provided as a software add-on. This can also eliminate the need for design changes to the energy system, which can have the advantage of regulatory and R&D costs.

[0140] In various embodiments, the system can also provide an audible means for communicating seal quality information. In some embodiments, an audio feedback module, such as any suitable audio feedback module described elsewhere herein, can emit a first acoustic effect during the sealing phase of activation of the device, a secondary acoustic effect in the case of a high-quality seal, and a third acoustic effect in the case of a low-quality seal. Various other embodiments are envisioned where the acoustic effect is not emitted in the case of a high seal quality so as not to distract the surgeon. Rather, the audio feedback module provides feedback only in the case of a poor seal.

[0141] In various embodiments, an energy device (e.g., instrument 112, surgical instrument 790, or surgical instruments 1204, 1206, 1208) can be designed using a haptic device (e.g., a piezoelectric motor) that makes a noise or clicks when predicting inadequate seal quality. Various embodiments are envisioned where the haptic device provides haptic feedback in the case of a high-quality seal. Various other embodiments are envisioned where the haptic device does not provide haptic feedback in the case of a high-quality seal so as not to distract the surgeon.

[0142] The entire disclosures of U.S. Patent Nos. 10,624,691, 10,842,523, 11,291,510, 11,311,342, 11,259,830, 11,304,699, 11,109,866, 11,298,129, 11,229,437, 11,241,235, and U.S. Patent Application Publication Nos. 2019 / 0206562, 2019 / 0200981, 2019 / 0208641, 2019 / 0201594, 2019 / 0201045, 2019 / 0200844, 2019 / 0201136, 2019 / 0206569, 2019 / 0201137, 2019 / 0125459, 2019 / 0125458, 2019 / 0125455, 2019 / 0125454, 2019 / 0274706, 2019 / 0201046, 2019 / 0201047, 2019 / 0104919, 2019 / 0125361, 2019 / 0200977, 2019 / 0298350, 2019 / 0206564, 2019 / 0206565, 2020 / 0100830, 2020 / 0078070, 2020 / 0078076, 2020 / 0078106, 2020 / 0100825, 2021 / 0196334, 2021 / 0196354, 2021 / 0196302, 2020 / 0345353, and 2022 / 0031315 are hereby incorporated by reference in their entireties.

[0143] Although various devices have been described herein in connection with specific embodiments, modifications and changes may be implemented to those embodiments. Specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Thus, specific features, structures, or characteristics illustrated or described with respect to one embodiment may be combined, without limitation, in whole or in part with the features, structures, or characteristics of one or more other embodiments. Also, although materials have been disclosed with respect to specific components, other materials may be used. Further, in accordance with various embodiments, a single component may be replaced with a plurality of components, or a plurality of components may be replaced with a single component, to perform a given function. The foregoing description and the following claims are intended to embrace all such modifications and variations.

[0144] The devices disclosed herein can be designed to be discarded after a single use or can be designed to be used multiple times. However, in either case, the device can be reconditioned for reuse after at least one use. Reconditioning can include, but is not limited to, any combination of a disassembly step of the device, followed by a cleaning step or replacement step of specific parts of the device, and a subsequent reassembly step of the device. Specifically, a reconditioning facility and / or surgical team can disassemble the device, clean and / or replace specific parts of the device, and then reassemble the device for subsequent use. One of ordinary skill in the art will understand that various techniques for disassembly, cleaning / replacement, and reassembly are available for reconditioning the device. The use of such techniques and the resulting reconditioned device are all within the scope of this application.

[0145] The devices disclosed herein can be processed prior to surgery. First, a new or used instrument is obtained and may be cleaned as necessary. The instrument can then be sterilized. In one sterilization technique, the instrument is placed in a closed and sealed container such as a plastic bag or a TYVEK bag. The container and the instrument can then be placed in a radiation field that can penetrate the container, such as gamma rays, x-rays, and / or high-energy electrons. The radiation can kill bacteria on the instrument and within the container. The sterilized instrument can then be stored within the sterilization container. The sealed container can keep the instrument in a sterilized state until it is opened at a medical facility. The device can also be sterilized using any other technique known in the art, including but not limited to beta rays, gamma rays, ethylene oxide, hydrogen peroxide plasma, and / or steam.

[0146] Although the invention has been described as having representative designs, the invention may be further modified within the spirit and scope of the present disclosure. Accordingly, this application is intended to cover any variations, uses, or adaptations of the invention using its general principles.

[0147] The foregoing detailed description has described various forms of devices and / or processes using block diagrams, flow diagrams, and / or examples. As should be understood by those skilled in the art, as long as such block diagrams, flow diagrams, and / or examples include one or more functions and / or operations, each function and / or operation included in such block diagrams, flow diagrams, and / or examples can be implemented individually and / or collectively by a variety of hardware, software, firmware, or virtually any combination thereof. It should be understood by those skilled in the art that all or part of some aspects of the forms disclosed herein can be implemented equivalently on an integrated circuit as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or in virtually any combination thereof. Designing circuits and / or writing software and / or firmware code is, in view of the present disclosure, within the skill of those skilled in the art. It should be understood by those skilled in the art that the mechanisms of the subject matter described herein can be distributed in a variety of forms as one or more program products, and the specific forms described herein apply regardless of the particular type of signal carrier medium used to actually effect the distribution.

[0148] Instructions used to program logic to implement various disclosed aspects may be stored in system memory such as Dynamic Random Access Memory (DRAM), cache, flash memory, or other storage. Further, the instructions may be distributed via a network or by other computer-readable media. Thus, a machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), but is not limited to tangible machine-readable storage such as floppy disks, optical disks, compact disks, Compact Disc Read Only Memory (CD-ROM), and magneto-optical disks, Read-Only Memory (ROM), Random Access Memory (RAM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), magnetic or optical cards, flash memory, or information transmission via the Internet via electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Thus, a non-transitory computer-readable medium can include any type of tangible machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0149] When used in any aspect of this specification, the terms "control circuit" or "control system" may refer to, for example, a hardwired circuit, a programmable circuit (e.g., a computer processor, a processing unit, a processor, a microcontroller, a microcontroller unit, a controller, a Digital Signal Processor (DSP), a Programmable Logic Device (PLD), a Programmable Logic Array (PLA), or a Field Programmable Gate Array (FPGA) that includes one or more individual instruction processing cores), a state machine circuit, firmware that stores instructions executed by a programmable circuit, and any combination thereof. The control circuit may be embodied, collectively or individually, as, for example, a circuit that forms part of a larger system such as an Integrated Circuit (IC), an Application-Specific Integrated Circuit (ASIC), a System on-Chip (SoC), a desktop computer, a laptop computer, a tablet computer, a server, or a smartphone.Accordingly, as used herein, "control circuit" includes, but is not limited to, an electrical circuit having at least one discrete electrical circuit, an electrical circuit having at least one integrated circuit, an electrical circuit having at least one application-specific integrated circuit, an electrical circuit forming a general-purpose computing device configured by a computer program (e.g., a general-purpose computer configured by a computer program that at least partially executes the processes and / or devices described herein, or a microprocessor configured by a computer program that at least partially executes the processes and / or devices described herein), an electrical circuit forming a memory device (e.g., in the form of a random access memory), and / or an electrical circuit forming a communication device (e.g., a modem, a communication switch, or an optical-electrical facility). Those skilled in the art will recognize that the subject matter described herein may be implemented in analog form, digital form, or some combination thereof.

[0150] As used in any aspect of this specification, the term "logic" can refer to an app, software, firmware, and / or circuitry for performing any of the foregoing operations. The software may be embodied as a software package, code, instructions, instruction sets, and / or data recorded on a non-transitory computer-readable storage medium. The firmware may be embodied as code, instructions, or instruction sets within a memory device, and / or hard-coded (e.g., non-volatile) data.

[0151] As used in any aspect of this specification, the terms "component", "system", "module" can refer to a computer-related entity that is either hardware, a combination of hardware and software, software, or software in execution.

[0152] As used in any aspect of this specification, "algorithm" refers to a self-collision-free sequence of steps leading to a desired result, and "step" refers to an operation of a physical quantity and / or logical state that, although not necessarily required, can take the form of an electrical or magnetic signal capable of being stored, transferred, combined, compared, and otherwise manipulated. It is common practice to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc. These and similar terms may be associated with appropriate physical quantities or may simply be convenient labels applied to these quantities and / or states.

[0153] Unless otherwise expressly defined, as will be apparent from the foregoing disclosure, throughout the foregoing disclosure, the use of the terms "processing," "computing," "calculating," "determining," "displaying" contemplates actions and processes of a computer system or similar electronic computing device that operate on and transform data represented as physical (electronic) quantities within the registers and memories of the computer system into other data similarly represented as physical quantities within the memories or registers of the computer system or other such information storage, transmission, or display devices.

[0154] One or more components may be referred to herein as "to", "configurable to", "operable / operative to", "adapted / adaptable", "able to", "conformable / conformed to", etc. Those skilled in the art will understand that "to" generally may include components in an active state and / or components in an inactive state and / or components in a standby state, unless otherwise construed in context.

[0155] Those skilled in the art will generally understand that the terms used herein, and in particular those used in the appended claims (e.g., the body of the appended claims), are generally intended to be "open" terms (e.g., the term "including" should be construed as "including but not limited to", the term "having" should be construed as "having at least", the term "includes" should be construed as "includes but is not limited to", etc.). It will further be understood by those skilled in the art that where a specific number is intended in an introduced claim recitation, such intent is clearly recited in the claim, and where there is no such recitation, there is no such intent. For example, for the sake of illustration, the following appended claims may include introductory phrases such as "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed as suggesting that any particular claim containing such an introduced claim recitation is limited to a claim containing only one such recitation, even where the claim recitation is introduced by the indefinite article "a" or "an" and the claim contains both an introductory phrase such as "one or more" or "at least one" and the indefinite article "a" or "an" within the same claim (e.g., "a" and / or "an" should generally be construed as meaning "at least one" or "one or more"). The same applies when introducing a claim recitation using a definite article.

[0156] Even when a specific number is explicitly stated in the introduced claim description, it should be recognized by those skilled in the art that such a description should typically be interpreted as meaning at least the stated number (for example, in the case of a simple description of "two items" without any other modifiers, it generally means at least two items, or two or more items). Further, when a notation similar to "at least one of A, B, and C, etc." is used, generally, such a syntax is intended in a sense that those skilled in the art will understand it (for example, "a system having at least one of A, B, and C" includes, without limitation, a system having only A, only B, only C, both A and B, both A and C, both B and C, and / or all of A, B, and C). When a notation similar to "at least one of A, B, or C, etc." is used, generally, such a syntax is intended in a sense that those skilled in the art will understand it (for example, "a system having at least one of A, B, or C" includes, without limitation, a system having only A, only B, only C, both A and B, both A and C, both B and C, and / or all of A, B, and C). Further, typically, any disjunctive word and / or phrase representing two or more alternative terms should be understood to be intended to include one of those terms, any of those terms, or both of those terms, whether in the specification, in the claims, or in the drawings, unless the context indicates otherwise. For example, the phrase "A or B" will typically be understood to include the possibilities of "A" or "B" or "A and B".

[0157] Regarding the appended claims, those skilled in the art will understand that the recited operations in this specification can generally be performed in any order. Also, although the flowcharts of various operations are shown in a sequence (s), it will be understood that the various operations may be performed in an order other than that shown, or may be performed simultaneously. Such examples of alternative orderings may include, unless otherwise interpreted in context to have some other meaning, repetition, interleaving, interruption, reordering, incremental, preparatory, additional, simultaneous, reverse, or other different orderings. Further, terms such as "responsive to", "related to", or other past tense adjectives are generally not intended to exclude such variations, unless otherwise interpreted in context to have some other meaning.

[0158] Any reference to "one aspect", "an aspect", "an exemplification", "one exemplification", etc. is worth noting in that it means that the particular feature, structure, or characteristic described in relation to that aspect is included in at least one aspect. Thus, the phrases "in one aspect", "in an aspect", "in an exemplification", and "in one exemplification" that appear in various places throughout this specification do not necessarily all refer to the same aspect. Further, a particular feature, structure, or characteristic may be combined in any suitable manner in one or more aspects.

[0159] Any patent application, patent, non-patent publication, or other disclosure material referenced in this specification and / or listed in any application data sheet is incorporated herein by reference to the extent that the incorporated material is not inconsistent with this specification. The disclosure clearly set forth herein, in itself and to the extent necessary, shall supersede any conflicting description incorporated herein by reference. Although it is referred to as being incorporated herein by reference, any content, or portions thereof, that conflict with the current definitions, opinions, or other disclosure content described herein shall be incorporated only to the extent that no conflict arises between the incorporated content and the current disclosure content.

[0160] The terms “comprise,” “comprises,” “comprising” (and any other word forms of comprise), “have,” “has,” “having” (and any other word forms of have), “include,” “includes,” “including” (and any other word forms of include), and “contain,” “contains,” “containing” (and any other word forms of contain) are open-ended conjunctive verbs. As a result, a system that “comprises,” “has,” “includes,” or “contains” one or more elements possesses those one or more elements but is not limited to possessing only those one or more elements. Similarly, an element of a system, device, or apparatus that “comprises,” “has,” “includes,” or “contains” one or more features has those one or more features but is not limited to having only those one or more features.

[0161] As used herein, the terms "substantially", "about" or "approximately" mean, unless otherwise specified, an acceptable error with respect to a particular value as determined by one of ordinary skill in the art, which depends in part on the method by which the value is measured or determined. In certain embodiments, the terms "substantially", "about" or "approximately" mean one, two, three, or four standard deviations. In certain embodiments, the terms "substantially", "about" or "approximately" mean within 50%, 20%, 15%, 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, or 0.05% of a given value or range.

[0162] In summary, many benefits resulting from the use of the concepts described herein have been described. The foregoing description of one or more forms is presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Modifications or variations are possible in light of the above teachings. One or more forms are selected and described in order to illustrate the principles and practical applications thereof, thereby enabling one of ordinary skill in the art to utilize the various forms in various modifications as suitable for the particular uses contemplated. It is intended that the scope be defined by the claims presented with this specification.

[0163] 〔Embodiment〕 (1) A surgical device, An end effector, A first jaw, A second jaw, and an end effector comprising the same, A first sensor for detecting tissue disposed between the first jaw and the second jaw, A second sensor for detecting the end effector in a closed configuration, A third sensor for detecting movement of the end effector, A control circuit communicatively coupled to the first sensor, the second sensor, and the third sensor, Comprising, The control circuit includes a processor and a memory. When executed by the processor, the memory causes the control circuit to determine, based on first sensor data, that the end effector is in a closed configuration; determine, based on second sensor data, the presence of tissue disposed between the first jaw and the second jaw; monitor, based on third sensor data, the movement of the end effector in the closed configuration with tissue present between the first jaw and the second jaw; detect movement of the end effector outside a predetermined range based on the movement; provide feedback data based on the detected movement of the end effector. A surgical device that stores instructions. (2) The surgical device according to embodiment 1, wherein when executed by the processor, the memory stores further instructions that cause the control circuit to calculate the tension on the tissue based on the movement. (3) The surgical device according to embodiment 1 or 2, wherein the feedback is visual. (4) The surgical device according to embodiment 3, wherein the visual feedback is superimposed on a display image of the surgical site. (5) The surgical device according to any one of embodiments 1 to 4, wherein the feedback is auditory.

[0164] (6) The surgical device according to any one of embodiments 1 to 5, wherein the first jaw includes a clamp arm and the second jaw includes an ultrasonic blade. (7) The surgical device according to any one of embodiments 1 to 6, wherein the first jaw includes an anvil and the second jaw includes a staple cartridge. (8) A surgical device, an end effector, comprising a first jaw, and a second jaw. A first sensor for detecting tissue disposed between the first jaw and the second jaw; A second sensor for detecting that the end effector is in a closed configuration; A first reference mark; A second reference mark; A control circuit communicatively coupled to the first sensor, the second sensor, and a camera; Comprising; The control circuit includes a processor and a memory, and when the memory is executed by the processor, causes the control circuit to, Receive video data of the surgical site from the camera, Determine that the end effector is in the closed configuration based on first sensor data, Determine the presence of tissue disposed between the first jaw and the second jaw based on second sensor data, Determine the position of the device tip within the video data based on the first reference mark and the second reference mark, Determine a region of interest within the video data based on the position of the device tip within the video data, Analyze the region of interest of the end effector in the closed configuration with tissue present between the first jaw and the second jaw, Determine the tension on the tissue based on the analysis, Provide feedback based on the tension Instructions. A surgical device that stores. (9) The memory, when executed by the processor, causes the control circuit to store further instructions for detecting movement of the end effector outside a predetermined range based on the analysis, the surgical device according to embodiment 8. (10) The memory, when executed by the processor, causes the control circuit to store further instructions for determining the device type based on the first reference mark and the second reference mark, the surgical device according to embodiment 8 or 9.

[0165] (11) The feedback is visual, a surgical device according to any one of embodiments 8 to 10. (12) The visual feedback is superimposed on a display image of the surgical site, a surgical device according to embodiment 11. (13) The feedback is auditory, a surgical device according to any one of embodiments 8 to 12. (14) The first jaw comprises a clamp arm, and the second jaw comprises an ultrasonic blade, a surgical device according to any one of embodiments 8 to 13. (15) The first jaw comprises an anvil, and the second jaw comprises a staple cartridge, a surgical device according to any one of embodiments 8 to 14.

[0166] (16) A surgical system, A surgical instrument comprising an end effector for capturing tissue, the end effector comprising an electrode for applying radio frequency (RF) energy to the tissue captured by the end effector, a surgical instrument, An RF energy source for providing RF energy to the electrode, A control circuit, Transmitting a control signal to the RF energy source, the control signal causing the RF energy source to provide RF energy to the electrode and apply a seal to the tissue captured by the end effector, Predicting the quality of the seal, And providing feedback to a user based on the prediction, a control circuit for performing. A surgical system comprising. (17) To predict the quality of the seal, the control circuit Generates a value associated with the seal, Compares the value with a seal threshold, The surgical system according to embodiment 16, wherein the control circuit provides feedback to the user based on the result of the comparison in order to provide feedback to the user based on the prediction. (18) The surgical system according to embodiment 17, wherein the control circuit refrains from providing feedback based on the value reaching or exceeding the sealing threshold value. (19) The surgical system further includes a display, and the control circuit transmits a signal to the display based on the value falling below the sealing threshold value, and the feedback includes visual feedback on the display, and the visual feedback is based on the signal. The surgical system according to embodiment 17 or 18. (20) The surgical system further includes an audio feedback module, and the control circuit transmits a signal to the audio feedback module based on the value falling below the sealing threshold value, and the feedback includes audio feedback via the audio feedback module, and the audio feedback is based on the signal. The surgical system according to any one of embodiments 17 to 19.

[0167] (21) The surgical system further includes a tactile feedback module, and the control circuit transmits a signal to the tactile feedback module based on the value falling below the sealing threshold value, and the feedback includes tactile feedback via the tactile feedback module, and the tactile feedback is based on the signal. The surgical system according to any one of embodiments 17 to 20. (22) The surgical system according to any one of embodiments 16 to 21, wherein the RF energy source includes the control circuit. (23) The surgical system according to any one of embodiments 16 to 22, further including a processing unit including the control circuit.

Claims

1. A surgical device, An end effector, A first jaw, A second jaw, and an end effector comprising the same, A first sensor for detecting tissue disposed between the first jaw and the second jaw, A second sensor for detecting the end effector in a closed configuration, A third sensor for detecting movement of the end effector, A control circuit communicatively coupled to the first sensor, the second sensor, and the third sensor, Comprising, The control circuit includes a processor and a memory, and when the memory is executed by the processor, the control circuit is caused to, Based on first sensor data, determine that the end effector is in a closed configuration, Based on second sensor data, determine the presence of tissue disposed between the first jaw and the second jaw, Based on third sensor data, monitor the movement of the end effector in the closed configuration with tissue present between the first jaw and the second jaw, Based on the movement, detect movement of the end effector outside a predetermined range, Provide feedback data based on the detected movement of the end effector A surgical device that stores instructions.

2. The memory stores further instructions that, when executed by the processor, cause the control circuit to calculate a tension on the tissue based on the movement, according to the surgical device of claim 1.

3. The feedback is visual, according to the surgical device of claim 1 or 2.

4. The visual feedback is superimposed on a display image of the surgical site, according to the surgical device of claim 3.

5. The feedback is auditory, according to the surgical device of claim 1.

6. The first jaw comprises a clamp arm, and the second jaw comprises an ultrasonic blade, according to the surgical device of claim 1.

7. The first jaw comprises an anvil, and the second jaw comprises a staple cartridge, according to the surgical device of claim 1.

8. A surgical device, An end effector, A first jaw, A second jaw, and an end effector comprising the same, A first sensor for detecting tissue disposed between the first jaw and the second jaw, A second sensor for detecting that the end effector is in a closed configuration, A first reference mark, A second reference mark, A control circuit communicatively coupled to the first sensor, the second sensor, and the camera, Comprising, The control circuit includes a processor and a memory, and when the memory is executed by the processor, the control circuit causes, Receive video data of the surgical site from the camera, On the first sensor data, determine that the end effector is in the closed configuration, Based on the second sensor data, determine the presence of tissue disposed between the first jaw and the second jaw, Based on the first reference mark and the second reference mark, determine the position of the device tip within the video data, Based on the position of the device tip within the video data, determine a region of interest within the video data, Analyze the region of interest of the end effector in the closed configuration with tissue present between the first jaw and the second jaw, Determine the tension on the tissue based on the analysis, Provide feedback based on the tension A surgical device that stores instructions.

9. The memory stores further instructions that, when executed by the processor, cause the control circuit to detect movement of the end effector outside a predetermined range based on the analysis, according to the surgical device of claim 8.

10. The memory stores further instructions that, when executed by the processor, cause the control circuit to determine the device type based on the first reference mark and the second reference mark, according to the surgical device of claim 8 or 9.

11. The feedback is visual, according to the surgical device of claim 8.

12. The visual feedback is superimposed on a display image of the surgical site, according to the surgical device of claim 11.

13. The feedback is auditory, according to the surgical device of claim 8.

14. The first jaw includes a clamp arm, and the second jaw includes an ultrasonic blade, according to the surgical device of claim 8.

15. The first jaw includes an anvil, and the second jaw includes a staple cartridge, according to the surgical device of claim 8.

16. A surgical system, A surgical instrument comprising an end effector for capturing tissue, the end effector comprising an electrode for applying radio frequency (RF) energy to the tissue captured by the end effector, a surgical instrument, An RF energy source for providing RF energy to the electrode, A control circuit, Transmitting a control signal to the RF energy source, the control signal causing the RF energy source to provide RF energy to the electrode and apply a seal to the tissue captured by the end effector, Predicting the quality of the seal, A control circuit that performs providing feedback to the user based on the prediction, A surgical system comprising.

17. To predict the quality of the seal, the control circuit Generates a value associated with the seal, Compares the value to a seal threshold, To provide feedback to the user based on the prediction, the control circuit provides feedback to the user based on the result of the comparison. The surgical system according to claim 16.

18. The control circuit refrains from providing feedback based on the value reaching or exceeding the seal threshold. The surgical system according to claim 17.

19. The surgical system further comprises a display, and the control circuit transmits a signal to the display based on the value being below the seal threshold. The feedback includes visual feedback on the display, and the visual feedback is based on the signal. The surgical system according to claim 17 or 18.

20. The surgical system further comprises an audio feedback module, and the control circuit transmits a signal to the audio feedback module based on the value being below the seal threshold. The feedback includes audio feedback via the audio feedback module, and the audio feedback is based on the signal. The surgical system according to claim 17.

21. The surgical system further comprises a tactile feedback module, and the control circuit transmits a signal to the tactile feedback module based on the value being below the sealing threshold value, and the feedback includes tactile feedback via the tactile feedback module, and the tactile feedback is based on the signal, the surgical system according to claim 17.

22. The surgical system according to claim 16, wherein the RF energy source comprises the control circuit.

23. The surgical system according to claim 16, further comprising a processing unit comprising the control circuit.

Images