Systems and methods for machine learning control of surgical devices

Abstract

A computer-implemented method for controlling a surgical device includes accessing raw data captured by sensors of the surgical device during a procedure; filtering the original data by using a filter; generating difference data based on a difference between the raw data and the filtered data; generating zero-crossing data based on determining a point in time at which a magnitude of the difference data last crosses a zero magnitude value from a non-zero magnitude value to a non-zero magnitude value of an opposite sign; providing the zero-crossing data as input to a machine learning classifier; and predicting a probability of an end stop point based on the machine learning classifier. The end stop point includes a point in time when a knife of the surgical device pauses cutting tissue.

Description

Technical Field

[0001] The present disclosure relates to surgical devices. More particularly, the present disclosure relates to handheld electromechanical surgical systems for performing surgical procedures. Background Art

[0002] One type of surgical device is a linear clamping, cutting and stapling device. Such devices may be used in surgical procedures to remove cancerous or abnormal tissue from the gastrointestinal tract. Conventional linear clamping, cutting and stapling instruments include a pistol grip structure with an elongated shaft and a distal portion. The distal portion includes a pair of scissor-type gripping elements that clamp tissue, such as the open end of a tubular tissue. In this device, one of the two scissor-type gripping elements, such as an anvil portion, moves or pivots relative to the overall structure, while the other gripping element remains fixed relative to the overall structure. The actuation of this shearing device (pivoting of the anvil portion) is controlled by a grip trigger maintained in the handle.

[0003] In addition to the shearing device, the distal portion also includes a suturing mechanism. The fixed gripping element of the shearing mechanism includes a staple cartridge receiving area and a mechanism for driving the staples upward through the tubular tissue to abut the anvil portion to seal the previously opened end. The shearing element may be integrally formed with the shaft, or may be detachable so that various shearing and suturing elements are interchangeable.

[0004] In manual surgical devices, the user needs to use two hands to position the stapler in the desired articulation position. The user also needs to squeeze and release the handles multiple times (depending on the length of the loading unit) to clamp the loading unit and advance the knife along the loading unit. When the end stop is reached, the handles can no longer be squeezed. After the firing is completed, a second hand is often required to retract the knife and release the sutured tissue.

[0005] The advanced technology and information within these smart battery-powered suturing devices provides the ability to collect clinical data and drive design improvements to improve patient outcomes. However, there remains a need to better evaluate the conditions that affect staple formation and knife movement to create smarter suturing algorithms. Summary of the invention

[0006] In one aspect of the present disclosure, a computer-implemented method for controlling a surgical device is presented. The computer-implemented method includes accessing raw data captured by a sensor of the surgical device during a procedure and filtering the data with a filter. The filter includes a moving minimum filter. The method also includes: generating difference data based on the difference between the raw data and the filtered data; generating zero-crossing data based on determining the time point at which the amplitude of the difference data last passed from a non-zero amplitude value through a zero amplitude value to a non-zero amplitude value of the opposite sign; providing the zero-crossing data as input to a machine learning classifier; and predicting the probability of an end stop point based on the machine learning classifier. The end stop point may include a time point at which a knife of the surgical device pauses from cutting tissue.

[0007] In some aspects, the raw data may include one or more pulse width modulated (PWM) signals configured to control a motor of the surgical device.

[0008] In some aspects, the method may further include controlling the motor to prevent further movement of the knife.

[0009] In some aspects, the difference data may include time series data.

[0010] In some aspects, the method may also include determining at least one of safety or efficacy of the end effector of the surgical device based on the predicted probability.

[0011] In some aspects, the method can further include determining whether to form a staple based on the predicted end stop point probability.

[0012] In some aspects, the method may further include determining the presence of a slider of the surgical device based on the predicted end stop probability.

[0013] In some aspects, the method may also include processing the raw data to determine a root mean square value, a shape factor, and / or a crest factor of the raw data; and inputting the determined root mean square value, shape factor, and / or crest factor of the raw data into the machine learning classifier.

[0014] In some aspects, a machine learning classifier may comprise a decision tree.

[0015] In one aspect of the present disclosure, a system for controlling a surgical device is presented. The system includes a surgical stapling device, the surgical stapling device including a sensor configured to sense a signal, the signal being configured to control a motor of the surgical stapling device. The surgical stapling device includes at least one processor and at least one memory. The at least one memory includes instructions stored thereon, which, when executed by at least one processor, cause the system to access raw data captured by a sensor of the surgical device during a procedure; filter the raw data, the filter including a moving minimum filter; generate difference data based on the difference between the raw data and the filtered data; generate zero-crossing data based on determining the time point at which the amplitude of the difference data last passes from a non-zero amplitude value through a zero amplitude value to a non-zero amplitude value of the opposite sign; input the zero-crossing data into a machine learning classifier; predict the probability of an end stop point based on the machine learning classifier, wherein the end stop point includes the time point at which the knife of the surgical device pauses to cut tissue.

[0016] In some aspects, the surgical stapling device may further include a knife configured to cut tissue and a motor configured to advance the knife. The raw data may include a pulse width modulated signal configured to control the motor of the surgical stapling device.

[0017] In some aspects, the instructions, when executed by at least one processor, may further cause the system to determine the safety and / or efficacy of an end effector of a surgical stapling device based on the predicted probability.

[0018] In some aspects, the instructions, when executed by at least one processor, may further cause the system to determine whether a staple of the surgical stapling device is formed based on the predicted end stop point probability.

[0019] In some aspects, the instructions, when executed by at least one processor, may also cause the system to process the raw data to determine a root mean square value, a shape factor, and / or a crest factor of the signal; and input the determined root mean square value, shape factor, and / or crest factor of the signal into a machine learning classifier.

[0020] In one aspect of the present disclosure, a computer-implemented method for controlling a surgical device includes: accessing raw data captured by a sensor of the surgical device during a procedure; selecting a window of the raw data, wherein the window is configured to make the raw data non-periodic; extracting features from the windowed data; inputting the extracted features to a machine learning algorithm; and predicting a probability of presence of a sliding member of the surgical device based on the machine learning classifier. The sensor includes an ammeter, an accelerometer, an inertial measurement unit, and / or a strain gauge.

[0021] In some aspects, the extracted features may include an average current drawn by a motor of the surgical device and / or a force exerted on the motor.

[0022] In some aspects, the raw data may include time series data.

[0023] In some aspects, the method may also include determining at least one of safety or efficacy of the end effector of the surgical device based on the predicted probability.

[0024] In some aspects, the method may further include determining a probability of staple formation of the surgical device based on the predicted probability.

[0025] In some aspects, the method can also include determining the presence of a slider of the surgical device based on the predicted probability, and if the presence of the slider is not determined, refraining from firing the surgical device. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present disclosure relates to surgical devices. More particularly, the present disclosure relates to handheld electromechanical surgical systems for performing surgical procedures.

[0027] Figure 1 is a perspective view of a handheld surgical device and adapter assembly showing its connection to an end effector according to aspects of the present disclosure;

[0028] Figure 2A is a front perspective view of the power pack with the inner rear housing separated;

[0029] Figure 2B is a rear perspective view of the power pack with the inner rear housing removed;

[0030] Figure 3 yes Figure 1 A front perspective view of an end effector of a surgical device;

[0031] Figure 4 is configured according to aspects of the present disclosure for Figure 1 A functional block diagram of a controller of a handheld surgical device and an adapter assembly;

[0032] Figure 5 is a block diagram of a deep learning neural network and inputs and outputs of the deep learning neural network according to aspects of the present disclosure;

[0033] Figure 6 According to various aspects of the present disclosure Figure 5 Diagram of the layers of a deep learning neural network;

[0034] Figure 7 It is used to control Figure 1 A flowchart of a computer-implemented method of a surgical device;

[0035] Figure 8 is a graph of a difference signal generated by detecting the difference between the filtered PWM signal and the original PWM signal;

[0036] Fig. 9 is a graph showing a training data set of form factor speed with and without end points versus end stop time data;

[0037] Fig.10 is a graph of a training data set showing zero crossing versus end stop time data with and without endpoints;

[0038] Fig.11 is a graph showing a training data set with and without crest factor versus end stop time data;

[0039] Fig.12 It is used to control Figure 1 A flowchart of a computer-implemented method of a surgical device;

[0040] Figures 13 to 15 is a graph of the current of the generator when the surgical device is missing the slider;

[0041] Fig.16 is a graph of the current of the generator in the presence of a sliding member in the surgical device;

[0042] Fig.17 is Figure 1 a graph of windowed sensor data in the presence of a slider in a surgical device; and

[0043] Fig.18 is Figure 1 A graph of windowed sensor data without a slider in a surgical device.

[0044] The details of one or more aspects of the present disclosure are set forth in the following drawings and description. Other features, objectives, and advantages of the techniques described in the present disclosure will be apparent from the description and drawings, and from the claims that follow. DETAILED DESCRIPTION

[0045] Various aspects of the presently disclosed surgical devices and adapter assemblies and / or handle assemblies for surgical devices are described in detail with reference to the accompanying drawings, in which like reference numerals designate like or corresponding elements in each of the several figures. As used herein, the term "distal" refers to a portion of an adapter assembly or surgical device or a component thereof that is further away from a user, while the term "proximal" refers to a portion of an adapter assembly or surgical device or a component thereof that is closer to a user. The term "clinician" refers to a physician, nurse, or other care provider and may include support personnel.

[0046] A surgical device according to aspects of the present disclosure is designated 100 and is in the form of a powered, handheld electromechanical instrument configured for selective attachment to a plurality of different end effectors, each end effector configured for actuation and manipulation by the powered, handheld electromechanical surgical instrument. In addition to enabling powered actuation and manipulation, the surgical device 100 also incorporates various safety and control features that help ensure its proper, safe, and effective use.

[0047] The systems and methods described herein utilize various surgical device parameters, such as motor current, to train a machine learning classifier for use in controlling a surgical device.

[0048] like Figure 1 As shown, the surgical device is configured for selective connection with an adapter 410, and the adapter 410 is in turn configured for selective connection with an end effector 400 or a loading unit, which may be a single use loading unit ("SULU") or a multiple use loading unit ("MULU"). Although described with respect to the adapter 410 and the end effector 400, different adapters configured for different end effectors and / or different end effectors configured for the adapter 410 can also be used with the surgical device 100. Suitable end effectors configured for the adapter 410 and / or other adapters that may be used with the surgical device 100 include end effectors configured for performing, for example, an endoscopic gastrointestinal anastomosis (EGIA) procedure. The surgical device 100 includes a controller 200 configured to control actuation of the surgical device 100 and the firing and formation of staples in tissue. The controller 200 may be located on the main controller circuit board 142b ( Figure 2A ) and / or may be external to the surgical device 100.

[0049] refer to Figure 2A and Figure 2B , the surgical device 100 includes an inner handle housing 110. The inner handle housing 110 provides a housing in which the power pack core assembly 106 is located. The power pack core assembly 106 includes a rechargeable battery 144 configured to provide power to any of the electrical components of the surgical device 100, a battery circuit board 140, and a controller circuit board 142. The controller circuit board 142 includes a motor controller circuit board 142a, a main controller circuit board 142b, and a first ribbon cable 142c interconnecting the motor controller circuit board 142a with the main controller circuit board 142b. The motor controller circuit board 142a is communicatively coupled to the battery circuit board 140 to enable communication therebetween and between the battery circuit board 140 and the main controller circuit board 142b.

[0050] The power pack core assembly 106 also includes a display screen 146 supported on the main controller circuit board 142b. A clear or transparent window 110d (see FIG. 110 ) provided in the proximal half 110b of the inner handle housing 110 is provided. Figure 2A ) display screen 146 is visible. It is contemplated that at least a portion of the inner handle housing 110 may be made of a transparent rigid plastic or the like. It is also contemplated that the outer shell housing 10 may include a window formed therein (in visual registration with the display screen 146 and with the window 110d of the proximal half-section 110b of the inner handle housing 110), and / or the outer shell housing 10 may be made of a transparent rigid plastic or the like.

[0051] The power pack core assembly 106 also includes a first motor 152, a second motor 154, and a third motor 156, each electrically connected to the controller circuit board 142 and the battery 144. The motors 152, 154, 156 are arranged between the motor controller circuit board 142a and the main controller circuit board 142b. Each motor 152, 154, 156 includes a corresponding motor shaft 152a, 154a, 156a extending therefrom. Each motor shaft 152a, 154a, 156a has a three-petal transverse cross-sectional configuration for transmitting rotational force or torque. As an alternative to the motors 152, 154, 156, it is conceivable that more or fewer motors can be provided, or one or more other drive components (e.g., solenoids) can be utilized and controlled by an appropriate controller. Manual drive components can also be considered.

[0052] Each motor 152, 154, 156 is controlled by a respective motor controller "MC0", "MC1", "MC2". The motor controllers "MC0", "MC1", "MC2" are disposed on a motor controller circuit board 142a. The motor controllers are disposed on the motor controller circuit board 142a and are, for example, A3930 / 31K motor drivers from Allegro Microsystems, Inc. The A3930 / 31K motor drivers are designed to control 3-phase brushless DC (BLDC) motors such as motors 152, 154, 156 with N-channel external power MOSFETs. Each motor controller is coupled to a master controller or master chip 157 disposed on a master controller circuit board 142b via a first ribbon cable 142c, which connects the motor controller circuit board 142a with the master controller circuit board 142b. The master controller 157 communicates with the motor controllers "MC0", "MC1", "MC2" through a field programmable gate array (FPGA) 162 that provides control logic signals (e.g., coasting, braking, etc.). The control logic of the motor controllers "MC0", "MC1", "MC2" then outputs corresponding power supply signals to their respective motors 152, 154, 156 using fixed frequency pulse width modulation (PWM). The main controller 157 is also connected to a memory 165, which is also located on the main controller circuit board 142b. The main controller 157 is, for example, an ARM Cortex M4 processor from Freescale Semiconductor, Inc, which includes 1024 kilobytes of internal flash memory.

[0053] Each motor 152, 154, 156 is supported on the motor bracket 148 so that the motor shaft 152a, 154a, 156a can be rotatably disposed in a corresponding aperture of the motor bracket 148. Figure 2B As shown in FIG. 1 , the motor bracket 148 rotatably supports three rotatable drive connector sleeves 152b, 154b, 156b that are keyed to the respective motor shafts 152a, 154a, 156a of the motors 152, 154, 156. When the power pack 101 is placed in the housing shell 10, the drive connector sleeves 152b, 154b, 156b non-rotatably receive the proximal ends of the respective coupling shafts 64a, 64b, 64c of the plate assembly 60 of the housing shell 10. The drive connector sleeves 152b, 154b, 156b are each spring biased away from the respective motors 152, 154, 156.

[0054] Rotation of the motor shafts 152a, 154a, 156a by the respective motors 152, 154, 156 is used to drive the shaft and / or gear components of the adapter 410 to perform various operations of the surgical device 100. Specifically, the motors 152, 154, 156 of the power pack core assembly 106 are configured to drive the shaft and / or gear components of the adapter 410 to selectively move the tool assembly 404 of the end effector 400 relative to the proximal body portion 402 of the end effector 400, to rotate the end effector 400 about the longitudinal axis, to move the staple cartridge 314 relative to the anvil assembly 406 of the end effector 400, and / or to fire staples from the staple cartridge 314 of the end effector 400.

[0055] The motor bracket 148 also supports an electrical adapter interface socket 149. The electrical socket 149 is electrically connected to the main controller circuit board 142b via a second ribbon cable 142d. The electrical socket 149 defines a plurality of electrical slots for receiving corresponding electrical contacts or blades extending from the through connector 66 of the board assembly 60 of the housing shell 10.

[0056] In use, when the adapter 410 is mated to the surgical device 100, each coupling shaft 64a, 64b, 64c of the plate assembly 60 of the housing shell 10 of the surgical device 100 is coupled with a corresponding rotatable connector sleeve 218, 220, 222 of the adapter 410. In this regard, the interface between the corresponding first coupling shaft 64a and the first connector sleeve 218, the interface between the corresponding second coupling shaft 64b and the second connector sleeve 220, and the interface between the corresponding third coupling shaft 64c and the third connector sleeve 222 are keyed such that rotation of each of the coupling shafts 64a, 64b, 64c of the surgical device 100 results in corresponding rotation of the corresponding connector sleeve 218, 220, 222 of the adapter 410.

[0057] The mating of the coupling shafts 64a, 64b, 64c of the surgical device 100 with the connector sleeves 218, 220, 222 of the adapter 410 allows for independent transmission of rotational force via each of the three respective connector interfaces. The coupling shafts 64a, 64b, 64c of the surgical device 100 are configured to be independently rotated by the respective motors 152, 154, 156.

[0058] Because each coupling shaft 64a, 64b, 64c of the surgical device 100 has a keyed and / or substantially non-rotatable interface with the corresponding connector sleeve 218, 220, 222 of the adapter 410, when the adapter 410 is coupled to the surgical device 100, rotational force is selectively transmitted from the motors 152, 154, 156 of the surgical device 100 to the adapter 410.

[0059] Selective rotation of the coupling shafts 64a, 64b, 64c of the surgical device 100 allows the surgical device 100 to selectively actuate different functions of the end effector 400. The selective and independent rotation of the first coupling shaft 64a of the surgical device 100 corresponds to the selective and independent opening and closing of the tool assembly 404 of the end effector 400, and the driving of the stapling / cutting components of the tool assembly 404 of the end effector 400.

[0060] Actuation of the push button switch 172c corresponding to downward actuation of the toggle control button 30 causes the controller circuit board 142 to provide appropriate signals to the motor 152 to close the tool assembly 404 of the end actuator 400 and / or to fire staples from the staple cartridge 314 of the end actuator 400.

[0061] Actuation of the push button switch 172a corresponding to upward actuation of the toggle control button 30 causes the controller circuit board 142 to provide appropriate signals to the motor 152 to retract the staple sled and open the tool assembly 404 of the end effector 400.

[0062] like Figure 3 As shown, the end effector 400 of the surgical device 100 includes a staple cartridge 314 removably supported in a carrier. The staple cartridge 314 defines a central longitudinal slot 370 defined along the center thereof to receive the knife 312, and a plurality of linear rows of staple retention slots 316 located on each side of the longitudinal slot. Each staple retention slot 316 receives a single staple 315 and a portion of a staple pusher 311. During operation of the surgical device 100, a drive assembly (not shown) actuates the knife 312 adjacent to the actuation sled 313 and pushes the actuation sled through the cartridge 314. As the actuation sled 313 moves through the cartridge, the cam wedges of the actuation sled 313 sequentially engage the staple pushers 311 to vertically move the staple pushers 311 within the staple retention slots 316 and sequentially eject a single staple therefrom for formation against an anvil (not shown).

[0063] Reference now Figure 4 , the controller 200 includes a processor 220 connected to a computer-readable storage medium or memory 230. The computer-readable storage medium or memory 230 can be a volatile type of memory, such as RAM, or a non-volatile type of memory, such as flash media, disk media, etc. In various aspects of the present disclosure, the processor 220 can be another type of processor, such as a digital signal processor, a microprocessor, an ASIC, a graphics processing unit (GPU), a field programmable gate array (FPGA), or a central processing unit (CPU). In certain aspects of the present disclosure, as opposed to a processor, network inference can also be accomplished in a system that implements weights as memristors, chemically, or other inferred calculations.

[0064] In various aspects of the present disclosure, the memory 230 may be a random access memory, a read-only memory, a disk memory, a solid-state memory, an optical disk memory, and / or another type of memory. In some aspects of the present disclosure, the memory 230 may be separated from the controller 200 and may communicate with the processor 220 via a communication bus of a circuit board and / or via a communication cable (such as a serial ATA cable or other type of cable). The memory 230 includes computer-readable instructions that can be executed by the processor 220 to operate the controller 200. In other aspects of the present disclosure, the controller 200 may include a network interface 240 to communicate with other computers or servers. The storage device 210 may be used to store data. The disclosed method may be run on the controller 200 or on a user device, including, for example, running on a mobile device, an IoT device, or a server system.

[0065] refer to Figure 5 , a block diagram of a machine learning classifier 500 for classifying data according to some aspects of the present disclosure is shown. In some systems, the machine learning classifier 500 may include, for example, a convolutional neural network (CNN) and / or a recurrent neural network. A deep learning neural network includes multiple hidden layers. As explained in more detail below, the machine learning classifier 500 may utilize one or more classification models (e.g., CNN, decision tree, naive Bayes, k-nearest neighbor) to classify data sensed by the sensor 212 (see Figure 2A ) classification. Sensor 212 ( Figure 2A ) may include, for example, an ammeter configured to sense motor current, a strain gauge configured to sense force, an accelerometer, a battery controller, an inertial measurement unit configured to sense angular rate, force and / or magnetic field, an encoder configured to measure motor position and / or motor speed, a current sensing resistor, a Hall effect sensor configured to measure motor current, and / or a load element configured to sense load. The machine learning classifier 500 may be configured in the controller 200 ( Figure 4 Those skilled in the art will understand the machine learning classifier 500 and how to implement it.

[0066] In machine learning, CNN is a class of artificial neural networks (ANN) that are most commonly used to analyze visual images. The convolutional aspect of CNN involves applying matrix processing operations to local parts of an image, and the result of these operations (which may involve dozens of different parallel and serial computations) is a set of many features for delivery to the next layer. CNNs typically include convolutional layers, activation function layers, deconvolution layers (for example, in segmentation networks), and / or pooling (usually max pooling) layers to reduce dimensionality without losing too many features. Additional information can be included in the operations that generate these features. Providing unique information that produces features that give neural network information can be used to provide a way to distinguish between different aggregates of data input to the neural network.

[0067] refer to Figure 6 In general, the machine learning classifier 500 (e.g., a convolutional deep learning neural network) includes at least one input layer 610, multiple hidden layers 606, and at least one output layer 620. The input layer 610, multiple hidden layers 606, and the output layer 620 all include neurons 602 (e.g., nodes). The neurons 602 between the layers are interconnected via weights 604. Each neuron 602 in the deep learning neural network 500 calculates an output value by applying a specific function to the input value from the previous layer. The function applied to the input value is determined by a vector of weights 604 and a bias. In a deep learning neural network, learning is performed by iteratively adjusting these biases and weights. The vector of weights 604 and the bias are called filters (e.g., kernels) and represent specific features of the input (e.g., a specific shape). The machine learning classifier 500 can output logarithms.

[0068] The machine learning classifier 500 can be trained based on labeled training data to optimize weights. For example, the sensor can include motor data PWM signal data. In some methods according to the present disclosure, the training can include supervised learning. Those skilled in the art will understand training the machine learning classifier 500 and how to implement it.

[0069] In some methods according to the present disclosure, the deep learning neural network 500 may be used to classify sensor data captured by the sensor 212 (see Figure 2A The classification of the sensor signal 502 may be used to determine a classification score for each prediction 506 for determining the blade 312 of the surgical device 100 ( Figure 3 ) The point at which cutting of the tissue is paused (e.g., an end stop).

[0070] refer to Figure 7 , shows a computer-implemented method for controlling a surgical device. The method can be executed on the controller 200 or on a separate device such as a server or a mobile user device.

[0071] Figure 1 The surgical device 100 enables the user to perform single-handed articulation. Clamping, firing and retraction can also be completed with one hand, thereby achieving more stable tissue interaction during firing. The disclosed method has the benefit of reducing undesired motion after the end stop point.

[0072] Initially, at step 702, the controller 200 accesses the sensors 212 ( Figure 2A ) captured by the sensor. The raw data can represent a time series signal, such as the variation of motor current over time.

[0073] For example, motors 152, 154, 156 ( Figure 2B ) along the suture end effector 400 ( Figure 1 ) Propel knife 312 ( Figure 3 ). As the knife 312 advances, the knife 312 compresses the tissue, spreads the staples through the tissue, and cuts between the staples. At the end of the suturing end effector 400, the knife 312 reaches an end stop and cannot be advanced any further. The disclosed method algorithm uses Figure 1 The end stop point is determined by the sensed data of the suturing system. The knife 312 must reach the end stop point to fully deploy all staples. However, any additional rotation of the motor beyond the end stop point may cause undesired movement of the end actuator 400. It may be advantageous to stop the motors 152, 154, 156 as close to the end stop point as possible. As used herein, the end stop point includes the point where the knife 312 pauses in cutting tissue. The motor speed is controlled by a PWM (pulse width modulation) signal. Near the end of the firing, the PWM signal changes sharply ( Figure 8 ). PWM speed control works by driving the motor with a series of "ON-OFF" pulses and varying the duty cycle of the pulses (the fraction of time the output voltage is "ON" compared to when it is "OFF") while keeping the frequency constant. The power applied to the motor can be controlled by varying the width of these applied pulses and thereby varying the average DC voltage applied to the motor's terminals. By varying or modulating the timing of these pulses, the speed of the motor can be controlled, for example, the longer the pulse "ON", the faster the motor will spin, and likewise, the shorter the pulse "ON", the slower the motor will spin. The wider the pulse width, the greater the average voltage applied to the motor terminals, the stronger the magnetic flux inside the armature windings, and the faster the motor will spin.

[0074] Next, at step 704, the controller 200 filters the raw data. The filter includes a moving minimum filter. For example, the moving minimum filter can be applied to a PWM signal. The moving minimum filter returns an array of minimum values ​​centered at a local k point, where each minimum value is calculated over a sliding window of length k across neighboring elements.

[0075] Next, at step 706, the controller 200 generates difference data based on the difference between the original data and the filtered data. For example, the difference between the filtered PWM signal and the original unfiltered PWM signal is determined.

[0076] Next, at step 708, the controller 200 generates zero-crossing data based on determining the time point at which the difference data last crossed zero. A zero crossing is a point at which the sign of a mathematical function changes (e.g., from positive to negative), represented by an intercept (zero value) of the axis in a graph of the function. This time point may be associated with a physical location at which the knife 312 of the surgical device 100 paused in cutting tissue.

[0077] For example, a zero-crossing detector can be applied to the resulting difference signal. Observe that at the endpoints, the signal no longer crosses zero and the corresponding time information is also preserved ( Fig.10 In addition to the zero-crossing feature, several other signal features can be extracted, such as RMS value, shape factor ( Fig. 9 ) and / or crest factor ( Fig.11 ). Compared with the firing data without the endpoints, the firing data with the endpoints has different shapes for all the above features.

[0078] Next, at step 710, the controller 200 provides the zero-crossing data as input to the machine learning classifier 500 ( Figure 5 ). The zero-crossing data includes the time difference between consecutive zero-crossing points. For example, features such as generating zero-crossing data, RMS value, shape factor and / or crest factor can be used as input to the decision tree.

[0079] Next, at step 712, the machine learning classifier 500 predicts the probability of an end stop point based on the machine learning classifier 500. The time series signal data may be used to train the machine learning classifier 500. Firing data from a variety of configurations may be used.

[0080] In various aspects, the controller 200 may control the motors 152 , 154 , 156 to prevent further movement of the surgical device 100 .

[0081] The controller 200 may determine the safety and / or efficacy of the end effector 400 of the surgical device 100 based on the predicted probability. For example, if the predicted probability is above a threshold, it may be determined that the end effector 400 is safe.

[0082] In various aspects, the controller 200 can determine whether to form a staple based on the predicted end stop probability. For example, if the predicted end stop probability is less than a predetermined number (eg, less than about 40%), the controller can determine that a staple is not formed.

[0083] refer to Fig. 9, shows a graph of a training data set illustrating form factor velocity versus end stop time data. The plot includes data with and without end points. Fig.10 A graph of a training data set illustrating zero crossing versus end stop time data is shown. Fig.11 , shows a graph of a training data set illustrating crest factor versus end stop time data.

[0084] See also Fig.12 , shows the control Figure 1 A computer-implemented method for a stapler 100. In conventional stapler reloading, a possible failure is when the device cuts tissue without sealing it. The most common way this happens is if the stapler SULU is manufactured without the slider 313 ( Fig.17 ). The controller 200 collects data on more than about 250 device usage parameters. Device parameters may include, for example, motor speed, position, force, type of reload / adapter, age of the instrument, and / or battery readings. A visual difference in this data can be seen when firing a SULU that is forming staples relative to a SULU that is not forming staples. However, this data may vary depending on the use of the stapler, such as the angle of articulation, the tissue being fired, the motor current, and / or the firing force ( Figures 13 to 16 ). This variability in the data makes it difficult to accurately detect the difference between the two occurrences using conventional software algorithms. The disclosed method enables the use of machine learning to classify the signal characteristics recorded in the surgical device 100 by the controller 200 to detect the slider 313 ( Figure 3 )’s existence.

[0085] Initially at step 1202, the controller 200 accesses raw data (eg, a data set) captured by the sensors of the surgical device during a procedure. The sensor 212 ( Figure 2A ) may include an ammeter configured to sense motor current, a strain gauge configured to sense force, an accelerometer, a battery controller, and / or an inertial measurement unit configured to sense angular rate, force, and / or magnetic field.

[0086] Next, in step 1204, the controller 200 selects a window of the raw data. The window is configured to make the raw data non-periodic. For example, a small window of the data set can be taken so that the signal is limited in nature rather than periodic. This small window will also enable bit-by-bit signal processing with overlapping areas. In various aspects, the window size and / or overlap percentage can be changed and / or optimized. The window size can be optimized by balancing computational efficiency with accuracy.

[0087] The window size multiplied by the sampling rate gives the number of data samples used to make classification decisions. The window increment multiplied by the sampling rate gives the number of new samples for each successive analysis. The overlap percentage is optimized over classification error and inference latency.

[0088] Next, at step 1206, the controller 200 extracts features from the windowed data. Features represent identifiable measurements and distinguishing functional components obtained from segments of the pattern. The extracted features can achieve reduced loss of key information embedded in the signal.

[0089] The Recursive Feature Elimination (RFE) technique can be used to select informative features. RFE is a feature selection method that fits a model and removes the weakest feature (or features) until a specified number of features is reached. Features can be ranked by the model's feature importance property, and by recursively eliminating a small number of features per cycle, RFE attempts to eliminate dependencies and collinearity that may be present in the model.

[0090] Next, at step 1208, the controller 200 provides the extracted features as input to the machine learning classifier. For example, by using RFE, the average motor current ( Figures 13 to 16 ), motor beats (via encoders), revolutions per minute (RPM), and / or strain gauge readings for position to feed into the machine learning classifier 500.

[0091] Next, at step 1210, the controller 200 generates a classification algorithm based on the machine learning classifier 500 ( Figure 5 ) predicts the sliding member 313 of the surgical device 100 ( Figure 3 ) exists. For example, for classification, a gradient boosting machine (GBM) can be used. GBM is an ensemble of classification tree models. An ensemble of classification tree models is a forward learning ensemble method that uses gradually improved estimates to obtain prediction results. Boosting is a flexible nonlinear regression process that helps improve the accuracy of the tree. A weak classification algorithm is sequentially applied to incrementally changing data to create a series of decision trees, thereby generating an ensemble of weak prediction models. The output of this classification algorithm is a probability score, which is fed to the inference layer.

[0092] In various aspects, the controller 200 may make an inference based on the probability score, and this inference may be used to determine the correct surgical procedure in the surgical device 100 ( Figure 1 ) to make necessary decisions within a certain time frame. The threshold of the probability score can be set as the decision boundary in the inference layer.

[0093] The disclosed method has the benefit of enabling the ability to detect the presence of a slider in the end effector 400 under widely variable conditions. Another benefit of the disclosed method is the ability to learn and adapt to new circumstances with software updates utilizing offline learning at a research center.

[0094] In various aspects, the controller 200 may determine the surgical device 100 ( Figure 1 ) of the sliding member 313 ( Figure 3 ) and, if the slider is not determined to be present, the firing of the surgical device 100 is prohibited. The controller 200 may provide an indication that the slider 313 is not present based on the determination. For example, the indication may be an audio or visual alarm. In various aspects, the controller 200 may determine that the end effector 400 ( Figure 1 )’s safety and / or efficacy.

[0095] refer to Fig.17 , showing that in Figure 1 A graph of windowed sensor data in the presence of a slider in surgical device 100. Fig.18 Shown in Figure 1 A graph of windowed sensor data without a slider in a surgical device.

[0096] Although illustrated as being used in a handheld surgical device, it is contemplated and within the scope of the present disclosure that the disclosed systems and methods are configured for use with various electromechanical and / or electrosurgical instruments and systems. For example, the disclosed methods may be used in robotic surgical systems, such as those shown and described in U.S. Patent No. 8,828,023, the entire contents of which are incorporated herein by reference. Specifically, the various aspects disclosed herein may also be configured to work with a robotic surgical system, and it is generally referred to as "telesurgical procedures". Such systems employ various robotic elements to assist clinicians and allow for remote operation (or partial remote operation) of surgical instruments. Various robotic arms, gears, cams, pulleys, electric and mechanical motors, etc. may be used for this purpose, and a robotic surgical system may be designed to assist clinicians during the course of an operation or treatment. Such robotic systems may include remotely manipulable systems, automated flexible surgical systems, remote flexible surgical systems, remote articulated motion surgical systems, wireless surgical systems, modular or selectively configurable remotely operated surgical systems, and the like.

[0097] The robotic surgical system can be used with one or more consoles located near the operating room or at a remote location. In this case, a team of clinicians can prepare the patient for surgery and construct the robotic surgical system with one or more of the instruments disclosed herein, while another clinician (or another group of clinicians) remotely controls the instrument via the robotic surgical system. It will be appreciated that a highly skilled clinician can perform multiple operations in multiple locations without leaving his / her remote console, which is economically advantageous and beneficial to the patient or a series of patients. For a detailed description of an exemplary medical workstation and / or its components, reference may be made to U.S. Patent Application Publication No. 2012 / 0116416 and PCT Application Publication No. WO2016 / 025132, each of which is incorporated herein by reference in its entirety.

[0098] It should be understood that the various aspects disclosed herein may be combined in combinations different from those specifically presented in the specification and drawings. It should also be understood that, depending on the example, certain actions or events of any of the processes or methods described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events may be required to perform these techniques). In addition, although for clarity, certain aspects of the present disclosure are described as being performed by a single module or unit, it should be understood that the techniques of the present disclosure may be performed by a combination of units or modules associated with, for example, a surgical device and / or a robotic surgical system.

[0099] In one or more examples, the described techniques can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a non-transitory computer-readable medium that corresponds to a tangible medium, such as a data storage medium (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer).

[0100] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the aforementioned structures or any other physical structure suitable for implementing the described techniques. Additionally, these techniques may be fully implemented in one or more circuits or logic elements.

[0101] Those skilled in the art will appreciate that the structures and methods specifically described herein and shown in the accompanying drawings are non-limiting exemplary aspects, and descriptions, disclosures, and diagrams should be recognized as exemplary specific aspects only. Therefore, it should be understood that the present disclosure is not limited to the precise aspects described, and that those skilled in the art can implement various other changes and modifications without departing from the scope or spirit of the present disclosure. In addition, without departing from the scope of the present disclosure, the elements and features shown or described in conjunction with certain aspects can be combined with the elements and features of certain other aspects, and these modifications and variations are also included in the scope of the present disclosure. Therefore, the subject matter of the present disclosure is not limited by the content that has been specifically shown and described.

Claims

1. A computer-implemented method for controlling a surgical device, the computer-implemented method comprising: accessing raw data captured by sensors of surgical devices during a procedure; filtering the raw data using a filter, wherein the filter comprises a moving minimum filter; generating difference data based on a difference between the original data and the filtered data; generating zero-crossing data based on determining a point in time at which the amplitude of the difference data last crossed from a non-zero amplitude value through a zero amplitude value to a non-zero amplitude value of opposite sign; providing the zero-crossing data as input to a machine learning classifier; as well as Predicting a probability of an end stop point based on the machine learning classifier, wherein the end stop point comprises a point in time at which a knife of the surgical device pauses from cutting tissue.

2. The computer-implemented method of claim 1 , wherein the raw data comprises a PWM signal configured to control a motor of the surgical device.

3. The computer-implemented method of claim 2, further comprising controlling the motor to prevent further movement of the surgical device. The computer-implemented method of claim 1 , wherein the difference data comprises time series data.

5. The computer-implemented method of claim 1, further comprising determining at least one of safety or efficacy of an end effector of the surgical device based on the predicted probability.

6. The computer-implemented method of claim 1, further comprising determining whether a staple of the surgical device is formed based on the predicted end stop point probability.

7. The computer-implemented method of claim 1, wherein the presence of a slider of the surgical device is determined based on the predicted end stop probability.

8. The computer-implemented method of claim 1 , further comprising: processing the raw data to determine at least one of a root mean square value, a shape factor, or a crest factor of the plurality of data; as well as At least one of the determined root mean square value, shape factor, or crest factor of the plurality of data is input to the machine learning classifier.

9. The computer-implemented method of claim 1, wherein the machine learning classifier comprises a decision tree.

10. A system for controlling a surgical device, the system comprising: A surgical stapling device, comprising: a sensor configured to sense a signal configured to control a motor of the surgical stapling device; at least one processor; and at least one memory including instructions stored thereon that, when executed by the at least one processor, cause the system to: accessing raw data captured by sensors of surgical devices during a procedure; filtering the raw data, the filter comprising a moving minimum filter; generating difference data based on a difference between the original data and the filtered data; generating zero-crossing data based on determining a point in time at which the amplitude of the difference data last crossed from a non-zero amplitude value through a zero amplitude value to a non-zero amplitude value of opposite sign; inputting the zero-crossing data into a machine learning classifier; and A probability of an end stop point is predicted based on the machine learning classifier, wherein the end stop point comprises a point in time when the blade of the surgical device cuts tissue.

11. The system of claim 10, wherein the surgical stapling device further comprises: a knife configured to cut tissue; and a motor configured to advance the knife; The raw data includes a pulse width modulated signal configured to control the motor of the surgical stapling device.

12. The system of claim 10, wherein the instructions, when executed by the at least one processor, further cause the system to determine at least one of safety or efficacy of an end effector of the surgical stapling device based on the predicted probability.

13. The system of claim 10, wherein the instructions, when executed by the at least one processor, further cause the system to determine whether a staple of the surgical stapling device is formed based on the predicted end stop point probability.

14. The system of claim 10, wherein the instructions, when executed by the at least one processor, further cause the system to: processing the raw data to determine at least one of an RMS value, a shape factor, or a crest factor of the signal; and At least one of the determined root mean square value, shape factor, or crest factor of the signal is input to the machine learning classifier.

15. A computer-implemented method for controlling a surgical device, the computer-implemented method comprising: accessing raw data captured by a sensor of the surgical device during a procedure, wherein the sensor comprises at least one of an ammeter, an accelerometer, an inertial measurement unit, or a strain gauge; selecting a window of the raw data, wherein the window is configured to make the raw data non-periodic; Extract features from windowed data; Input the extracted features into a machine learning classifier; as well as A probability of presence of a slider of the surgical device is predicted based on the machine learning classifier.

16. The computer-implemented method of claim 15, wherein the extracted features include at least one of an average current of a motor of the surgical device or a force exerted on the motor.

17. The computer-implemented method of claim 15, wherein the raw data comprises time series data.

18. The computer-implemented method of claim 15, further comprising determining at least one of safety or efficacy of an end effector of the surgical device based on the predicted probability.

19. The computer-implemented method of claim 15, further comprising determining a probability of staple formation of the surgical device based on the predicted probability.

20. The computer-implemented method of claim 15, further comprising: determining the presence of a slider of the surgical device based on the predicted probability; as well as In the event that the presence of the slider is not determined, firing of the surgical device is prohibited.

Images