Detecting knock-off or counterfeit surgical devices

Abstract

A computing device, such as a surgical hub, can obtain performance data associated with a device, such as a surgical device, in an operating room. Based on the performance data, the computing device can identify a performance signature associated with the surgical device. The computing device can determine whether the surgical device is an original equipment manufacturer (OEM) device or a counterfeit device. The computing device can compare the operational data and / or the performance signature with data associated with the OEM device (e.g., data from an ML-trained model). Based on the comparison, the computing device can determine whether the operational data associated with the surgical device is within normal operating parameters. If the computing device determines that the operational data is outside of the normal operating parameters, the computing device can send a message to a medical professional.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is related to the following concurrently filed applications, the contents of each of which are incorporated herein by reference: U.S. patent application under attorney docket number END9438USNP1 entitled "A Method for Advanced Algorithm Support" [Background technology]

[0002] Patient care, in general, improves when it is tailored to the individual. Because every person has different needs, surgical and interventional solutions that center every patient's unique journey can represent an efficient and innovative path to healing. At the same time, the high stakes of patient care, particularly the surgical process, often focus on conservative, repeatable activities.

[0003] For example, innovative medical technologies such as advanced surgical support computing systems and intelligent surgical instruments can improve approaches to patient care and address the specific needs of healthcare providers.

[0004] The ever-increasing availability of data and computing resources makes non-traditional algorithms, such as machine learning algorithms, a particular technological opportunity in the healthcare system. However, incorporating such non-traditional algorithms into any medical technology presents many challenges.

[0005] A computing system, which may include a surgical hub, may configure data to train a machine learning model and use the machine learning model to detect whether one or more devices are knock-off or counterfeit devices. The machine learning / machine learning model may be used to improve the data, such as to determine whether a device is an original equipment manufacturer device or a counterfeit device. However, using the data to train a machine learning model may be time consuming and inconvenient. Summary of the Invention [Means for solving the problem]

[0006] According to an embodiment of the present invention, there is provided a method executed by a processor that includes acquiring operational data associated with a surgical device for use in a surgical procedure, identifying an operational signature associated with the surgical device based on the acquired operational data, determining whether the surgical device is an authentic device based on the identified operational signature associated with the surgical device, determining whether the acquired operational data is within an operational range associated with the authentic device based on a determination that the surgical device is an authentic device, and sending a warning message to an operator based on a determination that the acquired operational data is not within an operational range associated with the authentic device.

[0007] In this way, the operator is alerted if the surgical device is not operating within the operating range for which the authentic surgical device is designed. In particular, by using the device's operating signature, it is possible to verify whether the surgical device (e.g., including any consumable parts) is authentic. Then, in the case of an authentic surgical device, since the operating range of that particular type of authentic surgical device is known, it can be determined whether the surgical device is operating within that operating range. Alerting the operator means that the operator can take corrective action, such as replacing the device or adjusting the surgical technique, to minimize the risk of surgical complications. This can help improve the safety and effectiveness of surgery and ultimately lead to better outcomes for patients.

[0008] In embodiments, the operational data is retrieved from memory (e.g., from a previous surgical procedure). In embodiments, the operational data associated with the surgical device includes operational data originating from the surgical device and / or from other devices used during the surgical procedure (e.g., an imaging device imaging the surgical site). In embodiments, the operational data includes multiple data types (e.g., two or more of firing load (e.g., motor current), circuit impedance (e.g., capacitance), frequency, feedback time, amount of force to fire curve, force peak, frequency of force peak, magnitude of buckling load, peak between single driver and / or dual driver).

[0009] In embodiments, determining whether the surgical device is an authentic device includes determining whether all components of the surgical device are authentic. In some embodiments, a non-authentic staple may be used in combination with an authentic surgical stapler. For example, a non-authentic staple may be used in combination with an authentic surgical stapler.

[0010] In embodiments, the warning message may be transmitted to the operator visually. For example, the surgical device or another device, such as a monitor, may have a display configured to display the warning message to the operator. In other embodiments, the warning message is audible.

[0011] The surgical procedure may be performed by an operator. Identifying a motion signature associated with the surgical data may include inputting the acquired motion data into a first machine learning model, where the first machine learning model is trained using motion data associated with a plurality of authentic devices and / or motion data associated with a plurality of non-authentic devices.

[0012] In this way, the motion signature identified by the first machine learning model is an embedding that maps the motion data into an embedding space. The embedding space has fewer dimensions than the motion data so as to capture the latent structure of the motion dataset (e.g., derived from multiple authentic and non-authentic surgical devices). Similar examples, e.g., embedding vectors for two authentic surgical devices, are close to each other in the embedding space.

[0013] In embodiments, the first machine learning model may be trained using a supervised machine learning algorithm. In embodiments, the first machine learning model may be a deep neural network (i.e., may have more than three layers). For example, the first machine learning model may be an autoencoder or a predictor. The input to the deep neural network is operational data, and the output is an embedding, referred to herein as an operational signature.

[0014] Operational data associated with multiple genuine devices may be labeled as originating from the genuine devices, and / or operational data associated with multiple non-genuine devices may be labeled as originating from the non-genuine devices.

[0015] Alternatively, identifying a motion signature associated with the surgical data may be performed without using machine learning. In these embodiments, the motion signature may represent a transformation of the motion data to simplify (i.e., reduce data size) the motion data. This may allow a comparison of the similarity of the acquired motion data to the motion data of each of multiple authentic devices and / or multiple non-authentic devices to require less computational power. Furthermore, data reduction may allow portions of the motion data to be generalized, allowing for slight differences between different authentic devices to be taken into account when comparing their similarity. Suitable data transformation techniques for simplifying data, such as data compression techniques, are known in the art.

[0016] Provided the operational signature is similar to operational signatures associated with a plurality of authentic devices, the method may further include identifying the surgical device as an authentic device.

[0017] Each surgical device has a unique operational signature because each device is designed, constructed, and used differently, and therefore has its own unique set of characteristics and operational patterns. For example, a surgical drill may have a different operational signature from a surgical stapler because it is designed to cut tissue using a rotary motion, while a stapler is designed to close wounds using mechanical force. Even different models of the same type of surgical device (i.e., two different surgical staplers) may have different operational signatures because the devices have different features and may be made of different materials. Nevertheless, by presenting the operational data as operational signatures, the first machine learning algorithm can consider common trends and patterns between the operational signatures of genuine devices compared to non-genuine surgical devices. This means that operational signatures from genuine devices (assuming the devices are operating and operating correctly) will be similar to each other.

[0018] In an embodiment, similarity is determined using a similarity metric, which is a mathematical function that measures the difference between two pieces of data, in this case, between the behavioral signature of the captured behavioral data and each of the behavioral signatures of a plurality of authentic devices. Example similarity metrics include Euclidean distance (the distance between the ends of vectors), cosine (the cosine of the angle between the vectors), and dot product (the product of the cosine and the length of both vectors).

[0019] In an embodiment, determining whether the operational signature is similar to operational signatures associated with a plurality of authentic devices includes determining a similarity metric between the operational signature and each of the operational signatures associated with the plurality of authentic devices. If any of the similarity metrics is above (or below, depending on the similarity metric used) a predetermined similarity threshold, the operational data is considered to be from an authentic surgical device and the surgical device is identified as authentic.

[0020] In embodiments, the predetermined similarity threshold depends on the type of surgical device: for some surgical devices, the non-authentic device is very similar in function to the authentic device, and therefore the threshold must be lower to account for subtle differences.

[0021] Provided the operational signature is similar to operational signatures associated with a plurality of non-authentic devices, the method may further include identifying the surgical device as a non-authentic device.

[0022] Again, similarity can be determined using a similarity metric that measures the difference between two pieces of data, in this case, the difference between the behavioral signature of the acquired behavioral data and each of the behavioral signatures of multiple non-authentic devices.

[0023] In an embodiment, determining whether the operational signature is similar to operational signatures associated with a plurality of non-authentic devices includes determining a similarity metric between the operational signature and each of the operational signatures associated with the plurality of non-authentic devices. If any of the similarity metrics is above (or below, depending on the similarity metric used) a predetermined similarity threshold, the operational data is considered to be from a non-authentic surgical device and the surgical device is identified as non-authentic.

[0024] In embodiments, the predetermined similarity threshold depends on the type of surgical device: for some surgical devices, the non-authentic device is very similar in function to the authentic device, and therefore the threshold must be lower to account for subtle differences.

[0025] The method may further include, based on determining that the surgical device is a genuine device or a non-genuine device, identifying a non-genuine device from the plurality of genuine devices having an operational signature most similar to an operational signature of the surgical device, and retrieving information associated with the non-genuine device from the plurality of genuine devices having an operational signature most similar to the operational signature of the surgical device, wherein the information includes at least one of a manufacturing facility of the non-genuine device, an operator using the non-genuine device, or a surgical procedure associated with the non-genuine device.

[0026] In an embodiment, using a similarity metric determined between the operational signature of the surgical device and each of the operational signatures of multiple non-genuine surgical devices, the most similar one may be identified as the non-genuine surgical device having the highest similarity metric.

[0027] In an embodiment, the most similar ones are identified using a nearest neighbor search based on a similarity metric.

[0028] In embodiments, once a non-genuine device is identified, information about the non-genuine device may be retrieved, for example, from memory. Such information may assist in identifying the source of the non-genuine device.

[0029] Provided that the operational signature is not similar to operational signatures associated with a plurality of genuine devices or a plurality of non-genuine devices, the method may further include continuing to acquire operational data associated with the surgical device and further training the first machine learning model using the acquired operational data.

[0030] In this way, the first machine learning model can be updated to identify behavioral signatures derived from any new surgical device, including both genuine and non-genuine surgical devices.

[0031] The method may further include labeling the operational data as originating from an authentic device or a non-authentic device.

[0032] In this way, the first machine learning model can be updated using a supervised learning algorithm to identify the motion signatures resulting from any new surgical device.

[0033] In embodiments, operational data may be labeled by a human in the loop, typically after operational data that cannot be attributed to a known authentic or non-authentic device has been examined to determine whether the surgical device with which it is associated is authentic or inauthentic.

[0034] Determining whether the surgical device is an authentic device based on the identified operational signature associated with the surgical device may include providing the operational signature to a second machine learning model trained to classify whether the surgical device is authentic.

[0035] In embodiments, the second machine learning model may be trained to classify whether a surgical device is genuine using behavioral signatures associated with multiple genuine devices and, optionally, behavioral signatures associated with multiple non-genuine devices. Such behavioral signatures are likely to be highly unbalanced, i.e., there are significantly more behavioral signatures available for training associated with genuine devices than non-genuine devices. As a result, determining whether a surgical device is genuine is performed by determining whether the acquired behavioral signature is an outlier. Therefore, known machine learning algorithms for detecting outliers may be used for the second machine learning model.

[0036] In some embodiments, the second machine learning model can be trained using an unsupervised learning algorithm. For example, in one embodiment, the second machine learning model may be based on a one-class support vector machine (SVM). This is an unsupervised learning algorithm that is trained to detect outliers, and therefore non-genuine devices, by learning the boundaries of behavioral signatures associated with (only) a number of genuine devices. From this, the one-class SVM classifies any acquired behavioral signatures that fall outside the boundaries as outliers and classifies the associated surgical device as non-genuine.

[0037] In other embodiments, the second machine learning model may be based on various machine learning algorithms, such as Local Outlier Factor (LOF), Isolation Forest, Neural Network with Long Short-Term Memory (LSTM) Autoencoder, Density-Based Anomaly Detection with K-Nearest Neighbor (KNN), Clustering-Based Anomaly Detection with K-Means, or Density-Based Spatial Clustering of Applications with Noise (DBSCAN).

[0038] The captured operational data may not include an identifier associated with the surgical device.

[0039] In embodiments, the acquired operational data does not include identifiers associated with parts of the surgical device, such as consumable parts. For example, a surgical stapler may provide an identifier for the surgical stapler but not for the staples being used, for example, because the staples are not electronically coupled to the surgical device. Nevertheless, the authenticity of the entire surgical device, including the consumable parts, can still be determined, as non-authentic consumable parts used with an otherwise authentic surgical device will affect the operational signature, causing the surgical device to be identified as non-authentic.

[0040] The method may further include, if the acquired operational data includes an identifier associated with the surgical device, determining whether the identifier associated with the surgical device matches stored identifiers of a plurality of authentic devices, and determining whether the surgical device is an authentic device by setting a flag indicating that the surgical device is an authentic device based on a determination that the identifier matches the stored identifiers of the plurality of authentic devices, or setting a flag indicating that the surgical device is a non-authentic device based on a determination that the identifier does not match the stored identifiers of the plurality of authentic devices.

[0041] In this way, a surgical device that provides an identifier does not need to determine an operational signature from operational data (thus, for example, using machine learning), and therefore requires less computational power to determine whether the device is authentic.

[0042] In an embodiment, the identifier of the captured operational data may be encrypted, and determining whether the identifier matches stored identifiers of a plurality of authentic devices may include decrypting the identifier using a decryption key available to the processor making the determination.

[0043] In embodiments, this method may be used only if all parts of the surgical device, including consumable parts, provide identifiers. For example, if the surgical stapler provides an identifier but the staples being used do not, this method may not be used because it cannot be guaranteed that all parts of the surgical device are authentic.

[0044] The method may further include identifying an authentic device from the plurality of authentic devices and retrieving a third machine learning model associated with the authentic device, where the third machine learning model is trained to determine whether the acquired operational data is within an operational range associated with the authentic device.

[0045] In an embodiment, identifying the authentic device from the plurality of authentic devices is performed using a first machine learning model and a similarity metric, as described above.

[0046] In an embodiment, the third machine learning model may be trained to classify whether the acquired operational data is within an operational range associated with an authentic device. As with the second machine learning model, determining whether the acquired operational data is within an operational range associated with an authentic device is performed by determining whether the acquired operational data is an outlier. Thus, known machine learning algorithms for detecting outliers may be used for the third machine learning model.

[0047] In some embodiments, the third machine learning model can be trained using an unsupervised learning algorithm. For example, in one embodiment, the third machine learning model may be based on a one-class support vector machine (SVM). This is an unsupervised learning algorithm trained to learn the operational range boundaries of the operational data associated with the authentic device. From this, the one-class SVM classifies any acquired operational data that falls outside the boundaries as an outlier and therefore outside the operational range.

[0048] In other embodiments, the third machine learning model may be based on various machine learning algorithms, such as local outlier factor (LOF), separation forest, neural network with long short-term memory (Density-Based Spatial Clustering of Applications with Noise, LSTM) autoencoder, density-based anomaly detection with K-nearest neighbors (KNN), clustering-based anomaly detection with K-means, or density-based spatial clustering of applications with noise (DBSCAN).

[0049] In alternative embodiments, statistical techniques may be used to classify whether the acquired operational data is within an operational range associated with a genuine device. For example, statistical techniques such as Kriging modeling techniques, x̂k factorial designs, and / or the like may be used.

[0050] The third machine learning model may be trained using (non-fault) operational data, catastrophic failure operational data, and / or device failure operational data of a genuine device, and the method may further include inputting the acquired operational data into the third machine learning model, determining the catastrophic failure operational data or device failure operational data that is most similar to the acquired operational data, provided that the acquired operational data is not within the operational range, and retrieving information associated with the most similar catastrophic failure operational data or device failure operational data.

[0051] In embodiments, similarity is determined using a similarity metric, in this case between the operational data associated with the surgical device and the catastrophic failure operational data or the device failure operational data. Example similarity metrics include Euclidean distance (the distance between the ends of the vectors), cosine (the cosine of the angle between the vectors), and dot product (the product of the cosine and the length of both vectors).

[0052] In an embodiment, the most similar is determined using a nearest neighbor search based on a similarity metric.

[0053] In embodiments, once the most similar catastrophic or device failure operational data is identified, information regarding the most similar catastrophic or device failure operational data may be retrieved, for example, from memory. Such information may assist in identifying why the surgical device is operating outside of its operational range.

[0054] The retrieved information associated with the most similar catastrophic failure operation data or device failure operation data may include one or more potential errors and one or more potential solutions, and the method may further include including the identified potential errors and potential solutions in a warning message to an operator.

[0055] In this manner, the operator is provided with potential errors in the surgical device and potential solutions to address the potential errors. Upon receiving the alert, the operator can use their expertise to investigate the potential error and determine why the surgical device is not functioning within its normal operating range. The potential solutions help the operator find a solution to why the surgical device is not functioning within its normal operating range.

[0056] In accordance with a further embodiment of the present invention, there is provided a method executed by a processor that includes acquiring operational data associated with a surgical device used in a surgical procedure, the method including: acquiring operational data associated with the surgical device based on the acquired operational data; identifying an operational signature associated with the surgical device based on the identified operational signature associated with the surgical device; determining whether the surgical device is operating abnormally based on the identified operational signature associated with the surgical device; determining whether the acquired operational data is within an operational range associated with correct operation based on a determination that the surgical device is operating abnormally; and sending a warning message to an operator based on a determination that the acquired operational data is not within an operational range associated with correct operation.

[0057] In this way, the operator is alerted if the surgical device is operating outside of its designed operating range. In particular, by using the device's operating signature, it is possible to ascertain whether the surgical device is operating abnormally. Then, because the operating range of the surgical device is known, it can be determined whether the surgical device is nonetheless operating within that operating range. Alerting the operator means that the operator can take corrective action, such as replacing the device or adjusting the surgical technique, to minimize the risk of surgical complications. This can help improve the safety and effectiveness of surgery, ultimately resulting in better outcomes for patients.

[0058] This embodiment may be implemented in a similar manner to the first embodiment described above, but by training a second machine learning model using (non-fault) operational data, catastrophic failure operational data, and / or device failure operational data instead of genuine and / or non-genuine device operational data.

[0059] In accordance with a further embodiment of the present invention, there is provided a method executed by a processor, the method including acquiring operational data associated with a surgical device of a surgical device used in a surgical procedure performed by an operator, identifying an operational signature associated with the surgical device based on the acquired operational data, determining whether the surgical device is being operated abnormally by the operator based on the identified operational signature associated with the surgical device, determining whether the surgical device is being operated abnormally by the operator based on a determination that the surgical device is not being operated, determining whether the acquired operational data is within an operational range associated with correct operation, and sending a warning message to the operator based on a determination that the acquired operational data is not within an operational range associated with correct operation.

[0060] In this way, the operator is alerted if the surgical device is operating outside of its designed operating range. In particular, by using the device's operating signature, it is possible to ascertain whether the surgical device is being operated abnormally by the operator. Then, because the operating range of the surgical device is known, it can be determined whether the surgical device is nevertheless operating within that operating range. Alerting the operator means that the operator can take corrective action, such as replacing the device or adjusting the surgical technique, to minimize the risk of surgical complications. This can help improve the safety and effectiveness of surgery, ultimately resulting in better outcomes for patients.

[0061] This embodiment may be implemented in a manner similar to the first embodiment described above, but by training a first machine learning model using operational data from a different operator instead of operational data from genuine and / or non-genuine devices.

[0062] According to a further embodiment of the present invention, there is provided a computing system comprising a processor configured to perform any of the above methods.

[0063] According to a further embodiment of the present invention there is provided a computing program which, when executed by a processor, causes the processor to perform any of the methods described above.

[0064] For example, disclosed herein are methods, systems, and apparatus for computing systems and / or computing devices that use machine learning (ML) to determine whether a device is an authentic original equipment manufacturer (OEM) device or a counterfeit device. The computing device may utilize ML and / or ML algorithms to improve artificial intelligence algorithms, reduce the number of iterations used to train the artificial intelligence algorithms, and / or shorten the time it takes to train the machine learning. The adaptive learning algorithm may be used to aggregate one or more data streams. The adaptive learning algorithm may be used to generate and / or determine metadata from the data aggregation. The adaptive learning may be used to determine one or more improvements from a previous machine learning analysis. Improvements in the collection and / or processing of data feeds may be used to determine whether a device is an OEM device or a counterfeit device.

[0065] A computing system, computing device, and / or method may be used to apply ML to a data collection to determine whether a device associated with the data collection is a genuine OEM device or a counterfeit device. The computing system may be or include a processor capable of implementing the method.

[0066] A computing system and / or computing device, such as a surgical hub, may obtain performance data associated with a device. The device may be a surgical device used in a surgery in an operating room. For example, the computing device may perform data collection for the surgical device. The data from the surgical device may be or include data related to the surgical device being used during the surgery. The computing systems and computer devices described herein are used as exemplary methods, and one skilled in the art will understand that other devices may be used interchangeably.

[0067] The computing device may identify a performance signature associated with the surgical device. For example, the computing device may identify a performance signature of the surgical device based on acquired performance data. The performance data may be or may include configurations and / or parameters associated with the device while in use. For example, the performance data may be or may include at least one of a firing load (e.g., motor current), circuit impedance (e.g., capacitance), frequency, feedback time, amount of force to fire curve, force peak, frequency of force peak, magnitude of buckling load, peak between single driver and / or dual driver, etc.

[0068] Based on the identified performance signature associated with the device, the computing device may determine whether the device is an authentic OEM device or a counterfeit device. For example, the computing device may use and / or configure ML to compare the identified performance signature with information associated with an ML model. The ML model may be or include normal performance data associated with a group of devices (e.g., a group of OEM devices and / or a group of counterfeit devices). Based on the comparison, the computing device can determine whether the identified performance signature associated with the device is similar to (e.g., matches) or within preconfigured threshold boundaries of the ML model information associated with the authentic OEM devices. If the performance signature and information from the ML model are similar to (e.g., matches) or within preconfigured threshold boundaries, the computing device can determine that the device is one of the authentic OEM devices.

[0069] The computing device can determine that if the performance signature is not similar to (e.g., does not match) the ML model information and / or is outside preconfigured threshold boundaries, the computing device can send an alert to a health care professional (HCP). For example, the computing device can send an alert to the HCP indicating that the device is not an authentic OEM device and / or a counterfeit device.

[0070] The computing device can determine that the device is operating normally based on the performance data and / or the identified performance signature. For example, the computing device can compare the performance data and / or the identified performance signature with ML model information associated with authentic OEM devices operating under normal conditions.

[0071] This Summary is provided to introduce a selection of concepts in a concise form that are further described herein in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Other features are described herein. [Brief explanation of the drawings]

[0072] [Figure 1] FIG. 1 is a block diagram of a computer-implemented surgical system. [Figure 2] 1 illustrates an exemplary surgical system in an operating room. [Figure 3] 1 illustrates exemplary surgical hubs paired with various systems. [Figure 4] Illustrates a surgical data network having a set of communicating surgical hubs configured to connect with a set of sensing systems, an environmental sensing system, a set of devices, and the like. [Figure 5] 1 illustrates a logic diagram of a control system for a surgical tool. [Figure 6] 1 illustrates an exemplary surgical system including a handle having a controller and a motor, an adapter releasably coupled to the handle, and a loading unit releasably coupled to the adapter. [Figure 7A] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7B] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7C]1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7D] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 8A] 1A and 1B show an exemplary supervised learning framework and an exemplary unsupervised learning framework, respectively. [Figure 8B] 1A and 1B show an exemplary supervised learning framework and an exemplary unsupervised learning framework, respectively. [Figure 9] 1 illustrates a flow diagram of a computing device that determines whether a surgical device is an original equipment manufacturer (OEM) device. [Figure 10] 1 illustrates a genuine OEM device transmitting performance data to a computing device and a counterfeit device transmitting performance data. DETAILED DESCRIPTION OF THE INVENTION

[0073] 1 is a block diagram of a computer-implemented surgical system 100. An exemplary surgical system, such as surgical system 100, may include one or more surgical systems (e.g., surgical subsystems) 102, 103, 104. For example, surgical system 102 may include a computer-implemented bidirectional surgical system. For example, surgical systems 102, 103, 104 may include a surgical computing system, such as a surgical hub 106 and / or a computing device 116, that communicates with a cloud computing system 108. Cloud computing system 108 may include a cloud server 109 and a cloud storage unit 110.

[0074] The surgical systems 102, 103, 104 may each be computer-enabled surgical instruments and devices. For example, the surgical systems 102, 103, 104 may include a wearable sensing system 111, a human interface system 112, a robotic system 113, one or more intelligent instruments 114, an environmental sensing system 115, and / or others. The wearable sensing system 111 may include one or more devices used to sense aspects of an individual's condition and activity within the surgical environment. For example, the wearable sensing system 111 may include a healthcare provider sensing system and / or a patient sensing system.

[0075] The human interface system 112 may include devices that allow individuals to interact with the surgical systems 102, 103, 104 and / or the cloud computing system 108. The human interface system 112 may include human interface devices.

[0076] The robotic system 113 may include a surgical robotic device, such as a surgical robot. The robotic system 113 may enable robotic surgical procedures. The robotic system 113 may receive information, settings, programming, control, etc. from the surgical hub 106; for example, the robotic system 113 may transmit data, such as sensor data, feedback information, video information, and operation logs, to the surgical hub 106.

[0077] Environmental sensing system 115 may include, for example, one or more devices used to measure one or more environmental attributes, for example, as further described in Figure 2. Robotic system 113 may include multiple devices used to perform a surgical procedure, for example, as further described in Figure 2.

[0078] The surgical system 102 may communicate with a remote server 109, which may be part of a cloud computing system 108. In one example, the surgical system 102 may communicate with the remote server 109 via a networked connection, such as an Internet connection (e.g., business internet service, T3, cable / FIOS networking node, etc.). The surgical system 102 and / or its components may communicate with the remote server 109 via a cellular transmission / reception point (TRP) or base station using one or more of the following cellular protocols: GSM / GPRS / EDGE (2G), UMTS / HSPA (3G), long term evolution (LTE) or 4G, LTE-Advanced (LTE-A), new radio (NR), or 5G.

[0079] In one example, the surgical hub 106 may facilitate displaying images from a surgical imaging device, such as a laparoscope. The surgical hub 106 may have collaborative interaction with other local systems to facilitate displaying information related to those local systems. The surgical hub 106 may interact with one or more sensing systems 111, 115, one or more intelligent instruments 114, and / or multiple displays. For example, the surgical hub 106 may be configured to collect measurement data from one or more sensing systems 111, 115 and send notification or control messages to one or more sensing systems 111, 115. The surgical hub 106 may send and / or receive information, including notification information, to and / or from a human interface system 112. The human interface system 112 may include one or more human interface devices (HIDs). The surgical hub 106 may send and / or receive notification or control information for audio, display, and / or control information to various devices in communication with the surgical hub.

[0080] For example, the sensing systems 111, 115 may include a wearable sensing system 111 (which may include one or more HCP sensing systems and one or more patient sensing systems) and an environmental sensing system 115. The one or more sensing systems 111, 115 may measure data related to various biomarkers. The one or more sensing systems 111, 115 may measure the biomarkers using one or more sensors, such as optical sensors (e.g., photodiodes, photoresistors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, thermoelectric sensors, infrared sensors, etc. The one or more sensors may measure the biomarkers as described herein using one of many of the following sensing technologies: photoplethysmography, electrocardiography, electroencephalography, colorimetric, impedimentary, potentiometric, amperometric, etc.

[0081] Biomarkers measured by one or more sensing systems 111, 115 may include, but are not limited to, sleep, core body temperature, maximal oxygen consumption, physical activity, alcohol intake, respiratory rate, oxygen saturation, blood pressure, blood glucose, heart rate variability, blood potential of hydrogen, hydration status, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, coughing and sneezing, gastrointestinal motility, gastrointestinal imaging, airway bacteria, edema, mental status, sweat, circulating tumor cells, autonomic tone, circadian rhythm, and / or menstrual cycle.

[0082] The biomarkers may relate to physiological systems, which may include, but are not limited to, behavioral and psychological, cardiovascular, renal, dermatological, nervous, gastrointestinal, respiratory, endocrine, immune, oncological, musculoskeletal, and / or reproductive systems. Information from the biomarkers may be determined and / or used, for example, by the computer-implemented patient and surgical system 100. Information from the biomarkers may be determined and / or used, for example, by the computer-implemented patient and surgical system 100 to improve the system and / or improve patient outcomes. One or more sensing systems 111, 115, biomarkers, and physiological systems are described in more detail in U.S. patent application Ser. No. 17 / 156,287 (Attorney Docket No. END9290USNP1), entitled "METHOD OF ADJUSTING A SURGICAL PARAMETER BASED ON BIOMARKER MEASUREMENTS," filed on January 22, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0083] FIG. 2 shows an example of a surgical system 202 in an operating room. As illustrated in FIG. 2, a patient is being operated on by one or more health care professionals (HCPs). The HCPs are monitored by one or more HCP sensing systems 220 worn by the HCPs. The HCPs and the environment surrounding the HCPs may also be monitored by one or more environmental sensing systems including, for example, a set of cameras 221, a set of microphones 222, and other sensors that may be deployed in the operating room. The HCP sensing systems 220 and the environmental sensing systems may communicate with a surgical hub 206, which may then communicate with one or more cloud servers 209 of a cloud computing system 208, as shown in FIG. 1. The environmental sensing systems may be used to measure one or more environmental attributes, such as HCP position within the operating room, HCP movement, ambient noise within the operating room, temperature / humidity within the operating room, etc.

[0084] As illustrated in FIG. 2 , a primary display 223 and one or more audio output devices (e.g., speaker 219) are positioned in the sterile field for visibility to the operator at the operating table 224. Additionally, a visualization / notification tower 226 is positioned outside the sterile field. The visualization / notification tower 226 may include a first non-sterile human interactive device (HID) 227 and a second non-sterile HID 229 facing opposite each other. The HIDs may be displays or displays with touchscreens that allow a human to directly interface with the HIDs. A human interface system guided by the surgical hub 206 may be configured to utilize the HIDs 227, 229, and 223 to coordinate information flow to operators inside and outside the sterile field. In one example, the surgical hub 206 may cause an HID (e.g., primary HID 223) to display notifications and / or information regarding the patient and / or surgical procedure steps. In one example, the surgical hub 206 may prompt and / or receive input from a person in the sterile field or non-sterile area. In one example, the surgical hub 206 may cause the HID to display a snapshot of the surgical site as recorded by the imaging device 230 on the non-sterile HID 227 or 229 while maintaining a live view of the surgical site on the main HID 223. The snapshot on the non-sterile display 227 or 229 may, for example, allow a non-sterile operator to perform diagnostic steps related to the surgical procedure.

[0085] In one aspect, the surgical hub 206 may be configured to send diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 to a primary display 223 in the sterile field for viewing by a sterile operator at the operating table. In one example, the input may be in the form of a correction to a snapshot displayed on the non-sterile display 227 or 229, which may be sent by the surgical hub 206 to the primary display 223.

[0086] 2, a surgical instrument 231 is used as part of a surgical system 202 in a surgical procedure. A hub 206 may be configured to coordinate information flow to the display of the surgical instrument 231, as described, for example, in U.S. Patent Application Publication No. 2019-0200844(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 can be sent by the hub 206 to a surgical instrument display in the sterile field, where it can be viewed by the operator of the surgical instrument 231. Exemplary surgical instruments suitable for use with surgical system 202 are described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety.

[0087] FIG. 2 illustrates an example of a surgical system 202 being used to perform a surgical procedure on a patient lying on an operating table 224 in an operating room 235. A robotic system 234 may be used as part of the surgical system 202 in the surgical procedure. The robotic system 234 may include a surgeon's console 236, a patient side cart 232 (surgical robot), and a surgical robot hub 233. The patient side cart 232 can manipulate at least one detachably coupled surgical tool 237 through a minimally invasive incision within the patient's body while the surgeon views the surgical site through the surgeon's console 236. Images of the surgical site can be acquired by a medical imaging device 230, which can be manipulated by the patient side cart 232 to orient the imaging device 230. The robotic hub 233 can be used to process the images of the surgical site and then display them to the surgeon through the surgeon's console 236.

[0088] Other types of robotic systems can be readily adapted for use with surgical system 202. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0201137(A1), entitled "METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety.

[0089] Various examples of cloud-based analytics methods that may be performed by cloud computing system 208 and that are suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0206569(A1), entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,403), the disclosure of which is incorporated herein by reference in its entirety.

[0090] In various embodiments, the imaging device 230 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, charge-coupled device (CCD) sensors and complementary metal-oxide semiconductor (CMOS) sensors.

[0091] The optics of the imaging device 230 may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate a portion of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0092] The one or more illumination sources can be configured to emit electromagnetic energy in the visible spectrum as well as the invisible spectrum. The visible spectrum, sometimes referred to as the optical spectrum or luminous spectrum, is the portion of the electromagnetic spectrum that is visible to (i.e., detectable by) the human eye and is sometimes referred to as visible light or simply light. The human eye typically responds to wavelengths in air between about 380 nm and about 750 nm.

[0093] The invisible spectrum (e.g., non-radiative spectrum) is the portion of the electromagnetic spectrum located below and above the visible spectrum (i.e., wavelengths less than about 380 nm and greater than about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, which constitutes invisible infrared (IR), microwave, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, which constitutes invisible ultraviolet, x-ray, and gamma-ray electromagnetic radiation.

[0094] In various aspects, the imaging device 230 is configured for use in minimally invasive procedures. Examples of imaging devices suitable for use with the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, cholangioscopes, colonoscopes, duodenoscopes, enteroscopes, esophagogastroduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngological-nephroscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0095] The imaging device may use multispectral monitoring to distinguish between topography and underlying structures. Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. Wavelengths can be separated by filters or by using instruments sensitive to specific wavelengths, including frequencies beyond the visible light range, e.g., IR and UV light. Spectral imaging allows for the extraction of additional information that cannot be captured by the red, green, and blue receptors of the human eye. The use of multispectral imaging is described in detail under the heading "Advanced Imaging Acquisition Module" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Multispectral monitoring can be a useful tool for repositioning the surgical field after the surgical task is complete to perform one or more of the tests described above on the treated tissue. It is self-evident that strict sterilization of the operating room and surgical equipment is necessary during any surgical procedure. The strict hygiene and sterilization conditions required in an "operating room," i.e., an operating room or procedure room, require the highest possible sterility of all medical devices and equipment. Part of the sterilization process is the need to sterilize everything that comes into contact with the patient or enters the sterile field, including the imaging device 230 and its accessories and components. It will be understood that the sterile field may be considered a specific area, such as in a tray or on a sterile towel, that is deemed free of microorganisms, or the sterile field may be considered the area immediately surrounding the patient being prepared for the surgical procedure. The sterile field may include properly clothed and hand-washed team members, as well as all equipment and fixtures in the area.

[0096] The wearable sensing system 211 illustrated in FIG. 1 may include one or more sensing systems, such as an HCP sensing system 220 as shown in FIG. 2. The HCP sensing system 220 may include a sensing system that monitors and detects a set of physical conditions and / or a set of physiological conditions of a healthcare professional (HCP). An HCP may generally be one or more healthcare professionals assisting a surgeon or other healthcare provider. In one example, the sensing system 220 may measure a set of biomarkers to monitor the HCP's heart rate. In one example, the sensing system 220 worn on the surgeon's wrist (e.g., a watch or wristband) may use an accelerometer to detect hand movement and / or shaking and determine tremor magnitude and frequency. The sensing system 220 may transmit measurement data associated with the set of biomarkers and data associated with the surgeon's physical condition to the surgical hub 206 for further processing. One or more environmental sensing devices may transmit environmental information to the surgical hub 206. For example, the environmental sensing device may include a camera 221 for detecting the HCP's hand / body position. The environmental sensing devices may include a microphone 222 for measuring ambient noise within the operating room. Other environmental sensing devices may include devices such as a thermometer to measure temperature and a hygrometer to measure ambient humidity within the operating room. The surgical hub 206, alone or in communication with a cloud computing system, may use the surgeon biomarker measurement data and / or environmental sensing information to modify the control algorithms of handheld instruments or the average latency of a robotic interface, for example, to minimize tremor. In one example, the HCP sensing system 220 may measure one or more surgeon biomarkers associated with the HCP and transmit measurement data associated with the surgeon biomarkers to the surgical hub 206.The HCP sensing system 220 may use one or more of the following RF protocols to communicate with the surgical hub 20006: Bluetooth, Bluetooth Low-Energy (BLE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low Power Wireless Personal Area Network (6LoWPAN), Wi-Fi. Surgeon biomarkers may include one or more of stress, heart rate, etc. Environmental measurements from the operating room may include ambient noise levels associated with the surgeon or patient, surgeon and / or staff movement, surgeon and / or staff attention level, etc.

[0097] The surgical hub 206 may use the surgeon biomarker measurement data associated with the HCP to adaptively control one or more surgical instruments 231. For example, the surgical hub 206 may send a control program to the surgical instrument 231 to control its actuators to limit or compensate for fatigue and the use of fine motor skills. The surgical hub 206 may send the control program based on situational awareness and / or context regarding the importance or criticality of the task. The control program may instruct the instrument to change operation to provide more control when control is needed.

[0098] FIG. 3 illustrates an exemplary surgical system 302 having a surgical hub 306. The surgical hub 306 may be paired with a wearable sensing system 311, an environmental sensing system 315, a human interface system 312, a robotic system 313, and an intelligent instrument 314 via a modular control. The hub 306 includes a display 348, an imaging module 349, a generator module 350, a communications module 356, a processor module 357, a storage array 358, and an operating room mapping module 359. In certain embodiments, as illustrated in FIG. 3, the hub 306 further includes a smoke evacuation module 354 and / or a suction / irrigation module 355. The various modules and systems may be connected to the modular control either directly via a router or via the communications module 356. Operating room devices may be coupled to cloud computing resources and data storage via the modular control. The human interface system 312 may include a display subsystem and a notification subsystem.

[0099] The modular controller may be coupled to a non-contact sensor module. The non-contact sensor module may use ultrasonic, laser-type, and / or similar non-contact measurement devices to measure the dimensions of the operating room and generate a map of the operating room. Other distance sensors may be used to determine the boundaries of the operating room. As described in U.S. Provisional Patent Application No. 62 / 611,341, filed December 28, 2017, entitled "INTERACTIVE SURGICAL PLATFORM," which is incorporated herein by reference in its entirety, under the heading "Surgical Hub Spatial Awareness Within an Operating Room," an ultrasound-based non-contact sensor module may scan the operating room by transmitting bursts of ultrasound and receiving echoes as the bursts reflect off the exterior walls of the operating room. The sensor module may be configured to determine the size of the operating room and adjust Bluetooth pairing distance limits. For example, a laser-based non-contact sensor module can scan an operating room by transmitting laser light pulses, receiving laser light pulses that reflect off the exterior walls of the operating room, and comparing the phase of the transmitted pulses with the received pulses to determine the size of the operating room and adjust Bluetooth pairing distance limits.

[0100] During surgical procedures, the application of energy to tissue for sealing and / or cutting is generally associated with smoke evacuation, aspiration of excess fluid, and / or irrigation of tissue. Fluid, power, and / or data lines from different sources often become tangled during surgical procedures. Addressing this issue can result in valuable time being lost during a surgical procedure. Untangling the lines may require unplugging them from their corresponding modules, which may require resetting the modules. The hub modular enclosure 360 ​​reduces the frequency of such line tangles by providing a unified environment for managing power, data, and fluid lines. Aspects of the present disclosure present a surgical hub 306 for use in surgical procedures involving the application of energy to tissue at a surgical site.

[0101] The surgical hub 306 includes a hub enclosure 360 ​​and a combination generator module slidably received within a docking station of the hub enclosure 360. The docking station includes data and power contacts. The combination generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component housed within a single unit. In one aspect, the combination generator module also includes a smoke evacuation component, at least one energy delivery cable for connecting the combination generator module to a surgical instrument, at least one smoke evacuation component configured to evacuate smoke, fluid, and / or particulates generated by the application of therapeutic energy to tissue, and a fluid line extending from the remote surgical site to the smoke evacuation component. In one aspect, the fluid line may be a first fluid line and a second fluid line may extend from the remote surgical site to the suction and irrigation module 355, which is slidably received within the hub enclosure 360. In one aspect, the hub enclosure 360 ​​may include a fluid interface.

[0102] Certain surgical procedures may require the application of two or more energy types to tissue. One energy type may be more beneficial for cutting tissue, while another, different energy type may be more beneficial for sealing tissue. For example, a bipolar generator may be used to seal tissue, while an ultrasonic generator may be used to cut the sealed tissue. Aspects of the present disclosure present a solution in which a hub modular enclosure 360 ​​is configured to house different generators and facilitate bidirectional communication between them. The hub modular enclosure 360 ​​may allow for rapid removal and / or replacement of various modules. Aspects of the present disclosure present a modular surgical enclosure for use in surgical procedures involving the application of energy to tissue. The modular surgical enclosure includes a first energy generator module configured to generate a first energy for application to tissue and a first docking station including a first docking port including first data contacts and first power contacts, wherein the first energy generator module is slidably movable into electrical engagement with the power contacts and the data contacts, and the first energy generator module is slidably movable out of electrical engagement with the first power contacts and the first data contacts. In addition to the above, the modular surgical enclosure also includes a second energy generator module configured to generate a second energy different from the first energy for application to tissue and a second docking station including a second docking port including second data contacts and second power contacts, wherein the second energy generator module is slidably movable into electrical engagement with the power contacts and the data contacts, and the second energy generator module is slidably movable out of electrical engagement with the second power contacts and the second data contacts. In addition, the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port configured to facilitate communication between the first energy generator module and the second energy generator module.Referring to FIG. 3 , an aspect of the disclosure is presented regarding a hub modular enclosure 360 ​​that allows for modular integration of a generator module 350, a smoke evacuation module 354, and a suction / irrigation module 355. The hub modular enclosure 360 ​​further facilitates bidirectional communication between the modules 359, 354, and 355. The generator module 350 can comprise integrated monopolar, bipolar, and ultrasonic components supported within a single housing unit that is slidably insertable into the hub modular enclosure 360. The generator module 350 can be configured to connect to a monopolar device 351, a bipolar device 352, and an ultrasonic device 353. Alternatively, the generator module 350 may comprise a series of monopolar, bipolar, and / or ultrasonic generator modules that interact through the hub modular enclosure 360. The hub modular enclosure 360 ​​may be configured to facilitate the insertion of multiple generators and bidirectional communication between generators docked to the hub modular enclosure 360 ​​so that the multiple generators act as a single generator.

[0103] FIG. 4 illustrates a surgical data network having a set of communication hubs configured to connect a set of sensing systems, environmental sensing systems, and other modular devices located in one or more operating rooms, patient recovery rooms, or rooms within a medical facility specially equipped for surgery to a cloud in accordance with at least one embodiment of the present disclosure.

[0104] 4, surgical hub system 460 may include a modular communications hub 465 configured to connect modular devices located within a medical facility to a cloud-based system (e.g., a cloud computing system 464, which may include a remote server 467 coupled to remote storage 468). The modular communications hub 465 and devices may be connected in a room within the medical facility specially equipped for surgical procedures. In one aspect, modular communications hub 465 may include a network hub 461 and / or a network switch 462 in communication with a network router 466. Modular communications hub 465 may be coupled to a local computer system 463 to provide local computer processing and data manipulation.

[0105] The computer system 463 may include a processor and a network interface. The processor may be coupled to a communication module, storage, memory, non-volatile memory, and input / output (I / O) interfaces via a system bus. The system bus can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures, including, but not limited to, a 9-bit bus, an Industrial Standard Architecture (ISA), a Micro-Channel Architecture (MSA), an Extended ISA (EISA), an Intelligent Drive Electronics (IDE), a VESA Local Bus (VLB), a Peripheral Component Interconnect (PCI), a USB, an Advanced Graphics Port (AGP), a Personal Computer Memory Card International Association (PCMCIA), a Small Computer Systems Interface (SCSI), or any other proprietary bus.

[0106] The processor may be any single-core or multi-core processor, such as that known by Texas Instruments under the trade name ARM Cortex. In one aspect, the processor may be, for example, an LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments, including on-chip memory of 256 KB of single-cycle flash memory or other non-volatile memory up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, 32 KB of single-cycle serial random access memory (SRAM), internal read-only memory (ROM) loaded with StellarisWare® software, 2 KB of electrically erasable programmable read-only memory (EEPROM), and / or one or more pulse width modulation (PWM) modules, one or more Quadrature Encoder Input (QEI) analog, one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, details of which are available in the product data sheet.

[0107] In one example, the processor may include a safety controller, which includes two controller-based families such as the TMS570 and RM4x, also known under the trade name Hercules ARM Cortex R4, manufactured by Texas Instruments. The safety controller may be specifically configured for IEC 61508 and ISO 26262 safety limit applications, among others, to provide advanced integrated safety mechanisms while offering scalable performance, connectivity, and memory options.

[0108] It should be understood that the computer system 463, in a suitable operating environment, may include software that acts as an intermediary between the described users and the basic computer resources. Such software may include an operating system. The operating system, which may be stored on disk storage, may function to control and allocate resources of the computer system. System applications may take advantage of resource management by the operating system through program modules and program data stored either in system memory or on disk storage. It should be understood that the various components described herein may be implemented with various operating systems or combinations of operating systems.

[0109] A user may input commands or information into the computer system 463 through input devices coupled to the I / O interface. Input devices may include, but are not limited to, pointing devices such as a mouse, trackball, stylus, or touchpad; keyboards; microphones; joysticks; gamepads; satellite dishes; scanners; TV tuner cards; digital cameras; digital video cameras; webcams; and the like. These and other input devices connect to the processor through the system bus via interface ports. Interface ports include, for example, serial ports, parallel ports, game ports, and USB. Output devices use some of the same types of ports as input devices. Thus, for example, a USB port may be used to provide input to the computer system 463 and to output information from the computer system 463 to an output device. Output adapters may be provided to illustrate that there may be several output devices, such as monitors, displays, speakers, and printers, among other output devices that may require special adapters. Output adapters may include, by way of example and not limitation, video and sound cards that provide a means of connection between the output device and the system bus. It should be noted that other devices and / or systems of devices, such as remote computers, may provide both input and output capabilities.

[0110] The computer system 463 may operate in a networked environment using logical connections to one or more remote computers, such as a cloud computer, or a local computer. The remote cloud computer may be a personal computer, a server, a router, a network PC, a workstation, a microprocessor-based appliance, a peer device, or other common network node, but typically includes many or all of the elements described with respect to a computer system. For simplicity, only a memory storage device is illustrated with the remote computer. The remote computer may be logically connected to the computer system through a network interface and subsequently physically connected via a communication connection. The network interface may encompass communication networks such as a local area network (LAN) and a wide area network (WAN). LAN technologies may include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet / IEEE 802.3, Token Ring / IEEE 802.5, etc. WAN technologies may include, but are not limited to, point-to-point links, circuit-switched networks such as Integrated Services Digital Networks (ISDN) and its variants, packet-switched networks, and Digital Subscriber Lines (DSL).

[0111] In various examples, computer system 463 may include an image processor, image processing engine, media processor, or any specialized digital signal processor (DSP) used to process digital images. The image processor may use parallel computing using Single Instruction, Multiple Data (SIMD) or Multiple Instruction, Multiple Data (MIMD) techniques to increase speed and efficiency. The digital image processing engine may perform a variety of tasks. The image processor may be a system on a chip with a multi-core processor architecture.

[0112] The communications connection may refer to the hardware / software used to connect the network interface to the bus. The communications connection is shown internal to computer system 463 for clarity of illustration, but can also be external to computer system 463. For purposes of illustration only, the hardware / software required to connect to the network interface may include internal and external technologies such as modems, including regular telephone-grade modems, cable modems, fiber optic modems, and DSL modems, ISDN adapters, and Ethernet cards. In some examples, the network interface may also be provided using an RF interface.

[0113] The surgical data network associated with the surgical hub system 460 may be configured as passive, intelligent, or switched. A passive surgical data network acts as a conduit for data, allowing it to go from one device (or segment) to another and to cloud computing resources. An intelligent surgical data network includes additional features that allow traffic to pass through the monitored surgical data network and configure each port within the network hub 461 or network switch 462. An intelligent surgical data network may be referred to as a manageable hub or switch. A switching hub reads the destination address of each packet and then forwards the packet to the correct port.

[0114] Modular devices 1a-1n located in an operating room may be coupled to a modular communication hub 465. Network hub 461 and / or network switch 462 may be coupled to a network router 466 to connect devices 1a-1n to a cloud computing system 464 or a local computer system 463. Data associated with devices 1a-1n may be transferred to a cloud-based computer via the router for remote data processing and manipulation. Data associated with devices 1a-1n may also be transferred to local computer system 463 for local data processing and manipulation. Modular devices 2a-2m located in the same operating room may also be coupled to network switch 462. Network switch 462 may be coupled to network hub 461 and / or to network router 466 to connect devices 2a-2m to a cloud 464. Data associated with devices 2a-2m may be transferred to cloud computing system 464 via network router 466 for data processing and manipulation. Data associated with the devices 2a-2m may also be transferred to a local computer system 463 for local data processing and manipulation.

[0115] 4, a computing system such as surgical hub system 460 may include a modular communications hub 465 configured to connect modular devices (e.g., surgical devices) located within a medical facility to a cloud-based system (e.g., a cloud computing system 464 that may include a remote server 467 coupled to remote storage 468). The modular communications hub 465 and devices may be connected in a room within the medical facility specially equipped for surgical procedures. In one aspect, the modular communications hub 465 may include a network hub 461 and / or a network switch 462 in communication with a network router 466. The modular communications hub 465 may be coupled to a local computer system (e.g., a computing device) to provide local computer processing and data manipulation.

[0116] FIG. 5 illustrates a logic diagram of a surgical instrument or surgical tool control system 520 according to one or more embodiments of the present disclosure. The surgical instrument or surgical tool may be configurable. The surgical instrument may include surgical fasteners specific to the procedure at hand, such as imaging devices, surgical staplers, energy devices, endocutter devices, etc. For example, the surgical instrument may include any of a power stapler, a power stapler generator, an energy device, an advanced energy device, an advanced energy jaw device, an endocutter clamp, an energy device generator, an operating room imaging system, a smoke evacuation device, a suction irrigation device, an insufflation system, etc. The system 520 may include control circuitry. The control circuitry may include a microcontroller 521 with a processor 522 and a memory 523. For example, one or more of sensors 525, 526, 527 provide real-time feedback to the processor 522. A motor 530 driven by a motor driver 529 operably couples a longitudinally movable displacement member to drive the I-beam knife element. The tracking system 528 may be configured to determine the position of the longitudinally movable displacement member. The position information may be provided to the processor 522, which may be programmed or configured to determine the position of the longitudinally movable drive member, as well as the positions of the firing member, firing bar, and I-beam knife element. Additional motors may be provided to the tool driver interface to control the firing of the I-beam, the movement of the obturator tube, the rotation of the shaft, and the articulation. The display 524 may display various operating states of the instrument and may also include touch screen functionality for data entry. Information displayed on the display 524 may be overlaid with images acquired via the endoscopic imaging module.

[0117] Microcontroller 521 may be any single-core or multi-core processor, such as those known under the trade name ARM Cortex manufactured by Texas Instruments. In one aspect, main microcontroller 521 may be, for example, an LM4F230H5QR ARM Cortex-M4F Processor Core available from Texas Instruments, with on-chip memory of 256 KB of single-cycle flash memory or other non-volatile memory up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, 32 KB of single-cycle SRAM, internal ROM with StellarisWare® software, 2 KB of EEPROM, one or more PWM modules, one or more QEI analog, and / or one or more 12-bit ADCs with 12 analog input channels, details of which are available in the product datasheet.

[0118] The microcontroller 521 may include a safety controller, which includes two controller-based families such as the TMS570 and RM4x, also known under the trade name Hercules ARM Cortex R4, manufactured by Texas Instruments. The safety controller can be specifically configured for IEC 61508 and ISO 26262 safety limit applications, among others, to provide advanced integrated safety mechanisms while offering scalable performance, connectivity, and memory options.

[0119] The microcontroller 521 may be programmed to perform various functions, such as precise control over the speed and position of the knife and articulation system. In one embodiment, the microcontroller 521 may include a processor 522 and memory 523. The electric motor 530 may be a brushed direct current (DC) motor with a gearbox and mechanical linkage to the articulation or knife system. In one embodiment, the motor driver 529 may be an A3941 available from Allegro Microsystems, Inc. Other motor drivers may be easily substituted for use in the tracking system 528 with an absolute positioning system. A detailed description of the absolute positioning system is provided in U.S. Patent Application Publication No. 2017 / 0296213, entitled "SYSTEMS AND METHODS FOR CONTROLLING A SURGICAL STAPLING AND CUTTING INSTRUMENT," published October 19, 2017, which is incorporated herein by reference in its entirety.

[0120] The microcontroller 521 may be programmed to provide precise control over the velocity and position of the displacement members and articulation system. The microcontroller 521 may be configured to calculate a response in the microcontroller 521 software. The calculated response may be compared to the measured response of the actual system to obtain an "observed" response, which is used for actual feedback decision-making. The observed response may be a suitably adjusted value that balances the smooth, continuous nature of the simulated response with the measured response, which can detect external influences on the system.

[0121] The motor 530 may be controlled by a motor driver 529 and may be used by the surgical instrument or tool firing system. In various forms, the motor 530 may be a brushed DC drive motor having a maximum rotational speed of approximately 25,000 RPM. In some examples, the motor 530 may include a brushless motor, a cordless motor, a synchronous motor, a stepper motor, or any other suitable electric motor. The motor driver 529 may include, for example, an H-bridge driver including field-effect transistors (FETs). The motor 530 may be powered by a power supply assembly releasably attached to the handle assembly or tool housing to provide control power to the surgical instrument or tool. The power supply assembly may include a battery, which may include multiple battery cells connected in series, that may be used as a power source to power the surgical instrument or tool. Under certain circumstances, the battery cells of the power supply assembly may be replaceable and / or rechargeable. In at least one example, the battery cells may be lithium-ion batteries, which may be connectable to and detachable from the power supply assembly.

[0122] The motor driver 529 may be the A3941, available from Allegro Microsystems, Inc. The A3941 may be a full-bridge controller for use with external N-channel power metal-oxide semiconductor field-effect transistors (MOSFETs) specifically designed for inductive loads, such as brushed DC motors. The driver 529 may include an intrinsic charge pump regulator, which provides full (>10V) gate drive for battery voltages up to 7V, allowing the A3941 to operate with reduced gate drive down to 5.5V. A bootstrap capacitor may be used to provide the required battery supply voltage above the N-channel MOSFET. An internal charge pump for the high-side drive allows DC (100% duty cycle) operation. The full-bridge may be driven in fast or slow decay mode using diode or synchronous rectification. In slow decay mode, current recirculation is possible through either the high-side or low-side FET. The power FETs may be protected from shoot-through by a resistor-adjustable dead time. Integrated diagnostics indicate undervoltage, overtemperature, and power bridge faults and can be configured to protect the power MOSFETs under most short circuit conditions. Other motor drivers may be easily substituted for use in tracking system 528 with an absolute positioning system.

[0123] The tracking system 528 may include controlled motor drive circuitry including a position sensor 525 according to one aspect of the present disclosure. The position sensor 525 for the absolute positioning system may provide a unique position signal corresponding to the position of the displacement member. In some examples, the displacement member may represent a longitudinally movable drive member including a rack of drive teeth for meshing engagement with a corresponding drive gear of a gear reducer assembly. In some examples, the displacement member may represent a firing member that may be adapted and configured to include a rack of drive teeth. In some examples, the displacement member may represent a firing bar or an I-beam, each of which may be adapted and configured to include a rack of drive teeth. Thus, as used herein, the term displacement member may be used generally to refer to any movable member of a surgical instrument or tool, such as a drive member, firing member, firing bar, I-beam, or any element that can be displaced. In one aspect, a longitudinally movable drive member may be coupled to a firing member, firing bar, and I-beam. Thus, the absolute positioning system may actually track the linear displacement of the I-beam by tracking the linear displacement of the longitudinally movable drive member. In various aspects, the displacement member may be coupled to any suitable position sensor 525 for measuring linear displacement. Thus, the longitudinally movable drive member, firing member, firing bar, or I-beam, or combinations thereof, may be coupled to any suitable linear displacement sensor. The linear displacement sensor may include a contact displacement sensor or a non-contact displacement sensor.The linear displacement sensor may include a linear variable differential transformer (LVDT), a differential variable reluctance transducer (DVRT), a slide potentiometer, a magnetic sensing system with a movable magnet and a series of linearly arranged Hall effect sensors, a magnetic sensing system with a fixed magnet and a series of movable linearly arranged Hall effect sensors, an optical sensing system with a movable light source and a series of linearly arranged photodiodes or photodetectors, an optical sensing system with a fixed light source and a series of movable linearly arranged photodiodes or photodetectors, or any combination thereof.

[0124] The electric motor 530 may include a rotatable shaft operably interfaced with a gear assembly mounted for meshing engagement with a set of drive teeth, or rack, on the displacement member. The sensor element may be operably coupled to the gear assembly such that one rotation of the position sensor 525 element corresponds to several linear longitudinal translations of the displacement member. The gearing and sensor arrangement may be connected to a linear actuator by a rack-and-pinion arrangement or to a rotary actuator by a spur gear or other connection. A power source may provide power to the absolute positioning system, and an output indicator may display the output of the absolute positioning system. The displacement member may represent a longitudinally movable drive member with a rack of drive teeth formed thereon for meshing engagement with a corresponding drive gear of a gear reducer assembly. The displacement member may represent a longitudinally movable firing member, a firing bar, an I-beam, or a combination thereof.

[0125] One revolution of the sensor element associated with the position sensor 525 may correspond to a linear longitudinal displacement d1 of the displacement member, where d1 is the linear longitudinal distance the displacement member travels from point "a" to point "b" after one revolution of the sensor element coupled to the displacement member. The sensor arrangement may be connected via a gear reduction such that the position sensor 525 completes one or more revolutions for a full stroke of the displacement member. The position sensor 525 may complete multiple revolutions for a full stroke of the displacement member.

[0126] A series of switches (where n is an integer greater than 1) may be used alone or in combination with a gear reduction to provide a unique position signal for two or more revolutions of the position sensor 525. The state of the switches may be fed back to the microcontroller 521, which applies logic to determine a unique position signal corresponding to the longitudinal linear displacement d1+d2+...dn of the displacement member. The output of the position sensor 525 is provided to the microcontroller 521. The position sensor 525 of the sensor arrangement may comprise a magnetic sensor, an analog rotary sensor such as a potentiometer, or an array of analog Hall effect elements that output a unique combination of position signals or values.

[0127] The position sensor 525 may comprise any number of magnetic sensing elements, such as magnetic sensors classified according to whether they measure the total magnetic field or vector components of the magnetic field. The technologies used to produce both types of magnetic sensors may encompass many aspects of physics and electronics. Technologies used to sense magnetic fields may include search coils, fluxgates, optical pumping, nuclear precession, SQUIDs, Hall effect, anisotropic magnetoresistance, giant magnetoresistance, magnetic tunnel junctions, giant magnetoimpedance, magnetostrictive / piezoelectric composites, magnetodiodes, magnetotransistors, optical fiber, magneto-optical, and microelectromechanical systems-based magnetic sensors, among others.

[0128] The position sensor 525 of the tracking system 528 with an absolute positioning system may comprise a magnetic rotary absolute positioning system. The position sensor 525 may be implemented as an AS5055EQFT single-chip magnetic rotary position sensor available from Austria Microsystems, AG. The position sensor 525 interfaces with the microcontroller 521 to provide the absolute positioning system. The position sensor 525 may be a low-voltage, low-power component and may include four Hall-effect elements in an area of ​​the position sensor 525 that may be located above the magnet. A high-resolution ADC and a smart power management controller may also be provided on-chip. A coordinate rotation digital computer (CORDIC) processor, also known as the digit-by-digit method and the Boulder algorithm, may be provided to implement simple, efficient algorithms for calculating hyperbolic and trigonometric functions, requiring only addition, subtraction, bit shifting, and table lookup operations. The angular position, alarm bits, and magnetic field information may be transmitted to the microcontroller 521 through a standard serial communications interface, such as a serial peripheral interface (SPI) interface. The position sensor 525 may provide 12-bit or 14-bit resolution and may be an AS5055 chip provided in a small QFN 16-pin 4x4x0.85mm package.

[0129] The tracking system 528, which comprises an absolute positioning system, may include and / or be programmed to implement a feedback controller, such as a PID, state feedback, and adaptive controller. The power supply converts the signal from the feedback controller into a physical input to the system, in this case a voltage. Other examples include PWM of voltage, current, and force. In addition to the position measured by the position sensor 525, other sensors may be provided to measure physical parameters of the physical system. In some embodiments, other sensors may include sensor arrangements such as those described in U.S. Pat. No. 9,345,481, issued May 24, 2016, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety; U.S. Patent Application Publication No. 2014 / 0263552, published September 18, 2014, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety; and U.S. Patent Application No. 15 / 628,175, filed June 20, 2017, entitled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," which is incorporated herein by reference in its entirety. In a digital signal processing system, the absolute positioning system is coupled to a digital data acquisition system, where the output of the absolute positioning system has a finite resolution and sampling frequency. The absolute positioning system may include comparison and combination circuitry to combine the calculated response with the measured response using algorithms such as weighted averages and theoretical control loops that drive the calculated response towards the measured response. The calculated response of the physical system may take into account properties such as mass, inertia, viscous friction, induced drag, etc., in order to predict what the state and output of the physical system will be given knowledge of the input.

[0130] The absolute positioning system may provide the absolute position of the displacement member upon powering up of the instrument without forcing the displacement member to retract or advance to a reset (zero or home) position, as may be required with conventional rotary encoders that simply count the number of forward or backward steps taken by the motor 530 to infer the position of the device actuator, drive bar, knife, etc.

[0131] Sensor 526, such as a strain gauge or micro-strain gauge, may be configured to measure one or more parameters of the end effector, such as the amplitude of strain exerted on the anvil during clamping, which can be indicative of the closure force applied to the anvil. The measured strain may be converted to a digital signal and provided to processor 522. Alternatively, or in addition to sensor 526, sensor 527, such as a load sensor, can measure the closure force applied to the anvil by the closure drive system. For example, sensor 527, such as a load sensor, can measure the firing force applied to the I-beam during the firing stroke of the surgical instrument or tool. The I-beam is configured to engage a wedge-shaped sled, which is configured to cam the staple driver upward and drive the staples into deforming contact with the anvil. The I-beam may also include a sharp cutting edge that can be used to cut tissue as the I-beam is advanced distally by the firing bar. Alternatively, a current sensor 531 can be used to measure the current drawn by motor 530. The force required to advance the firing member may correspond, for example, to the current drawn by motor 530. The measured force may be converted to a digital signal and provided to processor 522.

[0132] For example, a strain gauge sensor 526 can be used to measure the force applied to tissue by the end effector. A strain gauge can be coupled to the end effector to measure the force applied by the end effector to the tissue being treated. A system for measuring the force applied to grasped tissue by the end effector may include a strain gauge sensor 526, such as, for example, a micro-strain gauge, which can be configured to measure one or more parameters of the end effector. In one aspect, the strain gauge sensor 526 can measure the amplitude or magnitude of strain exerted on the jaw members of the end effector during clamping, which can indicate tissue compression. The measured strain can be converted to a digital signal and provided to the processor 522 of the microcontroller 521. A load sensor 527 can measure the force used to operate the knife element, for example, to cut tissue captured between the anvil and the staple cartridge. A magnetic field sensor can be used to measure the thickness of the captured tissue. The magnetic field sensor's measurements can also be converted to a digital signal and provided to the processor 522.

[0133] Measurements of tissue compression, tissue thickness, and / or force required to close the end effector on the tissue, measured by sensors 526, 527, respectively, may be used by microcontroller 521 to characterize a selected position of the firing member and / or a corresponding value of firing member velocity. In one example, memory 523 may store techniques, equations, and / or look-up tables that may be used by microcontroller 521 during the evaluation.

[0134] The surgical instrument or tool control system 520 may also include wired or wireless communication circuitry for communicating with a surgical hub, such as surgical hub 460, as shown in FIG.

[0135] FIG. 6 illustrates an exemplary surgical system 680 according to the present disclosure and may include a surgical instrument 682 that can communicate with a console 694 or a portable device 696 through a local area network 692 and / or a cloud network 693 via wired and / or wireless connections. The console 694 and the portable device 696 may be any suitable computing devices. The surgical instrument 682 may include a handle 697, an adapter 685, and a loading unit 687. The adapter 685 releasably couples to the handle 697, and the loading unit 687 releasably couples to the adapter 685 such that the adapter 685 transmits force from the drive shaft to the loading unit 687. The adapter 685 or the loading unit 687 may include a force gauge (not explicitly shown) disposed therein to measure force applied to the loading unit 687. The loading unit 687 may include an end effector 689 having a first jaw 691 and a second jaw 690. The loading unit 687 may be an in vivo loading or multi-firing loading unit (MFLU) that allows a clinician to fire multiple fasteners multiple times without having to remove the loading unit 687 from the surgical site to reload it.

[0136] The first jaw 691 and the second jaw 690 may be configured to clamp tissue therebetween, fire fasteners through the clamped tissue, and sever the clamped tissue. The first jaw 691 may be configured to fire at least one fastener multiple times, or may be configured to contain a replaceable multi-fire fastener cartridge containing multiple fasteners (e.g., staples, clips, etc.) that may be fired two or more times before being replaced. The second jaw 690 may include an anvil that deforms or otherwise secures fasteners as they are ejected from the multi-fire fastener cartridge.

[0137] The handle 697 may include a motor coupled to the drive shaft to affect rotation of the drive shaft. The handle 697 may include a control interface for selectively activating the motor. The control interface may include buttons, switches, levers, sliders, a touch screen, and any other suitable input mechanism or user interface that may be engaged by a clinician to activate the motor.

[0138] The control interface of the handle 697 may be in communication with a controller 698 of the handle 697 to selectively activate the motors to affect rotation of the drive shaft. The controller 698 may be disposed within the handle 697 and configured to receive input from the control interface and adapter data from the adapter 685 or loading unit data from the loading unit 687. The controller 698 may analyze the input from the control interface and the data received from the adapter 685 and / or the loading unit 687 to selectively activate the motors. The handle 697 may also include a display viewable by a clinician while using the handle 697. The display may be configured to display a portion of the adapter or loading unit data before, during, or after firing of the instrument 682.

[0139] The adapter 685 may include an adapter identification device 684 disposed therein, and the loading unit 687 may include a loading unit identification device 688 disposed therein. The adapter identification device 684 may be in communication with a controller 698, and the loading unit identification device 688 may be in communication with the controller 698. It will be appreciated that the loading unit identification device 688 may be in communication with the adapter identification device 684, which relays or passes communications from the loading unit identification device 688 to the controller 698.

[0140] The adapter 685 may also include multiple sensors 686 (one shown) disposed about its periphery to detect various conditions of the adapter 685 or the environment (e.g., whether the adapter 685 is connected to the loading unit, whether the adapter 685 is connected to the handle, whether the drive shaft is rotating, the torque of the drive shaft, the strain on the drive shaft, the temperature within the adapter 685, the number of times the adapter 685 has been fired, the peak force of the adapter 685 during firing, the total amount of force applied to the adapter 685, the peak retract force of the adapter 685, the number of times the adapter 685 has been paused during firing, etc.). The multiple sensors 686 may provide input to the adapter identification device 684 in the form of data signals. The data signals of the multiple sensors 686 may be stored in the adapter identification device 684 or may be used to update adapter data stored in the adapter identification device 684. The data signals of the multiple sensors 686 may be analog or digital. The multiple sensors 686 may include a force gauge that measures the force exerted on the loading unit 687 during firing.

[0141] The handle 697 and adapter 685 may be configured to interconnect the adapter identification device 684 and the loading unit identification device 688 with the controller 698 via an electrical interface. The electrical interface may be a direct electrical interface (i.e., including electrical contacts that engage with each other to transmit energy and signals therebetween). Additionally or alternatively, the electrical interface may be a contactless electrical interface for wirelessly transmitting (e.g., inductively transmitting) energy and signals therebetween. It is also contemplated that the adapter identification device 684 and the controller 698 may communicate wirelessly with each other via a wireless connection that is separate from the electrical interface.

[0142] The handle 697 may include a transceiver 683 configured to transmit instrument data from the controller 698 to other components of the system 680 (e.g., the LAN 20292, the cloud 693, the console 694, or the portable device 696). The controller 698 may also transmit instrument data and / or measurement data associated with one or more sensors 686 to the surgical hub. The transceiver 683 may receive data (e.g., cartridge data, loading unit data, adapter data, or other notifications) from the surgical hub 670. The transceiver 683 may also receive data (e.g., cartridge data, loading unit data, or adapter data) from other components of the system 680. For example, the controller 698 may transmit instrument data to the console 694, including the serial number of an attached adapter (e.g., adapter 685) attached to the handle 697, the serial number of a loading unit (e.g., loading unit 687) attached to the adapter 685, and the serial number of a multi-fire fastener cartridge loaded in the loading unit. The console 694 may then return data associated with the attached cartridge, loading unit, and adapter (e.g., cartridge data, loading unit data, or adapter data), respectively, to the controller 698. The controller 698 may display a message on a local instrument display or may send a message via the transceiver 683 to the console 694 or portable device 696, respectively, to display the message on the display 695 or portable device screen.

[0143] 7A illustrates a surgical system 700 that may include a matrix of surgical information. This surgical information may include any discrete atoms of information related to a surgical procedure. Generally described, such surgical information may include information related to the context and scope of the surgical procedure itself (e.g., healthcare information 728). Such information may include data such as, for example, procedure data and patient record data. The procedure data and / or patient record data may be associated with an associated healthcare data system 716 that is in communication with the surgical computing device 704.

[0144] The procedure data may include information related to the instruments and / or replaceable instrument parts used in a given procedure, such as, for example, a master list. The surgical computing device 704 may record (e.g., capture barcode scans) the instruments and / or replaceable instrument components used in the procedure. Such surgical information may be used by an algorithm to verify that the proper configuration of surgical instruments and / or replaceable components is being used. See U.S. Patent Application Publication No. 2020-0405296(A1), entitled "PACKAGING FOR A REPLACEABLE COMPONENT OF A SURGICAL STAPLING SYSTEM," filed June 30, 2019 (U.S. Patent Application No. 16 / 458,103), the contents of which are incorporated herein by reference in their entirety.

[0145] For example, patient record data may be suitable for use in modifying the configuration of a particular surgical device. For example, patient data may be used to understand and improve algorithmic behavior of a surgical device. In one example, a surgical stapler can adjust operating parameters related to compression, speed of operation, location of use, and feedback based on information in the patient record (e.g., information indicative of a particular patient's tissue and / or tissue characteristics). See U.S. Patent Application Publication No. 2019-0200981(A1), filed December 4, 2018, entitled "METHOD OF COMPRESSING TISSUE WITHIN A STAPLING DEVICE AND SIMULTANEOUSLY DISPLAYING THE LOCATION OF THE TISSUE WITHIN THE JAWS" (U.S. Patent Application No. 16 / 209,423), the contents of which are incorporated herein by reference in their entirety.

[0146] The surgical information may include information related to the configuration and / or control of devices being used in the procedure (e.g., device operation information 729). Such device operation information 729 may include information regarding the initial configuration of a surgical device. Device operation information 729 may include information regarding changes to the configuration of a surgical device. Device operation information 729 may include information regarding controls sent to the device from the surgical computing device 704 and the information flow associated with such controls.

[0147] Surgical information may include information generated during the surgery itself (e.g., surgical information 727). Such surgical information 727 may include any information generated by a surgical data source 726. The data source 726 may include any device within the surgical context that may generate useful surgical information 727. This surgical information 727 may present itself as an observable quality of the data source 726. The observable quality may include static qualities such as a device's model number, serial number, etc. The observable quality may include dynamic qualities such as the state of a device's configurable settings. Surgical information 727 may present itself, for example, as the result of sensor observations. Sensor observations may include observations from specific sensors in the operating room, sensors for monitoring conditions such as the patient's condition, sensors embedded in surgical devices, etc. Sensor observations may include information used during the surgery, such as video, audio, etc. Surgical information 727 may present itself as device event data. The surgical device may generate notifications and / or log events, and such events may be included in the surgical information 727 for communication to the surgical computing device 704. The surgical information 727 may present itself, for example, as a result of manual recording. A medical professional may record during the procedure by asking the patient to take notes, capturing still images from the display, etc. The surgical data sources 726 may include, for example, modular devices (which may include, for example, sensors configured to detect parameters associated with the patient, the HCP, and the environment, and / or the modular devices themselves), local databases (e.g., a local EMR database containing patient records), patient monitoring devices (e.g., blood pressure (BP) monitors and electrocardiography (EKG) monitors), HCP monitoring devices, environmental monitoring devices, surgical instruments, surgical procedure support equipment, etc.

[0148] Intelligent surgical instruments can sense and measure certain operating parameters during the course of their operation. For example, intelligent surgical instruments, such as surgical robots, digital laparoscopic devices, and the like, may use such measurements to improve their operation, for example, limiting excessive compression, reducing collateral damage, minimizing tissue tension, optimizing application position, etc. See U.S. Patent Application Publication No. 2018-0049822(A1), filed August 16, 2016, entitled "CONTROL OF ADVANCEMENT RATE AND APPLICATION FORCE BASED ON MEASURED FORCES" (U.S. Patent Application No. 15 / 237,753), the contents of which are incorporated herein by reference in their entirety. Such surgical information can be communicated to the surgical computing device 704.

[0149] The surgical computing device 704 may be configured to derive contextual information regarding the surgical procedure from the data based, for example, on a particular combination of received data or a particular order in which data is received from the data sources 726. Contextual information inferred from the received data may include, for example, the type of surgical procedure being performed, the particular step of the surgical procedure the surgeon is performing, the type of tissue being operated on, or the body cavity that is the target of the procedure. This ability, according to some aspects of the surgical computing device 704, to derive or infer information regarding the surgical procedure from the received data may be referred to as “situational awareness.” For example, the surgical computing device 704 may incorporate a situational awareness system, which is hardware and / or programming associated with the surgical computing device 704 that derives contextual information regarding the surgical procedure from the received data and / or surgical planning information received from the edge computing system 714 or healthcare data system 716 (e.g., an enterprise cloud server). Such situational awareness capabilities may be used to generate surgical information (such as control and / or configuration information) based on sensed conditions and / or use. See U.S. Patent Application Publication No. 2019-0104919(A1) (U.S. Patent Application No. 16 / 209,478), filed December 4, 2018, entitled "METHOD FOR SITUATIONAL AWARENESS FOR SURGICAL NETWORK OR SURGICAL NETWORK CONNECTED DEVICE CAPABLE OF ADJUSTING FUNCTION BASED ON A SENSED SITUATION OR USAGE," the contents of which are incorporated herein by reference in their entirety.

[0150] During operation, this matrix of surgical information may exist as one or more information flows. For example, surgical information may flow from a surgical data source 726 to the surgical computing device 704. Surgical information may flow from the surgical computing device 704 to a surgical data source 726 (e.g., a surgical device). Surgical information may flow between the surgical computing device 704 and one or more healthcare data systems 716. Surgical information may flow between the surgical computing device 704 and one or more edge computing devices 714. Aspects of the information flows, including, for example, information flow endpoints, information storage, data interpretation, etc., may be managed for the surgical system 700 (e.g., for the medical facility). See U.S. Patent Application Publication No. 2019-0206564(A1), entitled "METHOD FOR FACILITY DATA COLLECTION AND INTERPRETATION," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,490), the entire contents of which are incorporated herein by reference.

[0151] The surgical information as presented in one or more information flows may be used in conjunction with one or more artificial intelligence (AI) systems to further enhance the operation of the surgical system 700. For example, a machine learning system, such as those described herein, may operate on one or more of the information flows to further enhance the operation of the surgical system 700.

[0152] 7B shows an exemplary computer-implemented surgical system 730 having multiple information flows 732. The surgical computing device 704 may communicate with and / or incorporate one or more surgical data sources. For example, the imaging module 733 (and endoscope) may exchange surgical information with the surgical computing device 704. Such information may include information from the imaging module 733 (and endoscope), such as video information, current settings, system status information, etc. The imaging module 733 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (software / firmware, etc.).

[0153] For example, the generator module 734 (and corresponding energy devices) may exchange surgical information with the surgical computing device 704. Such information may include information from the generator module 734 (and corresponding energy devices), such as electrical information (e.g., current, voltage, impedance, frequency, wattage), activity state information, sensor information such as temperature, current settings, system events, active duration, and startup timestamps. The generator module 734 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to the medical professional (e.g., changes in the pitch, duration, and melody of an audible tone), electrical application profiles and / or application logic that may instruct the generator module to provide energy having a defined characteristic curve over the application time, operational updates (e.g., software / firmware), and the like.

[0154] For example, the smoke evacuator 735 may exchange surgical information with the surgical computing device 704. Such information may include information from the smoke evacuator 735 such as operational information (e.g., revolutions per minute), activity status information, sensor information such as temperature, current settings, system events, active duration, and boot timestamps. The smoke evacuator 735 may receive information from the surgical computing device 704 such as control information, configuration information, operational updates (software / firmware, etc.).

[0155] For example, the aspirate / irrigate module 736 may exchange surgical information with the surgical computing device 704. Such information may include information from the aspirate / irrigate module 736, such as operational information (e.g., liters per minute), activity status information, internal sensor information, current settings, system events, active duration, and startup timestamps. The aspirate / irrigate module 736 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (software / firmware, etc.).

[0156] For example, the communications module 739, the processor module 737, and / or the storage array 738 may exchange surgical information with the surgical computing device 704. In one example, the communications module 739, the processor module 737, and / or the storage array 738 may comprise all or part of the computing platform on which the surgical computing device 704 operates. In one example, the communications module 739, the processor module 737, and / or the storage array 738 may provide local computing resources to other devices in the surgical system 730. Information from the communications module 739, the processor module 737, and / or the storage array 738 to the surgical computing device 704 may include logical computing related reports such as processing load, processing power, process identification, CPU %, CPU time, threads, GPU %, GPU time, memory utilization, memory threads, memory ports, energy usage, bandwidth related information, packets in, packets out, data rates, channel utilization, buffer status, packet loss information, system events, and other status information. The communications module 739, processor module 737, and / or storage array 738 may receive information, such as control information, configuration information, operational updates (software / firmware, etc.), etc., from the surgical computing device 704. The communications module 739, processor module 737, and / or storage array 738 may also receive information from the surgical computing device 704 generated by another element or device of the surgical system 730. For example, data source information may be transmitted to and stored in the storage array. For example, the data source information may be processed by the processor module 737.

[0157] For example, the intelligent instrument 740 (with or without a corresponding display) may exchange surgical information with the surgical computing device 704. Such information may include information from the intelligent instrument 740 regarding the operation of the instrument, such as device electrical and / or mechanical information (e.g., current, voltage, impedance, frequency, wattage, torque, force, pressure, etc.), load status information (e.g., information regarding the identity, type, and / or status of reusables such as staple cartridges), clamping force, tissue compression pressure, and / or internal sensor information such as time, system events, active duration, and activation timestamp. The intelligent instrument 740 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to the medical professional (e.g., changes in the pitch, duration, and melody of an audible tone), mechanical application profiles and / or application logic that may instruct the instrument's mechanical components to operate with defined characteristics (e.g., blade / anvil advancement speed, mechanical advantage, firing time, etc.), operational updates (software / firmware, etc.), and the like.

[0158] For example, in a surgical stapling and severing instrument, control and configuration information can be used to modify operating parameters such as, for example, motor speed. Data collection of surgical information can be used to define the power, force, and / or other functional operations and / or behavior of the intelligent surgical stapling and severing instrument. See U.S. Patent No. 10,881,399(B2) (U.S. Patent Application No. 15 / 628,175), filed June 20, 2017, entitled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," the contents of which are incorporated herein by reference in their entirety.

[0159] For example, energy devices can use control and configuration information (e.g., control and configuration information based on the situational awareness of the surgical computing device 704) to adapt their functionality and / or behavior for improved results. See U.S. Patent Application Publication No. 2019-0201047(A1), entitled "METHOD FOR SMART ENERGY DEVICE INFRASTRUCTURE," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,458), the entire contents of which are incorporated herein by reference. Similarly, combo-energy devices (e.g., devices capable of using two or more energy modalities) can use such control and / or configuration information to select an appropriate operating mode. For example, the surgical computing device 704 can use surgical information, including information received from patient monitoring, to send control and / or configuration information to the combo-energy device. See U.S. Patent Application Publication No. 2017-0202605(A1), filed December 16, 2016, entitled "MODULAR BATTERY POWERED HANDHELD SURGICAL INSTRUMENT AND METHODS THEREFOR" (U.S. Patent Application No. 15 / 382,515), the entire contents of which are incorporated herein by reference.

[0160] For example, the sensor module 741 may exchange surgical information with the surgical computing device 704. Such information may include information from the sensor module 741 about its sensor capabilities, such as the sensor results themselves, observation frequency and / or resolution, observation type, device alerts such as alerts for sensor failure, observations exceeding a defined range, observations exceeding an observable range, etc. The sensor module 741 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of the observations (e.g., frequency, resolution, observation type, etc.), triggers defining specific events for observations, on controls, off controls, data buffering, data pre-processing algorithms, operational updates (software / firmware, etc.), etc.

[0161] For example, the visualization system 742 may exchange surgical information with the surgical computing device 704. Such information may include information from the visualization system 742, such visualization data itself (e.g., still images, video, advanced spectral visualization, etc.), visualization metadata (e.g., visualization type, resolution, frame rate, encoding, bandwidth, etc.), etc. The visualization system 742 may receive information from the surgical computing device 704, such as control information, configuration information, changes in video settings (e.g., visualization type, resolution, frame rate, encoding, etc.), visual display overlay data, data buffering sizes, data pre-processing algorithms, operational updates (software / firmware, etc.), etc.

[0162] Surgical information may be exchanged and / or used with advanced imaging systems. For example, surgical information may be exchanged and / or used to provide context for imaging data streams. For example, surgical information may be exchanged and / or used to expand conditional understanding of such imaging data streams. See U.S. Patent Application No. 17 / 493,904, entitled "SURGICAL METHODS USING MULTI-SOURCE IMAGING," filed October 5, 2021, the contents of which are incorporated herein by reference in their entirety. See U.S. Patent Application No. 17 / 493,913, entitled "SURGICAL METHODS USING FIDUCIAL IDENTIFICATION AND TRACKING," filed October 5, 2021, the contents of which are incorporated herein by reference in their entirety.

[0163] For example, the surgical robot 743 may exchange surgical information with the surgical computing device 704. In one example, the surgical information may include information related to collaborative registration and interaction of surgical robotic systems. See U.S. Patent Application No. 17 / 449,765, filed October 1, 2021, entitled "COOPERATIVE ACCESS HYBRID PROCEDURES," the entire contents of which are incorporated herein by reference. Information from the surgical robot 743 may include any of the aforementioned information as it applies to robotic instruments, sensors, and devices. Information from the surgical robot 743 may also include information related to the robotic operation or control of such instruments, such as electrical / mechanical feedback of the robotic articulator, system events, system settings, mechanical resolution, control operation logs, articulator path information, etc. The surgical robot 743 may receive information from the surgical computing device 704, such as control information, configuration information, operation updates (software / firmware, etc.), etc.

[0164] Surgical devices in communication with the surgical computing device 704 may exchange surgical information to support coordinated operations between the devices. For example, the surgical robot 743 and the energy generator 734 may exchange surgical information with each other and / or with the surgical computing device 704 for cooperative operation. Cooperative operation between the surgical robot 743 and the energy generator 734 acting cooperatively can be used to minimize undesirable side effects, such as tissue sticking. Cooperative operation between the surgical robot 743 and the energy generator 734 acting cooperatively can be used to improve tissue welding. See U.S. Patent Application Publication No. 2019-0059929(A1), entitled "METHODS, SYSTEMS, AND DEVICES FOR CONTROLLING ELECTROSURGICAL TOOLS," filed August 29, 2017 (U.S. Patent Application No. 15 / 689,072), the contents of which are incorporated herein by reference in their entirety. The surgical information may be generated by the collaborating devices and / or the surgical computing device 704 in connection with their collaborative operation.

[0165] The surgical computing system 704 can record, analyze, and / or act on surgical information flows, such as those disclosed above. The surgical computing system 704 can aggregate such data for analysis. For example, the surgical computing system 704 can perform operations such as defining device relationships, establishing device collaboration behaviors, monitoring procedure details, and / or storing procedure details. Surgical information associated with such operations may be further analyzed to refine algorithms, identify trends, and / or adapt surgical procedures. For example, the surgical information may be further analyzed compared to patient outcomes responsive to such operations. See U.S. Patent Application Publication No. 2019-0206562(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, DISPLAY, AND CLOUD ANALYTICS," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,416), the contents of which are incorporated herein by reference in their entirety.

[0166] 7C illustrates an exemplary information flow associated with multiple surgical computing systems 704a, 704b within a common environment. As the overall configuration of a computer-implemented surgical system (e.g., computer-implemented surgical system 750) changes (e.g., as data sources are added and / or removed from the surgical computing system), additional surgical information may be generated to reflect the changes. In this example, a second surgical computing system 704b (e.g., a surgical hub) may be added to surgical system 750 (along with a corresponding surgical robot) comprising existing surgical computing system 704a. The messaging flows described herein represent additional surgical information flow 755 (e.g., further integrated, analyzed, and / or processed according to algorithms such as machine learning algorithms) used as disclosed herein.

[0167] Here, two surgical computing systems 704a, 704b request permission from the surgeon for the second surgical computing system 704b (with a corresponding surgical robot 756) to take control of the operating room from the existing surgical computing system 704a. The second surgical computing system 704b presents control of the corresponding surgical robot 756, robotic visualization tower 758, Monohat tool 759, and robotic stapler 749 in the operating room. Permission may be requested through the surgeon interface or console 751. Once permission is granted, the second surgical computing system 704b sends a message to the existing surgical computing system 704a requesting transfer of control of the operating room.

[0168] In one example, the surgical computing systems 704a, 704b can negotiate the nature of their interaction without external input based on previously collected data. For example, the surgical computing systems 704a, 704b may collectively determine that an upcoming surgical task requires the use of a robotic system. Such a determination may cause the existing surgical computing system 704a to autonomously hand over control of the operating room to a second surgical computing system 704b. Upon completion of the surgical task, the second surgical computing system 704b may then autonomously return control of the operating room to the existing surgical computing system 704a.

[0169] As illustrated in Figure 7C, the existing surgical computing system 704a has transferred control to a second surgical computing system 704b, which also assumes control of the surgeon interface 751 and secondary display 752. The second surgical computing system 704b assigns new identification numbers to the newly transferred devices. The existing surgical computing system 704a retains control of the handheld stapler 753, handheld powered dissector 754, and visualization tower 757. In addition, the existing surgical computing system 704a may perform a support role, with the processing and storage capabilities of the existing surgical computing system 704a now available to the second surgical computing system 704b.

[0170] 7D illustrates an exemplary surgical information flow in the context of a surgical procedure and corresponding exemplary uses of the surgical information for predictive modeling. The surgical information disclosed herein may provide data regarding one or more surgical procedures, including surgical tasks, instruments, instrument settings, motion information, procedural variations, and corresponding desirable metrics such as improved patient outcomes, lower costs (e.g., fewer resources utilized, shorter surgical time, etc.). The surgical information disclosed herein (e.g., that disclosed with respect to FIGS. 7A-7C ), in the context of one or more surgical systems and devices disclosed herein, provides a platform upon which certain machine learning algorithms and techniques disclosed herein may be used.

[0171] Surgical information 762 from multiple surgical procedures 764 (e.g., a subset of surgical information from each procedure) may be collected. Surgical information 762 may be collected from multiple surgical procedures 764, for example, by collecting data represented by one or more information flows disclosed herein.

[0172] To illustrate, an exemplary instance of surgical information 766 may be generated from an exemplary procedure 768 (e.g., a lung segmentectomy procedure as shown on timeline 769). Surgical information 766 may be generated during preoperative planning and may include patient record information. Surgical information 766 may be generated from data sources (e.g., data sources 726) during the course of a surgical procedure, including data generated each time medical personnel utilize a modular device paired with the surgical computing system (e.g., surgical computing system 704). The surgical computing system may receive this data from the paired modular device and other data sources. The surgical computing system itself may generate surgical information as part of its operation during a procedure. For example, the surgical computing system may record information related to configuration and control operations. The surgical computing system may record information related to situational awareness activities. For example, the surgical computing system may record recommendations, prompts, and / or other information provided to the medical team (e.g., provided via a display screen) that may be related to the next procedure step. For example, the surgical computing system may record configuration and control changes (e.g., adjustments to modular devices based on context) that may include activating a monitor, adjusting the field of view (FOV) of a medical imaging device, changing the energy level of an ultrasonic surgical instrument or an RF electrosurgical instrument, etc.

[0173] Hospital personnel retrieve the patient's EMR from the hospital's EMR database at 770. Based on the patient data selected in the EMR, the surgical computing system determines that the procedure to be performed is a thoracic procedure.

[0174] At 771, personnel scan incoming medical supplies for a procedure. The surgical computing system may cross-reference the scanned supplies with a list of supplies utilized in various types of procedures. The surgical computing system may verify that the mix of supplies corresponds to a thoracic procedure. Additionally, the surgical computing system may determine that the procedure is not a wedge resection (because the incoming supplies either do not include certain supplies needed for a thoracic wedge resection or are otherwise not compatible with a thoracic wedge resection). The medical personnel may scan a patient band via a scanner communicatively connected to the surgical computing system. The surgical computing system may verify the patient's identity based on the scanned data.

[0175] At 774, medical personnel turn on auxiliary equipment. The auxiliary equipment utilized may vary according to the type of surgical procedure and the technology used by the surgeon. In this example, the auxiliary equipment may include a smoke evacuator, an aspirator, and a medical imaging device. Once activated, the auxiliary equipment may pair with the surgical computing system. The surgical computing system may derive contextual information regarding the surgical procedure based on the paired type. In this example, the surgical computing system determines that the surgical procedure is a VATS procedure based on this particular combination of paired devices. The contextual information regarding the surgical procedure may be ascertained by the surgical computing system via information from the patient's EMR.

[0176] The surgical computing system may retrieve the steps of the procedure to be performed. For example, the steps may be associated with a treatment plan (e.g., a treatment plan specific to this patient's surgery, a treatment plan associated with a particular surgeon, a treatment plan template for the procedure in general, etc.).

[0177] At 775, staff attach EKG electrodes and other patient monitoring devices to the patient. The EKG electrodes and other patient monitoring devices pair with the surgical computing system. The surgical computing system may receive data from the patient monitoring devices.

[0178] At 776, medical personnel induce anesthesia in the patient. The surgical computing system may record information related to this procedure step, such as data from the modular devices and / or patient monitoring devices, including, for example, EKG data, blood pressure data, ventilator data, or a combination thereof.

[0179] At 777, the lung of the surgical patient is collapsed (and ventilation may be switched to the contralateral lung). The surgical computing system may determine that this procedure step has begun and may collect surgical information accordingly, including, for example, ventilator data, one or more timestamps, etc.

[0180] At 778, a medical imaging device (e.g., a scope) is inserted and video from the medical imaging device is initiated. The surgical computing system may receive medical imaging device data (i.e., video or image data) through a connection to the medical imaging device. The data from the medical imaging device may include imaging data and / or imaging metadata, such as the angle at which the medical imaging device is oriented with respect to visualization of the patient's anatomy, the number or currently active medical imaging devices, etc. The surgical computing system may record positioning information of the medical imaging device. For example, one technique for performing a VATS lobectomy is to position the camera in the anterior-inferior corner of the patient's chest cavity above the diaphragm. Another technique for performing a VATS segmentectomy is to position the camera in an anterior intercostal position relative to the segmental fissure.

[0181] The surgical computing system can be trained to recognize the positioning of the medical imaging device according to the visualization of the patient's anatomy, for example, using pattern recognition or machine learning techniques. For example, one technique for performing a VATS lobectomy utilizes a single medical imaging device. Another technique for performing a VATS segmentectomy uses multiple cameras. Yet another technique for performing a VATS segmentectomy uses an infrared light source (which may be communicatively coupled to the surgical computing system as part of the visualization system).

[0182] At 779, the surgical team begins the incision step of the procedure. The surgical computing system may collect data from the RF or ultrasonic generator indicating that the energy instrument is being fired. The surgical computing system may cross-reference the received data with the retrieved steps of the surgical procedure to determine that the energy instrument being fired at this point in the process (i.e., after previously discussed steps of the procedure have been completed) corresponds to the incision step. In one example, the energy instrument may be an energy tool mounted on a robotic arm of a robotic surgical system.

[0183] At 780, the surgical team proceeds to the ligation step of the procedure. The surgical computing system may collect surgical information 766 related to the surgeon ligating the arteries and veins based on receiving data from the surgical stapling and severing instrument indicating that such instrument is being fired. Next, the segmentectomy portion of the procedure is performed. The surgical computing system may collect information related to the surgeon transecting the parenchyma. For example, the surgical computing system may receive surgical information 766 from the surgical stapling and severing instrument, including data related to its cartridge, settings, firing details, etc.

[0184] At 782, the node dissection step is then performed. The surgical computing system may collect surgical information 766 regarding the surgical team dissecting the node and performing the leak test. For example, the surgical computing system may collect data received from the generator indicating that an RF or ultrasonic instrument is being fired, including electrical and status information associated with the firing. The surgeon periodically alternates between the surgical stapling / cutting instrument and the surgical energy (i.e., RF or ultrasonic) instrument depending on the particular step in the procedure. The surgical computing system may collect surgical information 766 taking into account the particular order in which the stapling / cutting instrument and the surgical energy instrument are used. In one example, a robotic tool may be used for one or more steps in the surgical procedure. The surgeon may, for example, alternate between using the robotic tool and a handheld surgical instrument and / or use the devices simultaneously.

[0185] The incisions are then closed and the post-operative portion of the procedure begins. The patient is de-anesthetized at 784. The surgical computing system can gather surgical information regarding the patient's emergence from anesthesia, for example, based on ventilator data (e.g., the patient's breathing rate begins to increase).

[0186] At 785, medical personnel remove various patient monitoring devices from the patient. The surgical computing system may collect information regarding the outcome of the procedure. For example, the surgical computing system may collect information related to the loss of EKG, BP, and other data from the patient monitoring devices.

[0187] The surgical information 762 (including the surgical information 766) may be structured and / or labeled. The surgical computing system may inherently provide such structure and / or labeling in the data collection. For example, the surgical information 762 may be labeled according to particular characteristics, desired results (e.g., efficiency, patient outcome, cost, and / or combinations thereof, etc.), particular surgical techniques, aspects of instrumentation (e.g., surgical instrument selection, timing, and activation, instrument settings, nature of instrument use, etc.), identities of medical professionals involved, particular patient characteristics, etc., each of which may be present in the data collection.

[0188] The surgical information (e.g., surgical information 762 collected over the procedure 764) may be used in connection with one or more artificial intelligence (AI) systems. AI may be used to perform computer cognitive tasks. For example, AI may be used to perform complex tasks based on observation of data. AI may be used to enable computing systems to perform cognitive tasks and solve complex tasks. AI may include using machine learning and machine learning techniques. ML techniques may include, for example, performing complex tasks without being programmed (e.g., explicitly programmed). For example, ML techniques may improve over time based on completing tasks with different inputs. An ML process may train itself, for example, using input data and / or a training dataset.

[0189] Machine learning (ML) techniques may be used, for example, in the medical field. For example, ML may be used on a set of data (e.g., a set of surgical data) to generate output (e.g., reduced surgical data, processed surgical data). In an example, the output of the ML process may include identified trends or relationships in the data input for processing. The output may include verifying results and / or outcomes associated with the input data. In an example, the input to the ML process may include medical data such as surgical images and patient scans. The ML process may output a determined medical condition based on the input surgical images and patient scans. The ML process may be used to diagnose a medical condition, for example, based on the surgical scans.

[0190] An ML process may improve itself, for example, using historical data and / or input data that trained the ML process. Thus, an ML process may continually improve with added inputs and processing. The ML process may update based on the input data. For example, over time, an ML process that generates medical outcomes based on medical data may improve and become more accurate and consistent in medical diagnoses.

[0191] ML processes may be used to solve different complex tasks (e.g., medical tasks). For example, ML processes may be used for data reduction, data preparation, data processing, trend identification, outcome determination, medical diagnosis, and / or the like. For example, an ML process may take surgical data as input and process the data for use in medical analysis. The processed data may be used to determine a medical diagnosis. Finally, an ML process may take raw surgical data and generate useful medical information (e.g., medical trends and / or diagnoses) associated with the raw surgical data.

[0192] ML processes may be combined to perform different discrete tasks on an input data set. For example, an ML process may include testing different combinations of ML subprocesses performing discrete tasks to determine which combination performs best (e.g., competitive use of different process / algorithm types and training to determine the best combination for a data set). For example, an ML process may include subprocess (e.g., algorithm) control and monitoring to refine and / or validate results and / or outcomes (e.g., error bounds).

[0193] An ML process may be initialized and / or set up to perform a task. For example, the ML process may be initialized based on initialization configuration information. The initialized ML process may be an untrained ML process and / or a base ML process for performing the task. An untrained ML process may be inaccurate in performing a specified task. As the ML process is trained, the task may be performed more accurately.

[0194] The initialization configuration information for the ML process may include initial settings and / or parameters. For example, the initial settings and / or parameters may include defined ranges for use by the ML process. The ranges may include manually entered and / or received data ranges. The ranges may include default ranges and / or randomized ranges for unreceived variables that may be used, for example, to complete the dataset for processing. For example, if a dataset lacks a data range, the default data range may be used as a substitute to run the ML process.

[0195] The initialization configuration information for the ML process may include data storage locations. For example, locations or data storage and / or databases associated with data interactions may be included. The databases associated with data interactions may be used to identify trends in the dataset. The databases associated with data interactions may include mappings of data to medical conditions. For example, the databases associated with data interactions may include mappings of heart rate data to arrhythmias, etc.

[0196] The initialization configuration information may include parameters associated with defining the system. The initialization configuration information may include instructions (e.g., methods) associated with displaying, verifying, and / or providing information to a user. For example, the initialization configuration may include instructions for an ML process to output data in a particular format for visualization to a user.

[0197] ML techniques may be used, for example, to perform data reduction. ML techniques for data reduction may include using multiple different data reduction techniques. For example, ML techniques for data reduction may include using one or more of the following: CUR matrix decomposition; decision trees; expectation-maximization (EM) processes (e.g., algorithms); explicit semantic analysis (ESA); exponential smoothing forecasting; generalized linear models; k-means clustering (e.g., nearest neighbor); naive Bayes; neural network processes; multivariate analysis; o-cluster; singular value decomposition; Q-learning; temporal difference (TD); deep adversarial networks; support vector machines (SVM); linear regression; dimensionality reduction; linear discriminant analysis (LDA); adaptive boosting (e.g., AdaBoost); gradient descent (e.g., stochastic gradient descent (SGD)); outlier detection; and / or others.

[0198] ML techniques may be used to perform data reduction, for example, using CUR matrix decomposition. CUR matrix decomposition may include using a matrix decomposition model (e.g., process, algorithm), such as a low-rank matrix decomposition model. For example, CUR matrix decomposition may include a low-rank matrix decomposition process that is expressed (e.g., explicitly expressed) in several (e.g., a small number) columns and / or rows of a data matrix (e.g., the CUR matrix decomposition may be interpretable). CUR matrix decomposition may include selecting columns and / or rows associated with statistical leverage and / or large influence in the data matrix. Using CUR matrix decomposition may enable identifying attributes and / or rows within the data matrix. Simplification of larger datasets (e.g., using CUR matrix decomposition) may enable users to review and interact (e.g., with the data). CUR matrix decomposition may facilitate regression, classification, clustering, and / or other processes.

[0199] ML techniques may be used to perform data reduction using, for example, decision trees (e.g., decision tree models). Decision trees may be used, for example, as a framework for quantifying outcome values ​​and / or the probability of an outcome occurring. Decision trees may be used, for example, to calculate values ​​for uncertain outcome nodes (e.g., in a decision tree). Decision trees may be used, for example, to calculate values ​​for decision nodes (e.g., in a decision tree). Decision trees may be models that enable classification and / or regression (e.g., applicable to classification and / or regression problems). Decision trees may be used to analyze numerical (e.g., continuous) and / or categorical data. Decision trees may be more successful and / or more efficient with large datasets (e.g., compared to other data reduction techniques).

[0200] Decision trees may be used in combination with other decision trees. For example, a random forest may refer to a collection of decision trees (e.g., an ensemble of decision trees). A random forest may include a collection of decision trees whose results may be aggregated into a result. A random forest may be a supervised learning algorithm. A random forest may be trained, for example, using a bagging training process.

[0201] A random decision forest (e.g., random forest) may add randomness (e.g., additional randomness) to a model, for example, while growing a tree. A random forest may be used, for example, to search for the best feature among a random subset of features, rather than searching for the most important feature (e.g., while splitting a node). Searching for the best feature among a random subset of features may result in a wide variety, which may result in a better (e.g., more efficient and / or accurate) model.

[0202] Random forests may include using parallel ensembles. Parallel ensembles may include, for example, fitting (e.g., several) decision tree classifiers in parallel on different dataset subsamples. Parallel ensembles may include using majority voting or averaging on the results or final outcome. Parallel ensembles may be used to minimize overfitting and / or increase prediction accuracy and control. Random forests with multiple decision trees may (e.g., generally) be more accurate than single decision tree-based models. A set of decision trees with controlled variation may be constructed, for example, by combining bootstrap aggregation (e.g., bagging) with random feature selection.

[0203] ML techniques may be used to perform data reduction, for example, using an expectation-maximization (EM) model (e.g., process, algorithm). For example, an EM model may be used to find likelihood (e.g., local maximum likelihood) parameters of a statistical model. An EM model may be used when equations cannot be solved directly. An EM model may consider latent variables and / or unknown parameters and known data observations. For example, an EM model may determine that missing values ​​are present in a dataset. An EM model receives configuration information indicating to assume the presence of missing (e.g., unobserved) data points in the dataset.

[0204] The EM model may use component clustering. For example, component clustering may allow EM components to be grouped into high-level clusters. For example, if component clustering is disabled (e.g., in the EM model), the components may be treated as clustered.

[0205] ML techniques may be used to perform data reduction, for example, using Explicit Semantic Analysis (ESA). ESA may be used at the level of semantics (e.g., meaning) rather than the vocabulary (e.g., surface form vocabulary) of words or documents. ESA may focus on the meaning of a set of text, for example, as a combination of concepts found within the text. ESA may be used for document classification. ESA may be used for semantic relevance computation (e.g., how similar words or fragments of text are to each other). ESA may be used for information retrieval.

[0206] ESAs may be used, for example, for document classification. Document classification may include tagging documents for management and sorting. Tagging documents (e.g., with keywords) may make them easier to search. Keyword tagging (e.g., using only keyword tagging) may limit the accuracy and / or efficiency of document classification. For example, using keyword tagging may reveal (e.g., only) documents that have the keyword, but not documents that have words with similar meanings to the keyword. Semantically classifying text (e.g., using ESAs) may improve a model's understanding of the text. Semantically classifying text may include representing documents as concepts and reducing reliance on specific keywords.

[0207] ML techniques may be used to perform data reduction, for example, using an exponential smoothing forecasting model. Exponential smoothing may be used to smooth time series data, for example, using an exponential window function. For example, in a moving average, past observations may be weighted equally, but using an exponential function, weights may be assigned that decrease exponentially over time.

[0208] ML techniques may be used to perform data reduction, for example, using linear regression. Linear regression may be used to predict continuous outcomes. For example, linear regression may be used to predict the value of a variable (e.g., a dependent variable) based on the values ​​of different variables (e.g., independent variables). Linear regression may apply a linear approach to model the relationship between a scalar response and one or more explanatory variables (e.g., a dependent variable and / or independent variables). Simple linear regression may refer to a linear regression use case associated with one explanatory variable. Multiple linear regression may refer to a linear regression use case associated with two or more explanatory variables. Linear regression may model the relationship, for example, using a linear prediction function. The linear prediction function may estimate unknown model parameters from a dataset.

[0209] For example, linear regression may be used to identify patterns within a training dataset. The identified patterns may relate to groupings of values ​​and / or labels. The model may learn the relationship between (e.g., each) label and expected outcomes. After training, the model may be used on raw data outside the training dataset (e.g., data that does not have a mapped and / or known output). A trained model using linear regression may determine a calculated prediction associated with the raw data, such as identifying seasonal changes in sales data.

[0210] ML techniques may be used to perform data reduction, e.g., generalized linear models (GLMs). GLMs may be used as flexible generalizations of linear regression. GLMs may generalize linear regression, for example, by allowing linear models to relate response variables.

[0211] ML techniques may be used to perform data reduction, for example, using k-means clustering (e.g., nearest neighbor models). K-means clustering may be used in vector quantization. K-means clustering may be used in signal processing. K-means clustering may, for example, aim to divide n observations into k clusters, with each observation falling into the cluster with the closest mean.

[0212] K-means clustering may include K-Nearest Neighbor (KNN) learning. KNN may be instance-based learning (e.g., non-generalized learning, lazy learning). KNN may refrain from building a general internal model. KNN may include storing instances corresponding to training data in an n-dimensional space. KNN may use the data to classify data points, for example, based on a similarity measure (e.g., a Euclidean distance function). Classification may be calculated, for example, based on a majority vote of the k neighbors of a point (e.g., each point). KNN may be robust to noisy training data. Accuracy may depend on data quality (e.g., for KNN). KNN may include selecting the number of neighbors to consider (e.g., an optimal number of neighbors to consider). KNN may be used for classification and / or regression.

[0213] ML techniques may be used to perform data reduction, for example, using a naive Bayes model (e.g., process). For example, a naive Bayes model may be used to build a classifier. Using the naive Bayes model, a class label may be assigned to a problem instance (e.g., represented as a vector of feature values). The class label may be drawn from a set (e.g., a finite set). Different processes (e.g., algorithms) may be used to train the classifier. A family of processes (e.g., a family of algorithms) may be used. A family of processes may be based on the principle that a naive Bayes classifier (e.g., all naive Bayes) classifier assumes that feature values ​​are independent of different feature values ​​(e.g., given a class variable).

[0214] ML techniques may be used to perform data reduction, for example, using neural networks. The neural network may learn (e.g., be trained) by processing examples, for example, to perform other tasks (e.g., similar tasks). The processed examples may include inputs and results (e.g., inputs mapped to results). The neural network may learn by forming probability-weighted associations between inputs and results. The probability-weighted associations may be stored within the neural network's data structure. Training the neural network from a given example may be performed by determining the difference between the network's processed output (e.g., prediction) and a target output. This difference may be an error. The neural network may adjust the weighted associations (e.g., stored weighted associations), for example, according to a learning rule and an error value.

[0215] ML techniques may be used to perform data reduction, for example, using multivariate analysis, which may include performing multivariate state estimation and / or non-negative matrix factorization.

[0216] ML techniques may be used to perform data reduction using, for example, a support vector machine (SVM). SVMs may be used in multidimensional spaces (e.g., high-dimensional spaces, infinite-dimensional spaces). SVMs may be used to construct hyperplanes (e.g., a set of hyperplanes). A hyperplane with the greatest distance (e.g., compared to other constructed hyperplanes) from the nearest training data point within a class (e.g., any class) may achieve strong separation (e.g., generally, the larger the margin, the lower the generalization error of the classifier). SVMs may be effective in high-dimensional spaces. SVMs may behave differently, for example, based on different mathematical functions (e.g., kernels, kernel functions). For example, kernel functions may include one or more of linear, polynomial, radial basis function (RBF), sigmoid, etc. Kernel functions may be used as SVM classifiers. SVMs may be limited, for example, in use cases where the dataset contains a large amount of noise (e.g., overlapping target classes).

[0217] ML techniques may be used to perform data reduction, such as, for example, dimensionality reduction. Reducing the dimensionality of a sample of data (e.g., unlabeled data) may help refine groups and / or clusters. Reducing the number of variables in a model may simplify the data's trends. Simplified data trends may allow for more efficient processing. Dimensionality reduction may be used, for example, when many (e.g., too many) dimensions obscure (e.g., adversely affect) insights, trends, patterns, outcomes, and / or the like.

[0218] Reducing dimensionality may include using principal component analysis (PCA). PCA may be used to establish principal components that govern the relationships between data points. PCA may focus on simplifying (e.g., simplifying only) the principal components. Dimensionality reduction (e.g., PCA) may be used to maintain the diversity of data groupings within a dataset, but rationalize the number of distinct groups.

[0219] ML techniques may be used to perform data reduction, e.g., linear discriminant analysis (LDA). LDA may refer to a linear decision boundary classifier, which may be created, for example, by fitting class conditional densities to data (e.g., and applying Bayes' rule). LDA may include a generalization of Fisher's Linear Discriminant (e.g., projecting a given dataset into a lower-dimensional space to reduce dimensionality, minimize model complexity, and reduce computational cost). An LDA model (e.g., a standard LDA model) may fit classes with Gaussian densities. An LDA model may assume that classes (e.g., all classes) share a covariance matrix. LDA may be similar to an analysis of variance (ANOVA) process and / or regression analysis. For example, LDA may be used to express a dependent variable as a linear combination of other features and / or measurements.

[0220] ML techniques may be used to perform data reduction, such as, for example, adaptive boosting (e.g., AdaBoost). Adaptive boosting may include creating a classifier (e.g., a powerful classifier). Adaptive boosting may include creating a classifier by combining multiple classifiers (e.g., poorly performing classifiers), for example, to obtain a resulting classifier with high accuracy. AdaBoost may be an adaptive classifier that improves the efficiency of a classifier. AdaBoost may trigger overfitting. AdaBoost may be used (e.g., most commonly used) to improve the performance of decision trees, base estimators, binary classification problems, and / or the like. AdaBoost may be sensitive to noisy data and / or outliers.

[0221] ML techniques may be used to perform data reduction, such as stochastic gradient descent (SGD). SGD may include an iterative process used to optimize a function (e.g., an objective function). SGD may be used, for example, to optimize an objective function with specific smoothness properties. Stochastic may refer to random probability. SGD may be used, for example, to reduce computational load in high-dimensional optimization problems. SGD may be used, for example, to enable faster iterations while trading off a slower convergence rate. Gradient may refer, for example, to the slope of a function that calculates the degree of change of a variable in response to a change in another variable. Gradient descent may refer to a convex function that outputs the partial derivative of its set of input parameters. For example, α may be the learning rate, and J may be the cost of the training example for the i-th iteration. The following equation may represent the stochastic gradient descent weight update method at the j-th iteration: In large-scale ML and sparse ML, SGD may be applied to problems in text classification and / or natural language processing (NLP). SGD can be sensitive to feature scaling (e.g., it may be necessary to use a range of hyperparameters, such as the regularization parameter and the number of iterations).

[0222] ML techniques may be used to perform data reduction, such as using outlier detection. An outlier may be a data point that contains information (e.g., useful information) about abnormal behavior of the system described by the data. Outlier detection processes may include univariate and multivariate processes.

[0223] The ML process may be trained, for example, using one or more training methods. For example, the ML process may be trained using one or more of the following training techniques: supervised learning; unsupervised learning; semi-supervised learning; reinforcement learning; and / or others.

[0224] Machine learning can be supervised (e.g., supervised learning). Supervised learning algorithms can create a mathematical model from training data sets (e.g., training data). FIG. 8A illustrates an exemplary supervised learning framework 800. Training data (e.g., training examples 802, e.g., as shown in FIG. 8) may consist of a series of training examples (e.g., input data mapped to labeled outputs, e.g., as shown in FIG. 8A). Training examples 802 may include one or more inputs and one or more labeled outputs. The labeled outputs may serve as supervisory feedback. In the mathematical model, training examples 802 may be represented by an array or vector, sometimes called a feature vector. The training data may be represented by rows of the feature vector that form a matrix. Through iterative optimization of an objective function (e.g., a cost function), supervised learning algorithms can learn a function (e.g., a prediction function) that can be used to predict outputs associated with one or more new inputs. A properly trained predictive function (e.g., a trained ML model 808) can determine outputs 804 (e.g., labeled outputs) for one or more inputs 806 that may not be part of the training data (e.g., input data without mapped labeled outputs, as shown in FIG. 8A). Exemplary algorithms may include linear regression, logistic regression, neural networks, nearest neighbors, naive Bayes, decision trees, SVMs, and / or others. Exemplary problems that can be solved by supervised learning algorithms may include classification, regression problems, etc.

[0225] Machine learning can be unsupervised (e.g., unsupervised learning). FIG. 8B illustrates an exemplary unsupervised learning framework 810. An unsupervised learning algorithm 814 may train on a dataset that may include input 811 and may find structure 812 in the data (e.g., pattern detection and / or descriptive modeling). The structure 812 in the data may resemble groupings or clusterings of data points. Thus, the algorithm 814 may learn from training data that may be unlabeled. Instead of responding to supervised feedback, the unsupervised learning algorithm may identify commonalities in the training data and may react based on the presence or absence of such commonalities in each training data. For example, training may include operating on training input data to generate a model and / or output with a particular energy (e.g., cost function, etc.), and such energy may be used to further refine the model (e.g., to define a model that minimizes the cost function given the training input data). Exemplary algorithms may include the Apriori algorithm, K-means, K-nearest neighbors (KNN), K-medians, etc. Representative problems that can be solved by unsupervised learning algorithms may include clustering problems, anomaly / outlier detection problems, etc.

[0226] Machine learning may be semi-supervised (e.g., semi-supervised learning). Semi-supervised learning algorithms may be used in scenarios where labeling data is costly (e.g., because a skilled expert is required to label the data) and labels for the data are limited. Semi-supervised learning models may take advantage of the idea that while the group membership of unlabeled data is unknown, the data still holds important information about group parameters.

[0227] Machine learning may include reinforcement learning, which may be an area of ​​machine learning that may concern how a software agent can take actions in an environment to maximize some notion of cumulative reward. Reinforcement learning algorithms may not assume knowledge of an exact mathematical model of the environment (e.g., represented by a Markov decision process (MDP)) and may be used when an exact model may not be feasible. Reinforcement learning algorithms may be used in autonomous vehicles or in learning to play games against human opponents. Exemplary algorithms may include Q-learning, temporal difference (TD), deep adversarial networks, and / or others.

[0228] Reinforcement learning may involve an algorithm (e.g., an agent) that continuously learns from an environment in an iterative manner. During the training process, the agent may learn from experience with the environment until the agent has explored the full range of states (e.g., possible states). Reinforcement learning may be defined by the type of problem. Reinforcement learning solutions may be classified as reinforcement learning algorithms. In a problem, an agent may determine an action to select (e.g., a best action) based on the agent's current state. When steps are repeated, the problem may be referred to as an MDP.

[0229] For example, reinforcement learning may include an action step. The action step in reinforcement learning may include an agent observing an input state. The action step in reinforcement learning may include causing an agent to perform an action using a decision-making function. The action step may include an agent receiving a reward and / or reinforcement from the environment (e.g., after an action is performed). The action step in reinforcement learning may include storing state-action pair information related to the reward.

[0230] Machine learning may be part of a technology platform called cognitive computing (CC), which may comprise various fields such as computer science and cognitive science. CC systems may be able to learn at scale, reason purposefully, and interact naturally with humans. Self-teaching algorithms, which may use data mining, visual recognition, and / or natural language processing, may enable CC systems to solve problems and optimize human processes.

[0231] The output of a machine learning training process may be a model for predicting outcomes for new data sets. For example, a linear regression learning algorithm may have a cost function that can minimize the prediction error of a linear prediction function during the training process by adjusting the coefficients and constants of the linear prediction function. If a minimum value can be reached, the linear prediction function with the adjusted coefficients may be considered trained and constitute the model produced by the training process. For example, a neural network (NN) algorithm for classification (e.g., a multilayer perceptron (MLP)) may include a hypothesis function represented by a network of layers of nodes that are assigned biases and interconnected with weighted connections. The hypothesis function may also be a nonlinear function (e.g., a highly nonlinear function) that may include linear and logistic functions nested together, with an outermost layer consisting of one or more logistic functions. The NN algorithm may include a cost function for minimizing classification error by adjusting biases and weights through a process of feedforward propagation and backpropagation. If a global minimum can be reached, the optimized hypothesis function with its adjusted bias and weight layers may be considered trained and constitute the model that the training process produced.

[0232] Data aggregation may be performed for machine learning as a first stage in a machine learning lifecycle. Data aggregation may include steps such as identifying various data sources, collecting data from the data sources, and integrating the data. For example, to train a machine learning model for predicting surgical complications and / or post-surgical recovery rates, data sources including pre-surgical data such as a patient's medical condition and biomarker measurement data may be identified. Such data sources may be a patient's electronic medical record (EMR), a computing system that stores the patient's pre-surgical biomarker measurement data, and / or other similar data stores. Data from such data sources may be retrieved and stored in a central location for further processing in the machine learning lifecycle. Data from such data sources may be linked (e.g., logically linked) and accessed as if they were centrally stored. Surgical data and / or post-surgical data may be similarly identified and collected. Furthermore, the collected data may be integrated. In examples, a patient's pre-surgical medical record data, pre-surgical biomarker measurement data, pre-surgical data, surgical data, and / or post-surgical data may be combined into a patient record, which may be an EMR.

[0233] Data preparation can be performed for machine learning as another stage of the machine learning lifecycle. Data preparation can include data preprocessing steps such as data formatting, data cleaning, and data sampling. For example, collected data may not be in a data format suitable for training a model. Such data records can be converted into a flat file format for model training. Such data can be mapped to numerical values ​​for model training. Such identifying data can be removed before model training. For example, identifying data can be removed for privacy reasons. As another example, data can be removed because there may be more available data than can be used for model training. In such cases, a subset of the available data can be randomly sampled and selected for model training, and the remainder can be discarded.

[0234] Data preparation may include data transformation operations (e.g., after preprocessing), such as scaling and aggregation. For example, the preprocessed data may include data values ​​at various scales. These values ​​may be scaled up or down, e.g., to be between 0 and 1, for model training. For example, the preprocessed data may include data values ​​that become more meaningful when aggregated.

[0235] Model training may be another aspect of the machine learning life cycle. The model training process described herein may depend on the machine learning algorithm used. A model may be considered suitably trained after it has been trained, cross-validated, and tested. Thus, a dataset from the data preparation stage (e.g., an input dataset) may be divided into a training dataset (e.g., 60% of the input dataset), a validation dataset (e.g., 20% of the input dataset), and a test dataset (e.g., 20% of the input dataset). After a model is trained on the training dataset, it may be run on the validation dataset to reduce overfitting. If the model's accuracy is increasing, but decreases when run on the validation dataset, this may indicate an overfitting problem. The test dataset may be used to test the accuracy of the final model to determine whether it is ready for deployment or whether more training may be required.

[0236] Model deployment can be another aspect of the machine learning lifecycle. Models may be deployed as part of a standalone computer program. Models may be deployed as part of a larger computing system. Models may be deployed with model performance parameters. Such performance parameters may monitor model accuracy as it is used to make predictions on a running dataset. For example, such parameters may track false positives and false positives of a classification model. Such parameters may further store false positives and false positives for further processing to improve the accuracy of the model.

[0237] Model updates after deployment may be another aspect of the machine learning cycle. For example, a deployed model may be updated as false positives and / or as false positives are predicted on the production data. In one example, for an MLP model deployed for classification, when a false positive occurs, the deployed MLP model may be updated to increase the probability cutoff for predicting a positive to reduce the false positives. In one example, for an MLP model deployed for classification, when a false negative occurs, the deployed MLP model may be updated to decrease the probability cutoff for predicting a positive to reduce the false negatives. In one example, for an MLP model deployed for classification of surgical complications, when both false positives and false negatives occur, the deployed MLP model may be updated to decrease the probability cutoff for predicting a positive to reduce the false negatives, as predicting a false positive may be less serious than a false negative.

[0238] For example, the deployed model may be updated as more live production data becomes available as training data. In such cases, the deployed model may be further trained, validated, and tested using such additional live production data. In one example, the updated biases and weights of the further trained MLP model may update the biases and weights of the deployed MLP model. Those skilled in the art will recognize that post-deployment model updates may not be a one-time occurrence, but may occur as frequently as is suitable to improve the accuracy of the deployed model.

[0239] ML techniques may be used independently of each other or in combination. Different problems and / or datasets may benefit from using different ML techniques (e.g., combinations of ML techniques). Different training types for models may be more suitable for particular problems and / or datasets. The optimal algorithm (e.g., combinations of ML techniques) and / or training type may be determined for a particular use, problem, and / or dataset. For example, processes may be performed to select one or more of the following: select a data reduction type, select a model and / or algorithm configuration, determine the location of data reduction, determine the efficiency of the reduction and / or results, and / or other.

[0240] For example, an ML technique and / or combination of ML techniques may be determined for a particular problem and / or use case. Multiple data reduction and / or data analysis processes may be performed to determine accuracy, efficiency, and / or compatibility associated with a dataset. For example, a first ML technique (e.g., a first set of combined ML techniques) may be used on a dataset to perform data reduction and / or data analysis. The first ML technique may generate a first output. Similarly, a second ML technique (e.g., a second set of combined ML techniques) may be used on a dataset (e.g., the same dataset) to perform data reduction and / or data analysis. The second ML technique may generate a second output. The first output may be compared to the second output to determine which ML technique produced a more desirable result (e.g., a more efficient result, a more accurate result). Multiple ML techniques may be compared on the same dataset to determine the optimal ML technique to use on future similar datasets and / or problems.

[0241] In an example, in a medical context, a surgeon or medical professional may provide feedback to the ML technique and / or model used on the dataset. The surgeon may input the feedback into the weighted results of the ML model. The feedback may be used as input by the model to determine reduction methods for future analysis.

[0242] In examples, a data analysis method (e.g., an ML technique to be used in the data analysis method) may be determined based on the dataset itself. For example, the origin of the data may influence the type of data analysis method to be used for the dataset. Available system resources may be used to determine the data analysis method to be used for a given dataset. The magnitude of the data may be considered, for example, in determining the data analysis method. For example, the need for an external dataset for local processing levels or magnitude of operational response may be considered (e.g., small device changes may be made using local data, while large device operational changes may require global compilation and validation).

[0243] Such ML techniques may be applied to surgical information (eg, combining the information flows of the surgical information in Figures 7A-7D) to generate useful ML models.

[0244] A computing device, such as a surgical hub, can use the data to train an ML model and detect changes in the device and / or healthcare professionals (HCPs). The computing device can detect whether the device and / or surgeon are performing differently in typical operations using data from the operations. For example, the computing device may provide data for training the ML model. The trained ML model may be or include collected performance data associated with the device and / or HCP. The computing device can compare how the device and / or HCP is performing with other data, such as other trained ML models associated with normal performance data for the device and / or other HCPs. The computing device can determine that current performance associated with the device and / or HCP differs from the generated ML performance data. The generated ML performance data can include and / or be configured to indicate aggregated typical operation data generated by an ML process and / or ML algorithm (e.g., ML models associated with normal performance data for the device and / or other HCPs). The computing device can determine whether the device and / or HCP has improved or degraded performance based on the comparison.

[0245] In an example, the computing device can compare performance data to detect and / or locate groups of devices and / or HCPs that have performed differently from the aggregated typical operational ML collected data. The computing device can itemize the detected / localized groups of devices and / or HCPs that have performed differently. The computing device can use the itemized performance data to identify trends, for example, using and / or configuring ML algorithms. Examples of ML algorithms and / or trends and / or patterns that ML identifies are described in U.S. Patent No. 11,410,259, entitled "Adaptive Control Program Updates For Surgical Devices," issued on August 9, 2022; U.S. Patent No. 11,423,007, entitled "Adjustment Of Device Control Programs Based On Stratified Contextual Data In Addition To The Data," issued on August 23, 2022; U.S. Patent No. 10,881,399, entitled "Techniques For Adaptive Control Of Motor Velocity Of A Surgical Stapling And Cutting Instrument," issued on January 5, 2021; U.S. Patent No. 10,695,081, entitled "Controlling A Surgical Instrument According To Sensed Closure Parameters," issued on June 30, 2020; or U.S. Patent No. 10,695,081, entitled "Data Pairing To Interconnect A Device Measured Parameter With An U.S. Patent Application No. 15 / 940, entitled "Bio-Rad Outcome,"649, all of which are incorporated herein by reference in their entireties. Additionally and / or alternatively, examples of ML algorithms and / or trends and / or patterns that ML identifies are described in U.S. patent application Ser. No. 16 / 209,423, entitled "Method Of Compressing Tissue Within A Stapling Device And Simultaneously Displaying the Location of the Tissue Within The Jaws," filed December 4, 2018; U.S. Patent No. 10,881,399, entitled "Techniques For Adaptive Control Of Motor Velocity Of A Surgical Stapling And Cutting Instrument," filed January 5, 2021; U.S. Patent Application Ser. No. 16 / 458,103, entitled "Packaging For A Replaceable Component Of A Surgical Stapling System," filed June 30, 2019; U.S. Patent Application Ser. No. 16 / 458,103, entitled "Control Of Advancement Rate And Application Force Based On Measured Speed," filed August 27, 2019; No. 10,390,895 entitled "Forces," U.S. Patent No. 10,932,808 entitled "Methods, Systems, And Devices For Controlling Electrosurgical Tools," filed March 2, 2021; U.S. Patent Application No. 16 / 209,458 entitled "Method For Smart Energy Device Infrastructure," filed December 4, 2018; U.S. Patent No. 10,842,523 entitled "Modular Battery Powered Handheld Surgical Instrument And Methods Therefor," issued November 24, 2020; and U.S. Patent No. 9,919,923 entitled "Articulatable Surgical Instrument Comprising A Firing Drive," issued June 27, 2017.Nos. 687,230, 687,230, the entireties of which are incorporated herein by reference.

[0246] In an example, a computing device may detect a device in an operating room (e.g., for a surgical procedure). The computing device may receive identifying information from the device. For example, the device may transmit identifying information to the computing device, and the computing device may use the identifying information received from the device to assign an ID to the device. Based on the identified device, the computing device may use ML algorithms and / or aggregated ML performance data to determine whether the device is performing differently from an aggregated typical operational performance.

[0247] In an example, a computing device may detect a device in an operating room. The device may be or include a surgical device used for a surgical procedure in the operating room. The device may not transmit identifying information. For a non-self-ID device, the computing device may monitor the device's performance to determine performance data over time. The computing device can input (e.g., provide) monitored performance data (e.g., surgical information as disclosed with respect to FIGS. 7A-7D ) of the non-self-ID device to identify the device. For example, the computing device can configure an ML process and / or use an ML algorithm to compare the monitored performance data with collected / aggregated ML performance data for a group of devices. The computing device can identify the non-self-ID device based on the comparison.

[0248] The computing device may identify trends, e.g., performance trends, associated with the identified non-self-ID device. Based on the identified trends, the computing device may determine that the non-self-ID device is performing differently from aggregated typical operational ML collected data. For example, as described herein, the computing device may determine that the non-self-ID device may have improved or degraded performance, e.g., compared to aggregated ML-generated data. The computing device may monitor the performance of the non-self-ID device and determine a relationship between the differences in the non-self-ID device and the aggregated ML performance data. For example, the computing device may monitor and / or determine that the non-self-ID device may account for differences in outcome, use, duration of use, and / or performance.

[0249] The computing device can analyze one or more differential outputs between current performance data and aggregate data associated with the device and / or HCP, for example, based on a comparison described herein. Based on the comparison of the differential outputs, the computing device can determine that the variance is improving or degrading the operational performance of the device.

[0250] In an example, a computing device may determine that a device's performance data (e.g., current performance data associated with a device used in a procedure) has a shorter usage time compared to aggregate data for a group of the same devices. The computing device may determine, for example, using an ML algorithm, that the device's performance data has the same or a higher success rate for a surgical procedure compared to aggregate data for the same group of devices. Based on the information (e.g., the shorter usage time and the same / higher success rate), the computing device may determine that the device's performance data improves the device's operational performance.

[0251] The data and / or aggregate data may be determined from an ML model. For example, aggregate data for a group of the same devices may be determined from information derived from an ML model associated with the group of the same devices. For example, an ML model associated with the group of the same devices may be configured to indicate information about the aggregate data for the group of the same devices.

[0252] In an example, a computing device may determine that a device's performance data (e.g., current performance data associated with a device used in a procedure) has a longer usage time compared to aggregate data for a group of the same devices. The computing device may determine, for example, using an ML algorithm, that the device's performance data has the same or a lower success rate for a surgical procedure compared to aggregate data for the same group of devices. Based on the information (e.g., the longer usage time and the same / lower success rate), the computing device may determine that the device's performance data is degrading the device's operational performance.

[0253] Additionally and / or alternatively, the computing device may collect information (e.g., related to performance data) associated with the device being monitored, as described herein. For example, the computing device may collect regional data associated with the device, other procedures using similar functions and / or subfunctions, or other local hospitals. The computing device may use the collected information as a benchmark for comparing performance data associated with the device. For example, the computing device may use the information as a benchmark for comparing at least one of outcome, complexity, throughput, efficiency, and / or cost for the device. The computing device may use the collected information to determine (e.g., further determine) improved or degraded operation of the device.

[0254] A computing device may determine a configuration associated with a device. For example, a computing device, such as a hub or surgical hub, may determine a current configuration associated with a surgical device in an operating room being used in a procedure. The computing device may determine a configuration, e.g., a current configuration, associated with a device based on the device transmitting information to the computing device.

[0255] In examples, a device can establish a connection with a computing device and / or identify itself to the computing device, and can send current configuration information to the computing device.

[0256] In examples, the device may not self-identify the computing device. The computing device may determine and / or record the configuration of the non-self-identified device. As described herein, the computing device may identify the non-self-identified device. For example, the computing device may obtain performance data, such as a configuration, associated with the non-self-identified device. The computing device may record and / or monitor performance data (e.g., a configuration) associated with the non-self-identified device. As described herein, the device may be a surgical device being used in a surgical procedure, and an HCP, such as a surgeon, is performing the surgical procedure using the surgical device.

[0257] The computing device may identify a performance signature associated with the device based on the acquired performance data. The performance signature may be or may include at least one of trends, characteristics, and / or configuration information associated with the device. The computing device may determine whether the non-self-identified device is an authentic original equipment manufacturer (OEM) device or a counterfeit device (e.g., an imitation device) based on the identified performance signature. For example, the computing device may compare the identified performance signature (e.g., using recorded and / or monitored configurations associated with the non-self-identified device as described herein) with configurations of known authentic OEM devices. For example, the computing device may compare recorded and / or monitored configurations associated with the non-self-identified device with a predefined list of configurations of known authentic OEM devices. The computing device may identify the non-self-identified device based on, for example, the comparison.

[0258] In an example, a computing device can identify a non-self-ID device using an ML algorithm. The computing device can utilize an ML algorithm that can review data from the non-self-ID device. For example, the computing device can be configured to analyze data from the non-self-ID device to look for trends, such as the performance signatures described herein. As described herein, the computing device can compare identified trends, such as performance signatures associated with the device, to a predefined list of known configurations of known devices. Based on the comparison, the computing device can determine whether the non-self-ID device is a genuine OEM device or a counterfeit. For example, if the computing device determines that the identified trends are similar (e.g., match) to the predefined list of known configurations of known devices, the computing device can determine that the non-self-ID device is a genuine OEM device. If the computing device determines that the identified trends are not similar (e.g., do not match) to the predefined list of known configurations of known devices, the computing device can determine that the non-self-ID device is a counterfeit. The computing device can continue to monitor and / or record the configuration of the non-self-ID device.

[0259] In an example, a computing device can identify a non-self-identified device using data, such as data using an ML model and / or an ML algorithm. The computing device can configure an ML algorithm to analyze data from the non-self-identified device (e.g., to train an ML model). The data from the non-self-identified device may be associated with surgical information described herein (e.g., with respect to FIGS. 7A-7D ). The computing device may use and / or be configured to use the data to train an ML model (e.g., using an ML process and / or an ML algorithm) described herein. For example, the computing device can use surgical information, such as data from the non-self-identified device, as input to the ML process and / or ML algorithm. The computing device can use the surgical information to train an ML model, e.g., using an ML process and / or an ML algorithm. The computing device can use one or more training methods suitable for using surgical information (e.g., with respect to FIGS. 8A-8B ) to train the ML model. For example, the data from the non-self-identified device can be used to train an ML model using supervised learning, such as the supervised learning algorithm described herein. Output data from the ML process and / or ML algorithm (e.g., a trained ML model) may be or include data suitable for a computing device to identify trends associated with non-self-identified devices. For example, the trained ML model (e.g., output data from the ML process and / or ML algorithm) may be or provide information (e.g., equivalent information) to a computing device that the non-self-identified device is artificial, tampered with, or irregular based on data from the non-self-identified device (e.g., data associated with a counterfeit device).As described herein, a computing device can configure an ML algorithm to look for trends, determine reliability, and / or identify the likelihood that data from the device is artificial, altered, or irregular. The computing device can configure (e.g., enable) the ML algorithm to identify the source of the error if the computing device and / or the ML algorithm determines that the device's data is altered and / or irregular. The computing device can configure (e.g., enable) the ML algorithm to adjust and / or remove suspicious data, such as artificial and / or altered. The computing device can process other data based on the adjustment and / or removal of suspicious data, e.g., improve processing of the data to identify the device.

[0260] The ML process and / or algorithm may use data from a device, such as a surgical device. For example, the data from the device may be or may include the surgical information described with respect to FIGS. 7A-7D . The ML process and / or algorithm may train the data, for example, using supervised training as described with respect to FIGS. 8A-8B . For example, the ML process and / or algorithm may use the data from the surgical device to train an ML model and output trained model data (e.g., ML model data). The trained model data may be or include information associated with normal parameter(s) associated with the surgical device. For example, the computing device may use a list of configurations from known devices (e.g., authentic OEM devices) to train the ML model. The trained ML model may be or may include information associated with normal parameters of the known device. The computing device may identify and / or recognize normal parameters associated with a known device, such as an authentic OEM device, based on the trained model and / or ML model data. For example, the computing device may analyze data from a surgical device (e.g., monitored / recorded data from the device and / or ML trained data associated with a device described herein). The computing device may compare the data from the device with other data (e.g., information related to normal parameters of known devices and / or other ML training data for known devices described herein). Based on the comparison, the computing device may determine whether the data from the device is irregular and / or whether the data from the device is out of range. The computing device may have trained models for data of authentic OEM devices having one or more of normal operating data, catastrophic failure data, device failure data, etc.As described herein, a computing device can use one or more trained models to determine whether a device, such as a non-self-identified device, is an authentic OEM device, an imitation device, is operating under normal parameters, or is behaving erratically, such as experiencing a catastrophic failure and / or device failure.

[0261] In examples, the computing device may determine whether monitored / recorded data from the device is normal performance data, out-of-bounds data, or irregular data using at least one of Kriging modeling techniques, x̂k factorial design, etc. The computing device may use one or more of the techniques described herein based on an analysis of the monitored / recorded data from the device that the data is bad, corrupted, or outside of normal parameters.

[0262] In an example, a computing device can use an ML algorithm to collect information associated with a device, such as a non-self-identified device, a determined counterfeit device, and / or a genuine OEM device determined to be behaving abnormally. As described herein, the computing device may determine that a device is behaving abnormally. If the computing device determines that the device does not conform to normal performance, the computing device can identify at least one of the facility associated with the device, the HCP who used the device, and the surgical procedure that does not conform to the device's normal performance. The computing device can use the collected information to generate a trained model, for example, using the ML processes and / or algorithms described herein.

[0263] The computing device may use the collected information, for example, as described herein, to identify and / or locate pools (e.g., sub-pools) of devices that are performing abnormally. For example, the computing device may use the collected information as a means of identifying pools of devices for further operation and / or reuse of devices.

[0264] The computing device may collect data / information associated with the device. The computing device may collect data and / or information about the device to establish and / or update a safe operating envelope for the device's normal operation. In an example, the computing device may collect data generated from engineering trials. In an example, the computing device may collect data associated with the device during use by the HCP. For example, the device may upload usage data and / or performance data to the computing device periodically, upon request from the computing device, etc. The HCP may manually upload data to the computing device. The computing device can determine product reliability and / or downtime based on the data. The data from the device may be or include normal usage data, catastrophic failure condition data, device failure condition data, etc. As described herein, the computing device can use the collected data to train a model, for example, using the ML processes and / or algorithms described herein.

[0265] In examples, a computing device may collect information about the device when one or more conditions are met. For example, the computing device may collect controlled samples and / or partner actions in which one or more factors are controlled (e.g., based on one or more conditions being met).

[0266] In an example, a computing device may collect information about a device based on a region. For example, the computing device may collect information about a device for region-specific operation. The computing device may use an ML algorithm to determine that one particular region has degraded device performance. Based on such analysis, the computing device may collect information about the device for the identified region where the device is experiencing performance degradation.

[0267] In an example, a computing device can automate data collection for the device. For example, the computing device may generate one or more boundaries. When the computing device determines that it has reached or is in the process of reaching one or more of the generated boundaries, the computing device can autonomously collect data associated with the device.

[0268] Based on the collected / recorded data associated with the device, the computing device can determine whether the device is known. For example, as described herein, the computing device can compare the collected / recorded data associated with the device to a list of configurations associated with known authentic OEM devices to determine whether the device is an authentic OEM device. In an example, the computing device can compare the collected / recorded data associated with the device to a list of configurations associated with known counterfeit devices to determine whether the device is a counterfeit device.

[0269] In an example, the computing device may compare collected / recorded data associated with the device with a list of configurations associated with semi-unknown devices. For example, the computing device may have a list of configurations for semi-known devices that are neither known genuine OEM devices nor known counterfeit devices. If the computing device determines that the device is a semi-known device, the computing device may continue to collect information associated with the device. The computing device may transmit data of semi-known devices to other computing devices, edge devices, and / or the cloud for a user to research and / or create groups of semi-known devices.

[0270] In an example, a computing device may compare collected / recorded data associated with a device to a list of configurations associated with known authentic OEM devices, known counterfeit devices, and / or quasi-unknown devices. If the computing device determines that the collected / recorded data associated with the device is not similar to (e.g., does not match) at least one of the lists of configurations of known authentic OEM devices, known counterfeit devices, and / or quasi-unknown devices, the computing device may identify the device as an unknown device, such as a suspicious device. The computing device may transmit the data to other computing devices, edge devices, and / or the cloud for a user to investigate and / or classify the device as an unknown device.

[0271] The ML model may determine and / or classify data that the ML model and / or ML algorithm have previously seen. The ML algorithm may attempt to make an inference about what the algorithm best believes to be something. For example, the ML algorithm may not be able to discover and / or classify the collected and / or recorded data described herein as an unknown device. The computing device may transmit the collected and / or recorded data to other computing devices, edge devices, and / or the cloud for a user to investigate and / or further classify the device.

[0272] A computing device may use geographic data to determine and / or identify a device. For example, a computing device may utilize region-specific data such as electrical operating frequency and / or voltage to determine a geographic region. If the computing device determines that the electrical operating frequency is 50 Hz, the computing device may identify that the device is located in Europe or Asia. If the computing device determines that the electrical operating frequency is 60 Hz, the computing device may identify that the device is located in North America.

[0273] 9 illustrates a flow diagram 50800 of a computing device for determining whether a surgical device is an OEM device. As illustrated in 50802 and / or as described herein, a computing device, such as a surgical hub, can obtain performance data for one or more devices, such as one or more surgical devices in an operating room. The performance data may be or may include status information associated with the device, usage information associated with the device, an HCP using the device, etc.

[0274] As shown at 50804, the computing device may identify a performance signature of the surgical device. For example, as described herein, the computing device can identify the performance signature of the surgical device based on acquired performance data associated with the surgical device. Based on the identified performance signature of the surgical device, the computing device can determine whether the surgical device is an OEM device or a counterfeit device, for example, as illustrated at 50806. For example, as described herein, the computing device can compare the acquired performance data (e.g., 50802) and / or the identified performance signature (e.g., 50804) with data having a list of normal performance information associated with the surgical device. The data and / or ML-generated data (e.g., information from an ML-trained model) may be or include a list of capabilities associated with a list of OEM devices and / or a list of capabilities associated with a counterfeit device.

[0275] The computing device may compare the acquired performance data and / or the identified performance signature with data (e.g., information and / or data from a trained ML model). Based on the comparison, the computing device can determine whether the device is an OEM device or a counterfeit device. In an example, if the computing device determines that the acquired performance data and / or the identified performance signature are similar to (e.g., match) and / or within a predetermined threshold (e.g., associated with the OEM device), the computing device can determine that the device is an OEM device. If the computing device determines that the acquired performance data and / or the identified performance signature are dissimilar to (e.g., do not match) and / or exceed a predetermined threshold (e.g., associated with the OEM device), the computing device can determine that the device is a knock-off or counterfeit device. In an example, if the computing device determines that the acquired performance data and / or the identified performance signature are similar to (e.g., match) and / or within a predetermined threshold (e.g., associated with a counterfeit device), the computing device may determine that the device is a counterfeit device. If the computing device determines that the acquired performance data and / or the identified performance signature are not similar to (e.g., do not match) and / or exceed a predetermined threshold (e.g., associated with a counterfeit device), the computing device may determine that the device is an OEM device.

[0276] In an example, the computing device may determine, based on the comparison, that the acquired performance data and / or the identified performance signature are not similar to (e.g., do not match) data associated with the OEM device and / or the counterfeit device. For example, the computing device may determine that the acquired performance data and / or the identified performance signature are not similar to (e.g., may not match) data associated with the OEM device and / or the counterfeit device. The computing device may determine that the device may be an unknown device and / or an unidentified device. The computing device may monitor and acquire performance data associated with the surgical device. Based on the monitored / acquired data, the computing device may use and / or configure the data to train an ML model. The computing device may configure the ML trained model to identify and / or generate a list for classifying unknown / unidentified devices.

[0277] In an example, the computing device may determine that the surgical device is a counterfeit device and / or a knock-off device. The computing device may continuously acquire and / or monitor performance data associated with the surgical device. The computing device may input (e.g., provide) performance data associated with the identified counterfeit surgical device and train an ML model (e.g., using an ML process and / or an ML algorithm) to determine (e.g., further determine) information associated with the counterfeit surgical device. For example, the information associated with the counterfeit surgical device may be or may include a manufacturing facility of the counterfeit device, an HCP using the counterfeit device, a surgical procedure associated with the counterfeit device, a medical facility that used the counterfeit device, etc.

[0278] As shown in 50808, the computing device may be stabilized when the performance data is within normal operating parameters (e.g., when the computing device determines that the computing device is determined to be an OEM device).

[0279] In an example, the computing device may determine that the surgical device is an OEM device. If the computing device identifies the surgical device to the OEM device, the computing device may obtain an ML model (e.g., data) associated with the authentic OEM device. The data from the ML model associated with the authentic OEM device may be or include data associated with normal operating parameters for the authentic OEM device. Based on the obtained data associated with the authentic OEM device, the computing device may determine (e.g., based on a comparison) that the performance data is within normal operating parameters.

[0280] If the computing device determines that the performance data is outside of normal operating parameters, the computing device may send a warning to the HCP, as shown in 508010, for example. In an example, the computing device may receive data (e.g., information associated with an ML model) associated with a list of OEM devices. The data (e.g., ML model information) may be or include a list of collected performance data associated with the list of OEM devices. The list of collected performance data may be or include a list of normal performance data, a list of catastrophic failure performance data, a list of device failure performance data, etc. Based on the data, the computing device may compare the acquired performance data with machine learning data. If the computing device determines that the acquired performance data is similar to (e.g., matches) the data (e.g., is and / or includes a list of catastrophic failure performance data or a list of device failure performance data) and / or is within a threshold level, the computing device may determine a potential cause of the error. The computing device may provide a potential solution based on the data. For example, the computing device may send a warning message to the HCP. The warning message may be or may include the cause and / or potential solution of the identified potential error.

[0281] 10 illustrates an authentic OEM device transmitting performance data to a computing device and a counterfeit device transmitting performance data. For example, as described herein, a computing device can obtain performance data from one or more devices in an OR (e.g., 50802). In an example, computing device 50824 can obtain performance data 50822 from authentic OEM surgical stapler 50820. In an example, computing device 50824 can obtain performance data 50828 from counterfeit surgical stapler 50826. As described herein, performance data 50822, 50828 can be or include structured data (e.g., a serial number associated with the surgical stapler) and / or unstructured data (e.g., a firing force curve associated with the surgical stapler, an access change in frequency of force peaks associated with the surgical stapler, etc.).

[0282] As described herein, the authentic OEM surgical stapler 50820 and / or the counterfeit surgical stapler 50826 can transmit self-identification information in the performance data 50822, 50828. For example, the authentic OEM surgical stapler 50820 can include a serial number associated with the surgical stapler in the self-identification performance data 50822. The counterfeit surgical stapler 50826 can include a serial number associated with the surgical stapler in the self-identification performance data 50828 and can operate (e.g., mimic) the authentic OEM surgical stapler.

[0283] The computing device 50824 can identify a performance signature associated with one or more devices (e.g., 50804). For example, the computing device 50824 can identify a performance signature associated with the authentic surgical stapler 50820 and the counterfeit surgical stapler 50826 based on performance data 50822 obtained from the authentic surgical stapler 50820 and based on performance data 50828 obtained from the counterfeit surgical stapler 50826. The obtained performance data 50822, 50828 may be or include unstructured data. For example, the unstructured data may be or include a force versus firing curve associated with the surgical stapler, an access change in frequency of force peaks associated with the surgical stapler, etc.

[0284] As described herein, the computing device 50824 may be configured with a processor to execute an ML algorithm to identify performance signatures associated with the authentic surgical stapler 50820 and / or the counterfeit surgical stapler 50826. For example, the computing device 50824 can identify performance signatures associated with the authentic surgical stapler 50820 and / or the counterfeit surgical stapler 50826 based on the unstructured data in the performance data 50822, 50828. The computing device 50824 can use the identified performance data to train an ML model. The computing device 50824 can use the trained ML data to determine whether one or more devices, such as surgical staplers, are authentic OEM devices or counterfeit devices.

[0285] As described herein, the computing device can use data from the trained ML model (e.g., output from the ML algorithm) to compare the identified performance signature of the surgical stapler to a list of configuration and / or performance data of authentic OEM surgical staplers. Based on the comparison, the computing device 50824 can determine that the identified performance signature and / or the obtained performance data 50822 are similar (e.g., match) to the list of configuration and / or performance data of authentic OEM surgical staplers. Based on the determination (e.g., similarity and / or match), the computing device 50824 can determine that the surgical stapler is an authentic OEM surgical stapler 50820.

[0286] As described herein, the computing device can compare the identified performance signature of the surgical stapler to a list of configuration and / or performance data of counterfeit surgical staplers. Based on the comparison, the computing device 50824 can determine that the identified performance signature and / or the obtained performance data 50828 are similar to (e.g., match) the list of configuration and / or performance data of counterfeit surgical staplers. Based on the determination (e.g., similarity and / or match), the computing device 50824 can determine that the surgical stapler is a counterfeit surgical stapler 50828.

[0287] In an example, the computing device can identify a device for a surgical procedure in an operating room. For example, the computing device may detect a surgical stapler connected to the computing device. As described herein, the computing device can acquire performance data associated with the surgical stapler, for example, while the HCP is using the surgical stapler. The computing device can identify a performance signature associated with the surgical stapler based on the acquired performance data. The computing device can use an ML algorithm to compare the identified performance signature of the surgical stapler to a list of configurations and / or performance data of authentic OEM surgical staplers. In an example, the computing device can determine access changes in force to fire curves and frequency of force peaks based on the performance data associated with the surgical stapler. The computing device can compare the force versus firing curve and / or force peak associated with the surgical stapler to data (e.g., data from an ML trained model). For example, the data (e.g., data from an ML trained model) can be or include a list of force versus firing curves and / or force peaks associated with a group of authentic OEM surgical staplers. The computing device can determine whether the surgical stapler being used is an authentic OEM surgical stapler based on a comparison of the curves and / or force peaks. In an example, as described herein, if the computing device determines that the curves and / or force peaks (e.g., based on performance data) of the surgical stapler are similar to (e.g., match) and / or are within a threshold difference from data of the authentic OEM surgical stapler (e.g., data from an ML-trained model), the computing device can determine that the surgical stapler is an authentic OEM device.In an example, as described herein, if the computing device determines that the curves and / or force peaks (e.g., based on performance data) of the surgical stapler differ (e.g., by more than a threshold difference) from data of an authentic OEM surgical stapler (e.g., data from an ML-trained model), the computing device can determine that the surgical stapler is a counterfeit surgical stapler. As described herein, the computing device can compare the performance data (e.g., curves and / or force peaks) of the surgical stapler with other data (e.g., authentic OEM surgical staplers that are not functioning properly, such as in a catastrophic and / or device failure situation) to determine whether the surgical stapler is not functioning properly.

[0288] In an example, a computing device may obtain performance data of a surgical stapler. The computing device may identify a magnitude of the buckling load of the surgical stapler. The magnitude of the buckling load may be affected by buckling characteristics of the staple wire, such as wire diameter and / or unsupported length of the wire. The computing device may compare the performance data (e.g., the magnitude of the buckling load of the surgical stapler) to data (e.g., data from an ML-trained model) that is a list of or includes the magnitudes of the buckling loads of authentic OEM surgical staplers. Based on the comparison, the computing device may determine a relative peak between a single driver and a dual driver of the surgical device. The computing device may identify whether the surgical stapler being used is an authentic OEM device or a counterfeit device based on the different buckling loads of the staple wire (e.g., between an authentic OEM surgical stapler and a counterfeit surgical stapler).

[0289] In an example, a computing device can obtain performance data associated with a device, such as a surgical device, as the load increases and the time to buckle. Based on a comparison with data (e.g., data from an ML trained model) that may be or may include performance data associated with an authentic OEM device and the time to achieve the different loads, the computing device can determine whether the device is an OEM device or a counterfeit device.

[0290] In an example, a computing device can compare operational data of devices such as surgical staplers. The operational data may be or may include a firing load and / or motor current associated with the surgical stapler. The computing device can access the characteristics (e.g., the firing load and / or motor current) and determine expected outcomes (e.g., expected firing timing, range, frequency, etc.) of the surgical stapler based on the data (e.g., data and / or data from an ML-trained model associated with the authentic OEM device). As described herein, the computing device can compare the actual outcomes, and based on the differences, the computing device can determine whether the device is an OEM device or a counterfeit device.

[0291] In an example, a computing device may acquire operational data of a device, such as a radio frequency (RF) handpiece device. For example, the computing device may acquire operational data of an RF handpiece device connected to a generator. The computing device may acquire and / or determine a performance signature of the RF handpiece device connected to the generator. In an example, the computing device may acquire circuit impedance (e.g., measure capacitance). In an example, the computing device may utilize a powered closure to ping a motor (e.g., during re-closing), for example, as part of a wake-up cycle. Based on the acquired performance signature of the RF handpiece device, the computing device may compare the acquired performance signature with data (e.g., data from an ML trained model) of a group of authentic OEM RF handpiece devices. As described herein, the computing device may determine, for example, based on the comparison, whether the RF handpiece device is an authentic OEM RF handpiece device or a counterfeit RF handpiece device. The data (e.g., data from an ML trained model) of the group of authentic OEM RF handpiece devices may be or include performance data of circuit impedance in a narrow band. The computing device may flag (e.g., send a warning message) if the device is a counterfeit device. For example, as described herein, if the computing device determines that the device is a counterfeit device, the computing device may send a warning to the HCP and / or continue to monitor performance data of the determined counterfeit device.

[0292] In an example, a computing device may obtain operational data of a device, such as a surgical cutting device, connected to a generator. The computing device may ping a blade associated with the surgical cutting device and, for example, obtain a frequency associated with the ping as operational data. As described herein, the computing device may have data (e.g., data from an ML-trained model) of frequency information associated with an authentic OEM surgical cutting device. The computing device may compare the operational data of the surgical cutting device with the data (e.g., data from an ML-trained model) to determine whether the surgical cutting device is an OEM device or a counterfeit device.

[0293] The computing device may determine that the device is a counterfeit device (e.g., a non-OEM device). If the computing device determines that the device is a counterfeit device, the computing device may flag the device as a knock-off device, a counterfeit device, an imitation device, etc. The computing device may continue to acquire performance data of the counterfeit device. In an example, the computing device may use the acquired performance data to generate data (e.g., data from an ML-trained model) for configuration information associated with the counterfeit device.

[0294] Alternatively and / or additionally, the computing device may notify the HCP that the device they are using is a counterfeit device. For example, the computing device may send an alert to the HCP that the device is a counterfeit device. The computing device may send a warning to inform the HCP of potential dangers associated with using the counterfeit device.

[0295] Alternatively and / or additionally, the computing device may prevent the HCP from using a counterfeit device. As described herein, a counterfeit device may have different performance data compared to an authentic OEM device. Differences in performance data may result in unexpected outcomes (e.g., timing delays, providing too much current, providing too little current, using a different frequency, etc.), potentially posing a risk to the patient and / or the HCP. The computing device may prevent the HCP from using a counterfeit device. For example, the computing device may block control inputs of an identified counterfeit device.

[0296] As described herein, the computing device can compare the device's performance data with data (e.g., data from an ML-trained model). The computing device can compare the device's performance data with data (e.g., data from an ML-trained model) associated with an authentic OEM device. Based on the comparison (e.g., if the comparison data is similar to, e.g., coincides with, and / or is within, a threshold boundary), the computing device can determine whether the device is an authentic OEM device or a counterfeit device. The computing device may use the data (e.g., data from an ML-trained model) associated with an authentic OEM device in different situations. For example, as described herein, the data (e.g., data from an ML-trained model) may be or include a situation in which the authentic OEM device is malfunctioning, such as a catastrophic failure situation, a device failure situation, or the like. If the computing device compares operational data associated with the device with data (e.g., data from an ML-trained model) and determines that the device is not similar to (e.g., does not match) data associated with a genuine OEM device (e.g., normal usage conditions, catastrophic failure conditions, device failure conditions, etc.), the computing device can classify the device as an unknown device.

[0297] As described herein, a computing device may continue to monitor unknown devices. The computing device may transmit operational data associated with the unknown device to other computing devices, edge devices, the cloud, etc. for investigation by other computing devices. The computing device may use the operational data associated with the unknown device to train an ML model (e.g., using an ML process and / or an ML algorithm). The trained ML model may be configured to classify the unknown device as a category for future identification.

[0298] The computing device may determine whether operational data from the device is bad data. For example, if the computing device determines that operational data from the device is corrupted and / or potentially bad data, the computing device may attempt to identify the cause of the error and / or provide troubleshooting information to the HCP. For example, the computing device may detect whether operational data from the device is corrupted and / or incompatible with data associated with a group of devices (e.g., data from an ML-trained model) under normal operating conditions. Based on the detection, the computing device may adjust the surgical plan and / or provide the best available information for surgical planning. The computing device may notify the HCP of the incompatible data from the device and / or send a warning, e.g., a potential avenue, to the HCP for more careful attention or attention.

[0299] In an example, a computing device can receive operational data from a device such as a footswitch. The footswitch may use the same plug. Wires associated with the footswitch may be switched. The footswitch may be mechanically integrated. Because the wires are switched, the footswitch may not operate or may operate in an incorrect manner. The computing system can determine that the operational data from the footswitch may be bad data (e.g., malfunctioning data) and identify the source of the potential error. The computing device may transmit the source of the potential error to the HCP. For example, as described herein, the computing device can transmit a warning to the HCP that the device being used is not operating properly (e.g., incompatible and / or bad data). The computing device can provide a checklist for the HCP to pay more attention to and / or a potential troubleshooting guide for correcting the incompatible and / or bad data associated with the device.

[0300] In an example, a computing device may utilize an ML algorithm to determine the source of a potential error. The computing device can discover the source of an error in device compatibility based on probabilistic hierarchy data from the ML algorithm. For example, the computing device can use the ML algorithm and / or the ML probabilistic hierarchy data to determine that a competing product is plugged into an OEM generator. The computing device can send an alert to the HCP that the competing product is likely to not function, providing a place to begin troubleshooting.

[0301] Using a surgical device outside its designed operating range can significantly increase the risk of surgical complications and poor surgical outcomes. Each surgical device is designed and tested for use within a specific operating range, such as a specific temperature or pressure range. If the device is used outside this operating range, it may not function as expected and may cause problems during surgery. For example, if a surgical drill is used at a speed faster than it was designed for, the surgical drill may overheat and cause tissue damage, or the surgical drill may break and leave debris inside the patient. In addition, using a surgical device outside its operating range may void the manufacturer's warranty and / or make it difficult to repair or replace the device if it malfunctions. In some situations, a device may not operate within its expected operating range because, at least in part, it is not genuine. For example, a non-genuine staple may be used in combination with a genuine surgical stapler.

[0302] Operational data may be collected to determine whether a surgical device is functioning within its normal operating range, whether the device is being operated by an operator within its normal operating range, and / or whether the surgical device is authentic. Surgical device operational data refers to information generated and collected during the use of a surgical device. This may include, for example, information about the device itself, such as its identifier, and information about its use, such as the duration and timing of its operations, the surgical procedure in which it was used, and any sensed operating parameters.

[0303] However, there are many factors that can affect how a surgical device operates. For example, device operational data can be affected by factors such as whether the device is authentic, how the operator is using the device, and the condition of the device. As a result, it can be difficult to compare operational data from one surgical device with operational data from another to determine whether the surgical device is operating outside its designed operational range. Furthermore, it is possible for a non-authentic device to function similarly to an authentic device, which can further complicate the process of determining whether a surgical device is operating outside its operational range based on operational data.

[0304] As used herein, the terms "operator" and "healthcare professional (HCP)" are interchangeable.

[0305] As used herein, the terms "machine learning process" and "machine learning model" are interchangeable. As used herein, the terms "machine learning technique" and "machine learning algorithm" are interchangeable. When the term "machine learning" is used herein, it refers to either a machine learning model or a machine learning algorithm, as will be clear from the context in which the term is used.

[0306] As used herein, the terms "authentic device" and "original equipment manufacturer (OEM) device" are interchangeable. A genuine surgical device is a device that is genuine, not counterfeit. In other words, an authentic surgical device is one that is manufactured by a company that retains the rights to the device's design and specifications, and that has been properly tested and certified to meet relevant safety and performance standards. Authenticity is important for surgical devices because the use of counterfeit or partially counterfeit devices can pose serious risks to patients.

[0307] As used herein, the terms "non-authentic device" and "counterfeit device" are interchangeable. A non-authentic surgical device is a device that is intentionally made to look or function like a genuine surgical device, but is not actually manufactured by the company that holds the rights to the device's design and specifications. Counterfeit surgical devices are often made of lower quality materials and may not be properly tested or certified, which can result in poor performance and potential harm to the patient.

[0308] As used herein, the terms “operational data” and “performance data” are interchangeable. Operational data of a surgical device refers to information generated and collected during use of the surgical device. This can include information about the device itself, such as its identifier, and information about its use. Specifically, operational data can include configurations and / or parameters associated with the device while it is in use. For example, operational data can be or include at least one of firing load (e.g., motor current), circuit impedance (e.g., capacitance), frequency, feedback time, amount of force to fire curve, force peak, frequency of force peak, magnitude of buckling load, peak between single driver and / or dual driver, etc. Operational data can include structured data (e.g., a serial number associated with a surgical stapler) and / or unstructured data (e.g., a firing force curve associated with a surgical stapler, access changes in frequency of force peaks associated with a surgical stapler, etc.).

[0309] As used herein, the terms "operational signature" and "performance signature" are interchangeable. A surgical device operational signature is a representation of acquired operational data. Generally, this representation is used to simplify the operational data to enable comparison with other operational data. Typically, the representation is an embedding. The embedding is generated by training a supervised deep neural network (DNN) on the operational data. The embedding maps the operational data to a vector in an embedding space. Generally, the embedding space has fewer dimensions than the operational data so as to capture the latent structure of the operational dataset (e.g., from multiple authentic and non-authentic surgical devices). Similar examples, e.g., the embedding vectors for two authentic surgical devices, are close to each other in the embedding space.

[0310] As used herein, the terms "operating range," "normal parameters," and "normal operating parameters" are interchangeable. An operating range refers to the range of operational inputs used to control a surgical device, preferably to perform a surgical step. Typically, the range does not correspond to all possible operational configurations of the device for a particular operational input. For example, a surgical device may have an output power delivery of 0 to 50 W, but the operational range may be 0 to 40 W. In other words, the operational range of a surgical device may be a subset of all possible operational configurations in which the surgical device can operate. Such an operational range is typically defined by the surgical device manufacturer and is based on the device's design and specifications. It is important that a surgical device operate within its operational range to ensure its safety and effectiveness. An operational range may have an upper limit and no lower limit. Alternatively, an operational range may have a lower limit and no upper limit. Alternatively, an operational range may have upper and lower limits that define a safe operating range. In some embodiments, the operational range may include an upper safe operating range and a lower safe operating range.

[0311] As used herein, the terms “identifier” and “self-ID information” are interchangeable. A surgical device can identify itself using an identifier, such as a serial number or unique code assigned to the device by the manufacturer. This identifier may be printed on the device itself or stored electronically in memory. When the surgical device is used in a surgical procedure, the identifier may be entered into or retrieved by a computer system or other device used to store operational data. In some cases, the identifier may also be used to access information about the device, such as instructions for use or other important safety information. This can help ensure that the device is used properly and safely during the surgical procedure, i.e., within its operating range.

[0312] The following is a non-exhaustive list of embodiments that may or may not be claimed.

[0313] 1. A computing system comprising: a processor, the processor comprising: Obtaining performance data associated with a surgical device being used in a surgical procedure performed by a healthcare professional (HCP); Identifying a performance signature associated with the surgical device based on the acquired performance data; determining whether the surgical device is an authentic original equipment manufacturer (OEM) device based on the identified performance signature associated with the surgical device; determining, based on a determination that the surgical device is an authentic OEM device, that the acquired performance data is within normal operating parameters associated with an authentic OEM device; and sending a warning message to the HCP based on a determination that the acquired performance data is outside of normal operating parameters associated with the authentic OEM device; It is configured as follows: Computing system.

[0314] 2. Identifying a performance signature associated with a surgical device; 2. The computing system of claim 1, wherein the processor is configured to compare the acquired performance data with information associated with a machine learning model; 1. A computing system, wherein the machine learning model is configured to indicate information regarding a list of capabilities associated with a list of OEM devices or a list of capabilities associated with a list of counterfeit devices.

[0315] 3. A computing system as described in embodiment 2, wherein the processor is configured to identify the surgical device as an authentic OEM device provided that the acquired performance data is similar to a list of capabilities associated with a list of OEM devices.

[0316] 4. A computing system as described in embodiment 2, wherein the processor is configured to identify the surgical device as a counterfeit device if the acquired performance data is similar to a list of capabilities associated with a list of counterfeit devices.

[0317] 5. The computing system of embodiment 2, wherein, provided that the obtained performance data is not similar to a list of capabilities associated with a list of OEM devices or a list of counterfeit devices, the processor: continuously capturing performance data associated with the surgical device; Using the obtained performance data to train a machine learning model; and adding the surgical device to a list of counterfeit devices based on the condition that the identified performance data is dissimilar to a list of capabilities associated with a list of OEM devices; 1. A computing system configured to:

[0318] 6. The computing system of embodiment 2, wherein the processor: Based on the determination that the surgical device is a counterfeit device, continuously acquiring performance data; and inputting the acquired performance data to train a machine learning model configured to determine information associated with the counterfeit device, wherein the information associated with the counterfeit device includes at least one of a manufacturing facility of the counterfeit device, an HCP using the counterfeit device, or a surgical procedure associated with the counterfeit device; 1. A computing system configured to:

[0319] 7. The computing system of embodiment 1, wherein the acquired performance data includes self-ID information associated with the surgical device, and determining whether the surgical device is a genuine OEM device comprises: a processor: determining whether the self-ID information matches a list of OEM devices; classifying the surgical device as a genuine OEM device based on a determination that the self-ID information matches a list of OEM devices; and classifying the surgical device as a counterfeit device based on a determination that the self-ID information does not match a list of OEM devices; 1. A computing system, comprising:

[0320] 8. The computing system of embodiment 1, wherein the processor: Identifying a surgical device from a list of OEM devices; and obtaining a machine learning model associated with the authentic OEM device based on the identified surgical device, the machine learning model being configured to indicate normal operating parameters associated with the authentic OEM device; It is configured as follows: Computing system.

[0321] 9. The computing system of embodiment 1, wherein the processor: receiving a machine learning model configured to indicate a list of collected performance data associated with the authentic OEM device, wherein the list of collected performance data associated with the authentic OEM device includes a list of normal performance data, a list of catastrophic failure performance data, or a list of device failure performance data; comparing the obtained performance data with information associated with the machine learning model; and determining potential error causes and potential solutions, provided that the acquired performance data is similar to the list of catastrophic failure performance data or the list of device failure performance data; 1. A computing system configured to:

[0322] 10. The computing system of embodiment 1, wherein the processor: receiving a machine learning model including data associated with a genuine OEM device, the data associated with the genuine OEM device including error data and potential solution data for addressing the corresponding error; Identifying potential sources of error associated with the surgical device and potential solutions for correcting the potential sources of error associated with the surgical device based on information associated with the received machine learning model; and Include identified potential error causes and potential solutions in warning messages to HCPs; 1. A computing system configured to:

[0323] 11. A method comprising: Obtaining performance data associated with a surgical device being used in a surgical procedure performed by a healthcare professional (HCP); identifying a performance signature associated with the surgical device based on the acquired performance data; determining whether the surgical device is an authentic original equipment manufacturer (OEM) device based on the identified performance signature associated with the surgical device; determining, based on a determination that the surgical device is an authentic OEM device, that the acquired performance data is within normal operating parameters associated with an authentic OEM device; sending a warning message to the HCP based on a determination that the acquired performance data is outside of normal operating parameters associated with the authentic OEM device; A method comprising:

[0324] 12. To identify a performance signature associated with a surgical device, the method comprises: comparing the obtained performance data with information about a machine learning model, the machine learning model configured to indicate information about a list of capabilities associated with the list of OEM devices or a list of capabilities associated with the list of counterfeit devices; 12. The method of embodiment 11.

[0325] 13. Provided that the performance data obtained is similar to a list of capabilities associated with a list of OEM devices, the method comprises: 13. The method of embodiment 12, comprising identifying the surgical device as an authentic OEM device.

[0326] 14. Provided that the obtained performance data is similar to a list of capabilities associated with a list of counterfeit devices, the method comprises: 13. The method of embodiment 12, comprising identifying the surgical device as a counterfeit device.

[0327] 15. The method of embodiment 12, provided that the obtained performance data is not similar to a list of capabilities associated with a list of OEM devices or a list of counterfeit devices, the method further comprising: continuously acquiring performance data associated with the surgical device; using the obtained performance data to train a machine learning model; adding the surgical device to a list of counterfeit devices based on the condition that the identified performance data is not similar to a list of capabilities associated with the list of OEM devices; A method comprising:

[0328] 16. The method is Based on the determination that the surgical device is a counterfeit device, continuously acquiring performance data; inputting the acquired performance data to train a machine learning model configured to determine information associated with the counterfeit device, wherein the information associated with the counterfeit device includes at least one of a manufacturing facility of the counterfeit device, an HCP using the counterfeit device, or a surgical procedure associated with the counterfeit device; 13. The method of embodiment 12, comprising:

[0329] 17. The method further comprising: determining whether the surgical device is an authentic OEM device, wherein the obtained performance data includes self-ID information associated with the surgical device; determining whether the self-ID information matches a list of OEM devices; classifying the surgical device as a genuine OEM device based on a determination that the self-ID information matches a list of OEM devices; classifying the surgical device as a counterfeit device based on a determination that the self-ID information does not match the list of OEM devices; 12. The method of embodiment 11, comprising:

[0330] 18. The method is identifying a surgical device from a list of OEM devices; obtaining a machine learning model associated with the authentic OEM device based on the identified surgical device, the machine learning model being configured to indicate normal operating parameters associated with the authentic OEM device; 12. The method of embodiment 11, comprising:

[0331] 19. The method is receiving a machine learning model configured to indicate a list of collected performance data associated with a genuine OEM device, wherein the list of collected performance data associated with the genuine OEM device includes normal performance data, a list of catastrophic failure performance data, or a list of device failure performance data; comparing the obtained performance data with information associated with the machine learning model; determining potential error causes and potential solutions, provided that the acquired performance data is similar to the list of catastrophic failure performance data or the list of device failure performance data; 12. The method of embodiment 11, comprising:

[0332] 20. The method is receiving a machine learning model including data associated with a genuine OEM device, the data associated with the genuine OEM device including error data and potential solution data for addressing the corresponding error; identifying potential sources of error associated with the surgical device and potential solutions for correcting the potential sources of error associated with the surgical device based on information associated with the received machine learning model; Including the identified potential error causes and potential solutions in the warning message to the HCP; 12. The method of embodiment 11, comprising:

[0333] [Embodiment] (1) A processor-implemented method, the method comprising: obtaining operational data associated with a surgical device being used in a surgical procedure; identifying a motion signature associated with the surgical device based on the acquired motion data; determining whether the surgical device is an authentic device based on the identified operational signature associated with the surgical device; determining whether the acquired operational data is within an operational range associated with the authentic device based on a determination that the surgical device is the authentic device; sending a warning message to an operator based on the determination that the acquired operational data is not within the operational range associated with the authentic device; A method comprising: (2) The method of embodiment 1, wherein the surgical procedure is performed by the operator. (3) identifying the motion signature associated with the surgical data includes: A method as described in embodiment 1 or 2, comprising inputting the acquired operational data into a first machine learning model, wherein the first machine learning model is trained using operational data associated with a plurality of genuine devices and / or operational data associated with a plurality of non-genuine devices. (4) The method of embodiment 3, wherein the operational data associated with the plurality of genuine devices is labeled as originating from a genuine device, and / or the operational data associated with the plurality of non-genuine devices is labeled as originating from a non-genuine device. (5) providing the behavioral signature is similar to behavioral signatures associated with the plurality of authentic devices, the method further comprising: Identifying the surgical device as the authentic device. 4. The method of embodiment 3, further comprising:

[0334] (6) providing the behavioral signature is similar to behavioral signatures associated with the plurality of non-authentic devices, the method further comprising: Identifying the surgical device as a non-authentic device. 5. The method of embodiment 3 or 4, further comprising: (7) identifying the non-authentic device from the plurality of non-authentic devices having an operational signature that is most similar to the operational signature of the surgical device based on a determination that the surgical device is a non-authentic device; retrieving information associated with a non-genuine device of the plurality of non-genuine devices having an operational signature most similar to the operational signature of the surgical device, the information including at least one of a manufacturing facility of the non-genuine device, the operator using the non-genuine device, or the surgical procedure associated with the non-genuine device; 7. The method of embodiment 6, further comprising: (8) Provided that the behavioral signature is not similar to behavioral signatures associated with the plurality of genuine devices or the plurality of non-genuine devices, the method further comprises: continuously acquiring the operational data associated with the surgical device; further training the first machine learning model with the acquired motion data; 8. The method of any one of embodiments 3 to 7, further comprising: (9) Labeling the operational data as originating from a genuine device or a non-genuine device. 9. The method of embodiment 8, further comprising: (10) determining whether the surgical device is an authentic device based on the identified operational signature associated with the surgical device, providing the operational signature to a second machine learning model trained to classify whether the surgical device is authentic. 4. The method of any one of embodiments 1 to 3, comprising:

[0335] (11) A method according to any one of embodiments 1 to 10, wherein the acquired operational data does not include an identifier associated with the surgical device. (12) if the acquired operational data includes an identifier associated with the surgical device; determining whether the identifier associated with the surgical device matches stored identifiers of a plurality of authentic devices; and setting a flag indicating that the surgical device is an authentic device based on a determination that the identifier matches a stored identifier of the plurality of authentic devices; or setting a flag indicating the surgical device is a non-authentic device based on a determination that the identifier does not match stored identifiers of the plurality of authentic devices; determining whether the surgical device is an authentic device by 12. The method of any one of embodiments 1 to 11, further comprising: (13) identifying the authentic device from a plurality of authentic devices; retrieving a third machine learning model associated with the authentic device, the third machine learning model being trained to determine whether the captured operational data is within the operational range associated with the authentic device; 13. The method of any one of embodiments 1 to 12, further comprising: (14) The third machine learning model is trained using the genuine device (non-fault) operational data, catastrophic failure operational data, and / or device failure operational data, and the method further comprises: inputting the acquired motion data into the third machine learning model; determining catastrophic failure operational data or device failure operational data that is most similar to the acquired operational data, provided that the acquired operational data is not within the operational range; retrieving information associated with the most similar catastrophic failure operation data or device failure operation data; 14. The method of embodiment 13, further comprising: (15) The retrieved information associated with the most similar catastrophic failure operation data or device failure operation data includes one or more potential errors and one or more potential solutions, and the method further comprises: including the identified potential error and the potential solution in the warning message to the operator. 15. The method of embodiment 14, further comprising:

[0336] (16) A processor-implemented method, the method comprising: obtaining operational data associated with a surgical device being used in a surgical procedure; identifying a motion signature associated with the surgical device based on the acquired motion data; determining whether the surgical device is behaving abnormally based on the identified operational signature associated with the surgical device; determining whether the acquired operational data is within an operational range associated with correct operation based on a determination that the surgical device is operating abnormally; sending a warning message to an operator based on the determination that the acquired motion data is not within the motion range associated with correct motion; A method comprising: (17) A processor-implemented method, the method comprising: acquiring operational data associated with a surgical device being used in a surgical procedure performed by an operator; identifying a motion signature associated with the surgical device based on the acquired motion data; determining whether the surgical device is being operated abnormally by the operator based on the identified operational signature associated with the surgical device; determining whether the acquired operational data is within an operational range associated with correct operation based on a determination that the surgical device is not being operated; sending a warning message to an operator based on the determination that the acquired motion data is not within the motion range associated with correct motion; A method comprising: (18) A computing system comprising: A processor configured to execute the method according to any one of claims 1 to 17. A computing system comprising: (19) A computing program that, when executed by a processor, causes the processor to perform the method described in any one of embodiments 1 to 17.

Claims

1. 1. A processor-implemented method, the method comprising: obtaining operational data associated with a surgical device being used in a surgical procedure; identifying a motion signature associated with the surgical device based on the acquired motion data; determining whether the surgical device is an authentic device based on the identified operational signature associated with the surgical device; determining whether the acquired operational data is within an operational range associated with the authentic device based on a determination that the surgical device is the authentic device; sending a warning message to an operator based on the determination that the acquired operational data is not within the operational range associated with the authentic device; A method comprising:

2. The method of claim 1 , wherein the surgical procedure is performed by the operator.

3. Identifying the motion signature associated with the surgical data includes:

2. The method of claim 1, comprising inputting the acquired operational data into a first machine learning model, the first machine learning model being trained using operational data associated with a plurality of genuine devices and / or operational data associated with a plurality of non-genuine devices.

4. 4. The method of claim 3, wherein the operational data associated with the plurality of genuine devices is labeled as originating from a genuine device, and / or the operational data associated with the plurality of non-genuine devices is labeled as originating from a non-genuine device.

5. Provided that the behavioral signature is similar to behavioral signatures associated with the plurality of authentic devices, the method further comprises: Identifying the surgical device as the authentic device. The method of claim 3 further comprising:

6. Provided that the behavioral signature is similar to behavioral signatures associated with the plurality of non-authentic devices, the method further comprises: Identifying the surgical device as a non-authentic device. The method of claim 3 further comprising:

7. based on a determination that the surgical device is a non-authentic device, identifying the non-authentic device from the plurality of non-authentic devices having an operational signature that is most similar to the operational signature of the surgical device; retrieving information associated with a non-genuine device of the plurality of non-genuine devices having an operational signature most similar to the operational signature of the surgical device, the information including at least one of a manufacturing facility of the non-genuine device, the operator using the non-genuine device, or the surgical procedure associated with the non-genuine device; The method of claim 6 further comprising:

8. Provided that the behavioral signature is dissimilar to behavioral signatures associated with the plurality of genuine devices or the plurality of non-genuine devices, the method further comprises: continuously acquiring the operational data associated with the surgical device; further training the first machine learning model with the acquired motion data; The method of claim 3 further comprising:

9. Labeling the operational data as being from an authentic device or a non-authentic device. The method of claim 8 further comprising:

10. Determining whether the surgical device is an authentic device based on the identified operational signature associated with the surgical device includes: providing the operational signature to a second machine learning model trained to classify whether the surgical device is authentic; The method of claim 1 , comprising:

11. The method of claim 1 , wherein the acquired operational data does not include an identifier associated with the surgical device.

12. If the acquired operational data includes an identifier associated with the surgical device, determining whether the identifier associated with the surgical device matches stored identifiers of a plurality of authentic devices; and setting a flag indicating that the surgical device is an authentic device based on a determination that the identifier matches a stored identifier of the plurality of authentic devices; or setting a flag indicating the surgical device is a non-authentic device based on a determination that the identifier does not match stored identifiers of the plurality of authentic devices; determining whether the surgical device is an authentic device by The method of claim 1 further comprising:

13. identifying the authentic device from a plurality of authentic devices; retrieving a third machine learning model associated with the authentic device, the third machine learning model being trained to determine whether the captured operational data is within the operational range associated with the authentic device; The method of claim 1 further comprising:

14. the third machine learning model is trained using the genuine device (non-fault) operational data, catastrophic failure operational data, and / or device failure operational data, and the method further comprises: inputting the acquired motion data into the third machine learning model; determining catastrophic failure operational data or device failure operational data that is most similar to the acquired operational data, provided that the acquired operational data is not within the operational range; retrieving information associated with the most similar catastrophic failure operation data or device failure operation data; The method of claim 13 further comprising:

15. the retrieved information associated with the most similar catastrophic failure operation data or device failure operation data includes one or more potential errors and one or more potential solutions, the method comprising: including the identified potential error and the potential solution in the warning message to the operator. The method of claim 14 further comprising:

16. 1. A processor-implemented method, the method comprising: obtaining operational data associated with a surgical device being used in a surgical procedure; identifying a motion signature associated with the surgical device based on the acquired motion data; determining whether the surgical device is behaving abnormally based on the identified operational signature associated with the surgical device; determining whether the acquired operational data is within an operational range associated with correct operation based on a determination that the surgical device is operating abnormally; sending a warning message to an operator based on the determination that the acquired motion data is not within the motion range associated with correct motion; A method comprising:

17. 1. A processor-implemented method, the method comprising: acquiring operational data associated with a surgical device being used in a surgical procedure performed by an operator; identifying a motion signature associated with the surgical device based on the acquired motion data; determining whether the surgical device is being operated abnormally by the operator based on the identified operational signature associated with the surgical device; determining whether the acquired operational data is within an operational range associated with correct operation based on a determination that the surgical device is not being operated; sending a warning message to an operator based on the determination that the acquired motion data is not within the motion range associated with correct motion; A method comprising:

18. 1. A computing system comprising: A processor configured to carry out the method of any one of claims 1 to 17. A computing system comprising:

19. A computing program which, when executed by a processor, causes the processor to carry out the method of any one of claims 1 to 17.

Images