Surgical data processing associated with multiple system hierarchy levels

Abstract

A processor-executed method is described that can include acquiring first surgical data associated with a surgical procedure and identifying a first processing device for processing the surgical data based on at least one of a capability of the first processing device, a data magnitude of the first surgical data, a sensitivity to latency in processing the first surgical data, a data individuality level of the first surgical data, and an intended use of the first surgical data.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is related to the following concurrently filed applications, the contents of each of which are incorporated herein by reference: Attorney Docket No. END9438USNP1, Invention Title: "A METHOD FOR ADVANCED ALGORITHM SUPPORT". - Attorney docket number END9438USNP15, invention title "SURGICAL DATA PROCESSING ASSOCIATED WITH MULTIPLE SYSTEM HIERARCHICAL LEVELS". [Background technology]

[0002] Patient care, in general, improves when it is tailored to the individual. Because every person has different needs, surgical and interventional solutions that center every patient's unique journey can represent an efficient and innovative path to healing. At the same time, the high stakes of patient care, particularly the surgical process, often focus on conservative, repeatable activities.

[0003] For example, innovative medical technologies such as advanced surgical support computing systems and intelligent surgical instruments can improve approaches to patient care and address the specific needs of healthcare providers.

[0004] The ever-increasing availability of data and computing resources makes non-traditional algorithms, such as machine learning algorithms, a particular technological opportunity in the healthcare system. However, incorporating such non-traditional algorithms into any medical technology presents many challenges. Summary of the Invention [Means for solving the problem]

[0005] According to an embodiment of the present invention, there is provided a method executed by a processor, the method including acquiring first surgical data associated with a surgical procedure, and identifying a first processing device for processing the surgical data based on at least one of a capability of the first processing device, a data size of the first surgical data, a sensitivity to latency in processing the first surgical data, a data individuality level of the first surgical data, and an intended use of the first surgical data.

[0006] The present invention provides a method for identifying a processing device suitable for processing surgical data, ensuring that the data is processed by the most suitable processor (e.g., in terms of processing power, urgency of data processing, etc.).

[0007] The method may further include receiving surgical data associated with the surgical procedure, the surgical data including first surgical data and second surgical data, and identifying a second processing device for processing the second surgical data based on at least one of a capability of the second processing device, a data size of the second surgical data, a sensitivity to latency in processing the second surgical data, a data individuality level of the second surgical data, and an intended use of the second surgical data.

[0008] The method includes identifying an optimal processor for processing different data (or portions of data), the determination being based on processor capabilities or characteristics of the data, which may optimize processing of the data (or portions of data).

[0009] The method may further include transmitting the first surgical data to a first processing device and transmitting the second surgical data to a second processing device.

[0010] The first processing device may be different from the second processing device.

[0011] The optimal processor for processing a portion of the data may depend on the characteristics of the data and the capabilities of the processing device, which may result in different data (or portions of data) being processed by different processors.

[0012] The method may further include dividing the acquired surgical data into first surgical data and second surgical data.

[0013] The method may include separating the data into data portions based on requirements for processing of the data portions. The method may, for example, separate the data into time-critical and non-time-critical data to ensure that time-critical data can be processed as needed (e.g., a smaller portion of the data can be processed at a rate that may ultimately lead to faster results). In another example, the data may be divided based on data individuality level, where patient-identifying data is processed within a privacy boundary and the remainder is sent for processing based on other considerations, such as the size of the data.

[0014] The data division may be based on at least one of the following: data size of the first surgical data and the second surgical data, data granularity of the first surgical data and the second surgical data, timeliness of results associated with the first surgical data and the second surgical data, importance level associated with the first surgical data and the second surgical data, first data individuality level and the second data individuality level, frequency or accuracy level associated with the first surgical data and the second surgical data, resource-time availability of the first processing device and the second processing device, network bandwidth, and availability of other surgical data to be used in place of the first surgical data and the second surgical data.

[0015] The partitioning of the data and the identification of the associated processing devices for processing the partitioned data may be based on a combination of factors, for example, the data may be partitioned based on an individual level, patient identifying data may be processed within a privacy-preserving boundary, and the size of the data is considered as to whether it is sent to a hub device or an edge device.

[0016] The processor may have access to a set of rules and / or a database or look-up table that associates data characteristics with processing capabilities to identify a processor for processing the data.

[0017] The data may be pre-labeled with characteristics. For example, it may be associated with metadata or tags that identify it as time-sensitive data, patient identification data, etc. The pre-labeling may be provided by a surgeon or by a processor that is configured to label data from a particular sensor or device as having a given characteristic.

[0018] The at least one capability associated with the first processing device and / or the second processing device may include at least one of processing power, memory size, or time taken to process the first surgical data and / or the second surgical data.

[0019] The first surgical data may be first health data associated with the patient, the first health data having a first personality level, and the method may further include determining whether to transform the first health data based on a privacy protection level associated with the first processing device or a location of the first processing device and, optionally, the first personality level, and if it is determined that transformation is required, transforming the first health data such that the transformed first health data has a second data personality level, the second data personality level being lower than the first data personality level, and transmitting the first health data or the transformed first health data to the first processing device depending on whether transformation is required.

[0020] Surgical data can be generated during a surgical procedure, for example, from wearable sensors, room sensors, imaging devices, and intelligent medical devices used during the surgical procedure.

[0021] The surgical data may be processed locally for the treatment of a current patient, for example, the data may be processed to adjust the control of an intelligent surgical instrument used during a surgical procedure. Alternatively, or in addition, the surgical data may be used to improve surgical outcomes for future patients, for example, by comparing surgical instrument control settings to patient outcomes (e.g., comparing the firing force, closure force, firing rate, etc. of a surgical stapler with the integrity of the staple line it produced) to determine optimal control settings for a given step, etc. of a given surgical procedure.

[0022] The present invention provides a method for identifying a processing device suitable for processing health data and determining whether the health data requires transformation (by reducing data individuality) before sending to the identified processing device. The method thus ensures that the data is processed by the most suitable processor (e.g., in terms of processing power, urgency of processing the data, etc.) while maintaining the privacy of the data.

[0023] By reducing data individuality as data is processed away from its source, the risk from data leakage can be reduced because data being processed outside of a protected network has lower data individuality than data being processed within the protected network, and therefore, if a leakage occurs, is less likely to be traceable to the patient.

[0024] The method therefore reduces the risk of sensitive health data being leaked into the digital realm.

[0025] A privacy protection level may be associated with a data individuality level that is appropriate for data processed at that privacy protection level.

[0026] If the first processing device is located within the privacy protection boundary or associated with a high privacy protection level, conversion of the data is not required, even if it has a high data individuality. However, if the processing device is located outside the privacy protection boundary or associated with a lower privacy protection level, conversion may be required, but it depends on the data individuality level of the first health data; for example, if the data is associated with a low data individuality level, conversion is not required.

[0027] The first data individuality level may be provided by a patient or a healthcare provider, and the first health data may be associated with the provided first data individuality level (e.g., by metadata or a header associated with the first health data). Alternatively, the method may include determining the first data individuality level using a processor.

[0028] The identification of the first processing device may depend on the sensitivity of the data, with highly sensitive data being associated with data having a high data individuality level, and it is preferred that this data be processed only within a privacy-preserving device or by a device associated with a high privacy-preserving level.

[0029] Identifying the first processing device may be performed using one or more lookup tables, which may be combined with optional ranking among the lookup tables. For example, a first lookup table may associate the size of the data with the capabilities of the processing device to identify a processing device suitable for a given size of data. Similarly, the intended use of the data may be associated with the capabilities of the processing device; for example, if the intended use is for patient treatment, this may be associated with a processing device having lower capabilities, while an intended use is analysis of the data in parallel with other similar data for trend or correlation analysis may be associated with a processing device with higher capabilities. By combining lookup tables, data with an intended use associated with lower capabilities can be sent to a processing device with higher capabilities if the size of the data requires. The capabilities of the processing device can be increased when moving from the operating room, for example, by using an operating room processing device (e.g., a hub) with lower capabilities than the hospital processor, which has lower capabilities than the hospital network processing device, which has lower capabilities than the remote processing device.

[0030] The method may further include determining a first data individuality level of the first health data.

[0031] Determining the data individuality level allows for determining whether the data requires transformation before transmission to the identified processing device, potentially reducing the processing of the first health data.

[0032] For example, a parameter associated with a low data individuality level can correspond to a parameter that falls within the normal expected range for that parameter, a higher individuality level may be associated with a parameter that is outside the expected range but falls within a second, wider range, and an even higher data individuality level may be associated with a parameter that falls outside that second range but occurs only once (in the historical data). In one example, an individuality level may be associated with the standard deviation from the parameter's mean value (obtained from past surgical data). For example, a stapler firing force value may be identified as having a high data individuality (because that value was much higher than normal) and may be associated with a failed staple line as a result. This relatively unique event may thus identify the patient. Similarly, a parameter related to the rate or amount of blood loss from the staple line (e.g., as determined by staple line visualization laser Doppler) may identify a patient if the parameter is associated with an abnormally high rate / amount of blood loss.

[0033] As another example, a particular data type may be associated with a given, predetermined personality level. In such a case, the processor may use a lookup table that associates a particular type of data with its predetermined personality level. For example, a cortisol level may have a high personality level, while a potentiometer reading may have a low personality level.

[0034] As another example, if the processor has access to data from other historical procedures, the data individuality level may be determined by reviewing multiple similar data sets available for other patients.

[0035] The first data individuality level may be determined using a machine learning model.

[0036] Returning to the example of statistical significance of data, machine learning algorithms can be trained to determine the individuality level of data. In particular, a histogram (or other method of estimating a probability distribution) can be generated to calculate the standard deviation of historical data. The deviation of a given data point from the mean value can then be compared to the standard deviation or other predetermined range to classify the data point at a predetermined data individuality level.

[0037] In another example, the training data may include resource availability (e.g., memory and / or processing power availability) of various processing devices from previous surgical procedures, control algorithms associated with surgical instruments (e.g., stored locally or received from another entity, e.g., a remote server).

[0038] In another example, an unsupervised learning framework-based machine learning model may be trained on a dataset that may include inputs to find structures or patterns within the data. For example, the inputs may include parameters associated with the dataset to be processed (e.g., dataset size, acceptable latency values, etc.), a rule set (e.g., based on local privacy laws where the surgical procedure is performed), and parameters associated with various potential processing devices to which the dataset may be sent for processing. The results may be identification of one or more processing devices and / or system hierarchical levels to which the dataset may be sent for processing, and / or a data individuality level that may be applied to the dataset before sending to a selected processing device.

[0039] Identifying the first processing device may be done using machine learning.

[0040] In one example, the machine learning may be supervised. The training data used in training the local machine learning model may include previous surgical procedures, surgical parameters associated with those surgical procedures, and / or surgical datasets collected from simulated surgical procedures. The training data may include resource availability (e.g., memory and / or processing power availability) of various processing devices from previous surgical procedures, control algorithms associated with surgical instruments (e.g., stored locally or received from another entity, e.g., a remote server).

[0041] In one example, the machine learning may be unsupervised (e.g., unsupervised learning). For example, inputs may include parameters associated with the dataset to be processed (e.g., size of the dataset, acceptable latency values, etc.), a rule set (e.g., based on local privacy laws where the surgical procedure will be performed), and parameters associated with various potential processing devices to which the dataset may be sent for processing. The result may be identification of one or more processing devices and / or system hierarchical levels to which the dataset may be sent for processing, and / or a data individuality level that may be applied to the dataset before sending to a selected processing device. The data individuality level may be selected based on where the dataset is sent for processing.

[0042] The first health data may include a plurality of data points, each data point associated with a level of personality, where the first level of personality of the first health data is based on an aggregation of the levels of personality of the data points.

[0043] The transformation of the first health data may include anonymizing one or more of the plurality of data points.

[0044] This method may provide the advantage of transforming only data points with a particular data individuality level (e.g., transforming only those deemed to have a high data individuality), which may reduce data processing required before transmitting the first health data to the first processing device.

[0045] The capabilities of the first processing device may include the diversity of the data sets available to the first processing device and / or the processing power of the first processing device.

[0046] The first processing device may be identified based on data to which it has access. For example, the intended use of the first data may be to improve the outcome of a given surgical procedure by comparing the first health data and procedure outcome of a first patient with the health data and outcomes of multiple other patients. The more data that can be analyzed, the more accurate an identified trend in the data, such as that a particular firing force results in a poor outcome (blood leakage above a threshold amount), will be.

[0047] Remote devices such as cloud servers (i.e., outside the privacy boundary, e.g., outside the hospital or hospital network) may have access to much more data and may have better capacity to process that data and identify trends or correlations within the data.

[0048] In contrast, an operating room hub has access to a much smaller amount of data and may have less capacity to process the data. Thus, the hub may be used to process data during treatment but not for broader trend analysis.

[0049] The method may further include receiving second health data associated with a second patient and having a third data personality level and a third data magnitude; identifying a second processing device for processing the second health data based on at least one of the capabilities of the second processing device, the third data magnitude, sensitivity to latency in processing the second health data, and the intended use of the second health data; determining whether to transform the second health data based on a privacy protection level associated with the second processing device or a location of the second processing device and optionally the third data personality level; if conversion is determined to be required, converting the second health data such that the transformed second health data has a fourth data personality level and a fourth data magnitude, wherein the fourth data personality level is lower than the third data personality level; and transmitting the second health data or the transformed second health data to the second processing device depending on whether conversion is required.

[0050] The first processing device may be located outside the protected network and the second processing device is located inside the protected network.

[0051] The privacy protection level associated with the first processing device may be higher than the privacy protection level associated with the second processing device.

[0052] The protected network may be a Health Insurance Portability and Accountability Act (HIPAA)-based network that includes at least one healthcare facility.

[0053] One or more data points may be anonymized by one or more of editing the data points, randomizing the data points, aggregating the data points, summarizing the data points, averaging the data points, or expressing the data points by a range.

[0054] The first processing device may be identified based on the first processing device's ability to process data associated with a given data magnitude.

[0055] According to a further embodiment of the present invention there is provided a device comprising a processor configured to perform any one of the above methods.

[0056] According to a further embodiment of the present invention there is provided a computing program which, when executed by a processor, causes the processor to perform any one of the methods described above.

[0057] According to a further embodiment of the present invention, there is provided a device comprising a processor configured to: divide surgical data into first surgical data sub-blocks and second surgical data sub-blocks, wherein the first surgical data sub-block is associated with a first resource-time availability of the processing device and the second surgical data sub-block is associated with a second resource-time availability of the processing device; process the first surgical data sub-blocks using a first ML algorithm sub-block in accordance with the first resource-time availability of the processing device; and process the first surgical data sub-blocks using the second ML algorithm sub-block in accordance with the second resource-time availability of the processing device.

[0058] The processor may be further configured to divide the ML algorithm into a first ML algorithm sub-block and a second ML algorithm sub-block.

[0059] The processor may be further configured to divide the ML algorithm into a first ML algorithm sub-block and a second ML algorithm sub-block based on a magnitude or level of processing associated with the first ML algorithm sub-block and the second ML algorithm sub-block.

[0060] Each ML algorithm sub-block can perform the same algorithm steps as the others, but is configured to receive different sets of data to be processed simultaneously. For example, a data set may be divided into multiple data sub-blocks, the size of each sub-block depending on the magnitude or level of processing associated with a given ML algorithm sub-block. ML algorithm sub-blocks associated with higher levels of processing may be assigned larger data sub-blocks than ML sub-blocks associated with lower levels of processing.

[0061] The surgical data may be divided into a first surgical data sub-block and a second surgical data sub-block based on one or more of the timeliness of results, network bandwidth, at least one communication parameter, processing power, risk level, or one of task importance associated with each of the first surgical data sub-block and the second surgical data sub-block, importance level, or availability of other surgical data to be used in place of the first surgical data sub-block or the second surgical data sub-block.

[0062] The processor may be further configured to divide the ML algorithms into a first ML algorithm sub-block and a second ML algorithm sub-block based on one or more of: timeliness of results, network bandwidth, at least one communication parameter, processing power, and one of a risk level.

[0063] In this embodiment, the ML algorithm may be divided into sub-blocks in a similar manner to that described above; that is, each sub-block may perform the same algorithm step, and data may be allocated based on how quickly the ML sub-block returns a result. Alternatively, each ML sub-block may span different algorithm steps, which may be sequential or follow one another. A first ML algorithm sub-block may provide an output for later use in a second ML algorithm sub-block. For example, the first stage of the algorithm may take in high-risk input, and the output of that stage may transform the data into a low-risk one, so that the data can be transmitted more securely over a network. The first stage of the algorithm may form a first ML algorithm sub-block that performs processing steps locally before transmitting over a network to a second ML algorithm sub-block that contains the remaining steps of the algorithm to be executed. Similarly, the output of the first ML algorithm sub-block may require less bandwidth to transmit and therefore may result in an output that is easier to transmit to a remote location for the second ML algorithm sub-block to be executed. There may be stages of an ML algorithm that are more computationally intensive than the rest of the stages, such as convolution or fully connected layers versus pooling or output layers in a convolutional neural network. Such computationally intensive stages may be put into an ML algorithm sub-block separate from other steps that run on high-performance computing hardware, while less intensive steps are kept in sub-blocks that run on devices that are less powerful but have more availability to perform processing tasks.

[0064] The surgical data may be divided based on one or more of the amount of surgical data or the number of variables processed, the frequency or precision level associated with the surgical data.

[0065] The processor may be further configured to divide the ML algorithms into a first ML algorithm sub-block and a second ML algorithm sub-block based on at least one of an algorithm type, an error tolerance of the ML algorithm, a stacking level of the ML algorithm, and a verification or check of the results.

[0066] In this embodiment, the ML algorithm may be an ensemble ML algorithm (such as a stacking ensemble, adaptive boosting, or random forest), and each ML algorithm sub-block may cover one of the base-level algorithms. Because each of the sub-blocks is an algorithm itself, such as a decision tree, a regression model, or a classifier, the output of each sub-block may be different and may have different levels of accuracy, error, or uncertainty associated with it when given the same data as input. The outputs of each sub-block may then be combined in a meta-level ML algorithm trained using the outputs of each sub-block as its input. Each sub-block can be processed and executed independently of the others, and therefore may be independently resource-allocated before their results are combined in the meta-level ML algorithm.

[0067] Systems, methods, and means associated with allometry (e.g., increase and / or decrease) of surgical data as one moves up or down various hierarchical levels may be described herein. A surgical device (e.g., a surgical hub) may receive a plurality of surgical data parameters associated with a first patient. The plurality of surgical data parameters may be of a first data magnitude (e.g., a first data size) and a first data individuality level.

[0068] The surgical device may identify a processing device for processing the plurality of surgical data parameters. The processing device may be identified based on a magnitude of the first data, a first data personality level and / or a characteristic of the processing server, and / or a rule set.

[0069] The surgical device may convert the plurality of surgical data parameters into a transformed plurality of surgical data parameters, where the transformed plurality of surgical data parameters are at a second surgical data individuality level and a second surgical data magnitude. The conversion of the plurality of surgical data parameters may include anonymizing a subset of the plurality of surgical data parameters. The anonymizing may include at least one of editing, randomizing, aggregating, ranging, or averaging. The transformed plurality of surgical data parameters may be transmitted to the identified processing device for processing.

[0070] Systems, methods, and means may be described herein associated with surgical data processing at various system hierarchy levels. A surgical hub / edge server may obtain surgical data associated with a surgical task. The surgical data may include a data size and a data individuality level of data format. The data size may be the extent to which a portion of the surgical data is processed. The data format may be the individuality level of the portion of the surgical data being processed. The surgical hub / edge device may determine a set of parameters associated with a first surgical data sub-block of surgical data and a second surgical sub-block of surgical data. For example, the surgical hub / edge device may determine a first set of parameters associated with the first surgical data sub-block of surgical data and a second set of parameters associated with the second surgical data sub-block of surgical data.

[0071] The surgical hub / edge device may determine a processing level to be used to process each of the first sub-block of surgical data and the second sub-block of surgical data. For example, the surgical hub / edge device may determine a first processing level to be used to process the first surgical data sub-block. The first processing level may be obtained based on a first capability associated with a first processing device located at a first computing hierarchy level of the healthcare provider's network. The surgical hub / edge device may also determine a second processing level to be used to process the second surgical data sub-block. The second processing level may be obtained based on a second capability associated with a second processing device located at a second computing hierarchy level of the healthcare provider's network.

[0072] The surgical hub / edge device may transmit the first surgical data sub-block to the first processing device based on, for example, at least one of a first set of parameters associated with the first surgical data sub-block and a first processing level. The first set of parameters associated with the first surgical data sub-block may include, for example, a size of the first surgical data associated with the first surgical data sub-block, a first data granularity associated with the first surgical data sub-block, and a timeliness of results associated with the first surgical data sub-block.

[0073] The surgical hub / edge device may transmit the second sub-block to the second processing device based on, for example, at least one of a second set of parameters associated with the second surgical data sub-block and the second first processing level. The second set of parameters associated with the second surgical data sub-block may include, for example, a size of the second surgical data associated with the second surgical data sub-block, a second data granularity associated with the second surgical data sub-block, and a timeliness of the results associated with the second surgical data sub-block.

[0074] Systems, methods, and means may be described herein associated with adjusting / scaling at least one surgical data attribute analyzed by a machine learning (ML) algorithm based on a resource-time relationship associated with a computing resource. The resource-time relationship may be determined based on at least one of the following: timeliness of needed results, a level of computational processing associated with the surgical computing device or computational memory associated with the surgical computing device, network bandwidth between the surgical computing device and a location where needed results are to be transmitted, one or more communication parameters, a risk level of functioning without obtaining needed results, a level of importance of the surgical data or surgical task associated with the surgical task, or availability of other data that may be used as a substitute. The communication parameters may include a throughput rate at the surgical computing device or a latency between the surgical computing device and a location where needed results are to be transmitted.

[0075] The at least one surgical data attribute includes a size of the surgical data, a number of surgical data variables, a frequency associated with the surgical data, a precision level associated with the surgical data, an ML algorithm type, a tolerance associated with the ML algorithm, a number of stacking levels associated with the ML algorithm, or a validation or check of the results.

[0076] Scaling or adjusting at least one attribute associated with the ML algorithm may be performed based on a balance of the level of result required, the time associated with the result required, and the availability of computing resources within the time associated with the result required. [Brief explanation of the drawings]

[0077] [Figure 1] FIG. 1 is a block diagram of a computer-implemented surgical system. [Figure 2] 1 illustrates an exemplary surgical system in an operating room. [Figure 3] 1 illustrates exemplary surgical hubs paired with various systems. [Figure 4] Illustrates a surgical data network having a set of communicating surgical hubs configured to connect with a set of sensing systems, an environmental sensing system, a set of devices, and the like. [Figure 5] 1 illustrates a logic diagram of a control system for a surgical tool. [Figure 6] 1 illustrates an exemplary surgical system including a handle having a controller and a motor, an adapter releasably coupled to the handle, and a loading unit releasably coupled to the adapter. [Figure 7A] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7B] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7C] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 7D] 1A-1C show, respectively, an example surgical system information matrix, an example information flow in a surgical system, an example information flow in a surgical system with a surgical robot, and a diagram of surgical information in the context of a procedure. [Figure 8A] 1A and 1B show an exemplary supervised learning framework and an exemplary unsupervised learning framework, respectively. [Figure 8B] 1A and 1B show an exemplary supervised learning framework and an exemplary unsupervised learning framework, respectively. [Figure 9] FIG. 1 is a block diagram of an exemplary surgical system. [Figure 10] Illustrates an example of determining data individuality levels based on the system hierarchy level at which surgical data may be sent for processing. [Figure 11] 1 illustrates an example of a surgical system in which measurements taken within an operating room are received for processing by one or more respective surgical hub / edge devices. [Figure 12] 10 illustrates an example of transformation of surgical data parameters associated with a patient based on data identity and system hierarchy level. [Figure 13] 1 shows an example overview of data transmission to multiple system hierarchy levels. [Figure 14A] 1 shows examples of different system hierarchy levels. [Figure 14B] An example of segmenting a surgical dataset and transmitting the segmented surgical dataset to different system hierarchy levels is shown. [Figure 15] Illustrates compartmentalization of data and / or algorithms. [Figure 16] An example of a surgical hub / edge device / edge device and enterprise cloud server is shown. [Figure 17] 1 shows an example of a flowchart for determining where to process data. [Figure 18] shows an example of a flowchart for dividing an ML algorithm into different sub-blocks to process different parts of a dataset. [Figure 19] 1 shows an example flowchart of compartmentalization of ML algorithm processing of local data. DETAILED DESCRIPTION OF THE INVENTION

[0078] 1 is a block diagram of a computer-implemented surgical system 100. An exemplary surgical system, such as surgical system 100, may include one or more surgical systems (e.g., surgical subsystems) 102, 103, 104. For example, surgical system 102 may include a computer-implemented bidirectional surgical system. For example, surgical systems 102, 103, 104 may include a surgical computing system, such as a surgical hub 106 and / or a computing device 116, that communicates with a cloud computing system 108. Cloud computing system 108 may include a cloud server 109 and a cloud storage unit 110.

[0079] The surgical systems 102, 103, 104 may each be computer-enabled surgical instruments and devices. For example, the surgical systems 102, 103, 104 may include a wearable sensing system 111, a human interface system 112, a robotic system 113, one or more intelligent instruments 114, an environmental sensing system 115, and / or others. The wearable sensing system 111 may include one or more devices used to sense aspects of an individual's condition and activity within the surgical environment. For example, the wearable sensing system 111 may include a healthcare provider sensing system and / or a patient sensing system.

[0080] The human interface system 112 may include devices that allow individuals to interact with the surgical systems 102, 103, 104 and / or the cloud computing system 108. The human interface system 112 may include human interface devices.

[0081] The robotic system 113 may include a surgical robotic device, such as a surgical robot. The robotic system 113 may enable robotic surgical procedures. The robotic system 113 may receive information, settings, programming, control, etc. from the surgical hub 106; for example, the robotic system 113 may transmit data, such as sensor data, feedback information, video information, and operation logs, to the surgical hub 106.

[0082] Environmental sensing system 115 may include, for example, one or more devices used to measure one or more environmental attributes, for example, as further described in Figure 2. Robotic system 113 may include multiple devices used to perform a surgical procedure, for example, as further described in Figure 2.

[0083] The surgical system 102 may communicate with a remote server 109, which may be part of a cloud computing system 108. In one example, the surgical system 102 may communicate with the remote server 109 via a networked connection, such as an Internet connection (e.g., business internet service, T3, cable / FIOS networking node, etc.). The surgical system 102 and / or its components may communicate with the remote server 109 via a cellular transmission / reception point (TRP) or base station using one or more of the following cellular protocols: GSM / GPRS / EDGE (2G), UMTS / HSPA (3G), long term evolution (LTE) or 4G, LTE-Advanced (LTE-A), new radio (NR), or 5G.

[0084] In one example, the surgical hub 106 may facilitate displaying images from a surgical imaging device, such as a laparoscope. The surgical hub 106 may have collaborative interaction with other local systems to facilitate displaying information related to those local systems. The surgical hub 106 may interact with one or more sensing systems 111, 115, one or more intelligent instruments 114, and / or multiple displays. For example, the surgical hub 106 may be configured to collect measurement data from one or more sensing systems 111, 115 and send notification or control messages to one or more sensing systems 111, 115. The surgical hub 106 may send and / or receive information, including notification information, to and / or from a human interface system 112. The human interface system 112 may include one or more human interface devices (HIDs). The surgical hub 106 may send and / or receive notification or control information for audio, display, and / or control information to various devices in communication with the surgical hub.

[0085] For example, the sensing systems 111, 115 may include a wearable sensing system 111 (which may include one or more HCP sensing systems and one or more patient sensing systems) and an environmental sensing system 115. The one or more sensing systems 111, 115 may measure data related to various biomarkers. The one or more sensing systems 111, 115 may measure the biomarkers using one or more sensors, such as optical sensors (e.g., photodiodes, photoresistors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, thermoelectric sensors, infrared sensors, etc. The one or more sensors may measure the biomarkers as described herein using one of many of the following sensing technologies: photoplethysmography, electrocardiography, electroencephalography, colorimetric, impedimentary, potentiometric, amperometric, etc.

[0086] Biomarkers measured by one or more sensing systems 111, 115 may include, but are not limited to, sleep, core body temperature, maximal oxygen consumption, physical activity, alcohol intake, respiratory rate, oxygen saturation, blood pressure, blood glucose, heart rate variability, blood potential of hydrogen, hydration status, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, coughing and sneezing, gastrointestinal motility, gastrointestinal imaging, airway bacteria, edema, mental status, sweat, circulating tumor cells, autonomic tone, circadian rhythm, and / or menstrual cycle.

[0087] The biomarkers may relate to physiological systems, including, but not limited to, behavioral and psychological, cardiovascular, renal, dermatological, nervous, gastrointestinal, respiratory, endocrine, immune, oncological, musculoskeletal, and / or reproductive systems. Information from the biomarkers may be determined and / or used, for example, by the computer-implemented patient and surgical system 100. Information from the biomarkers may be determined and / or used, for example, by the computer-implemented patient and surgical system 100 to improve the system and / or improve patient outcomes. One or more sensing systems 111, 115, biomarkers, and physiological systems are described in more detail in U.S. patent application Ser. No. 17 / 156,287 (Attorney Docket No. END9290USNP1), entitled "METHOD OF ADJUSTING A SURGICAL PARAMETER BASED ON BIOMARKER MEASUREMENTS," filed on January 22, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0088] FIG. 2 shows an example of a surgical system 202 in an operating room. As illustrated in FIG. 2, a patient is being operated on by one or more health care professionals (HCPs). The HCPs are monitored by one or more HCP sensing systems 220 worn by the HCPs. The HCPs and the environment surrounding the HCPs may also be monitored by one or more environmental sensing systems including, for example, a set of cameras 221, a set of microphones 222, and other sensors that may be deployed in the operating room. The HCP sensing systems 220 and the environmental sensing systems may communicate with a surgical hub 206, which may then communicate with one or more cloud servers 209 of a cloud computing system 208, as shown in FIG. 1. The environmental sensing systems may be used to measure one or more environmental attributes, such as HCP position within the operating room, HCP movement, ambient noise within the operating room, temperature / humidity within the operating room, etc.

[0089] As illustrated in FIG. 2 , a primary display 223 and one or more audio output devices (e.g., speaker 219) are positioned in the sterile field for visibility to the operator at the operating table 224. Additionally, a visualization / notification tower 226 is positioned outside the sterile field. The visualization / notification tower 226 may include a first non-sterile human interactive device (HID) 227 and a second non-sterile HID 229 facing opposite each other. The HIDs may be displays or displays with touchscreens that allow a human to directly interface with the HIDs. A human interface system guided by the surgical hub 206 may be configured to coordinate information flow to operators inside and outside the sterile field using the HIDs 227, 229, and 223. In one example, the surgical hub 206 may cause an HID (e.g., primary HID 223) to display notifications and / or information regarding the patient and / or surgical procedure steps. In one example, the surgical hub 206 may prompt and / or receive input from a person in the sterile field or non-sterile area. In one example, the surgical hub 206 may cause the HID to display a snapshot of the surgical site as recorded by the imaging device 230 on the non-sterile HID 227 or 229 while maintaining a live view of the surgical site on the main HID 223. The snapshot on the non-sterile display 227 or 229 may, for example, allow a non-sterile operator to perform diagnostic steps related to the surgical procedure.

[0090] In one aspect, the surgical hub 206 may be configured to send diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 to a primary display 223 in the sterile field for viewing by a sterile operator at the operating table. In one example, the input may be in the form of a correction to a snapshot displayed on the non-sterile display 227 or 229, which may be sent by the surgical hub 206 to the primary display 223.

[0091] 2, a surgical instrument 231 is used as part of a surgical system 202 in a surgical procedure. A hub 206 may be configured to coordinate information flow to the display of the surgical instrument 231, as described, for example, in U.S. Patent Application Publication No. 2019-0200844(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 can be sent by the hub 206 to a surgical instrument display in the sterile field, where it can be viewed by the operator of the surgical instrument 231. Exemplary surgical instruments suitable for use with surgical system 202 are described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety.

[0092] FIG. 2 illustrates an example of a surgical system 202 being used to perform a surgical procedure on a patient lying on an operating table 224 in an operating room 235. A robotic system 234 may be used as part of the surgical system 202 in the surgical procedure. The robotic system 234 may include a surgeon's console 236, a patient side cart 232 (surgical robot), and a surgical robot hub 233. The patient side cart 232 can manipulate at least one detachably coupled surgical tool 237 through a minimally invasive incision within the patient's body while the surgeon views the surgical site through the surgeon's console 236. Images of the surgical site can be acquired by a medical imaging device 230, which can be manipulated by the patient side cart 232 to orient the imaging device 230. The robotic hub 233 can be used to process the images of the surgical site and then display them to the surgeon through the surgeon's console 236.

[0093] Other types of robotic systems can be readily adapted for use with surgical system 202. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0201137(A1), entitled "METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety.

[0094] Various examples of cloud-based analytics methods that may be performed by cloud computing system 208 and that are suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0206569(A1), entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,403), the disclosure of which is incorporated herein by reference in its entirety.

[0095] In various embodiments, the imaging device 230 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, charge-coupled device (CCD) sensors and complementary metal-oxide semiconductor (CMOS) sensors.

[0096] The optics of the imaging device 230 may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate a portion of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0097] The one or more illumination sources can be configured to emit electromagnetic energy in the visible spectrum as well as the invisible spectrum. The visible spectrum, sometimes referred to as the optical spectrum or luminous spectrum, is the portion of the electromagnetic spectrum that is visible to (i.e., detectable by) the human eye and is sometimes referred to as visible light or simply light. The human eye typically responds to wavelengths in air between about 380 nm and about 750 nm.

[0098] The invisible spectrum (e.g., non-radiative spectrum) is the portion of the electromagnetic spectrum located below and above the visible spectrum (i.e., wavelengths less than about 380 nm and greater than about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, which constitutes invisible infrared (IR), microwave, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, which constitutes invisible ultraviolet, x-ray, and gamma-ray electromagnetic radiation.

[0099] In various aspects, imaging device 230 is configured for use in minimally invasive procedures. Examples of imaging devices suitable for use with the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, cholangioscopes, colonoscopes, cytoscopes, duodenoscopes, enteroscopes, esophagogastroduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngo-neproscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0100] The imaging device may use multispectral monitoring to distinguish between topography and underlying structures. Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. Wavelengths can be separated by filters or by using instruments sensitive to specific wavelengths, including frequencies beyond the visible light range, e.g., IR and UV light. Spectral imaging allows for the extraction of additional information that cannot be captured by the red, green, and blue receptors of the human eye. The use of multispectral imaging is described in detail under the heading "Advanced Imaging Acquisition Module" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Multispectral monitoring can be a useful tool for repositioning the surgical field after the surgical task is complete to perform one or more of the tests described above on the treated tissue. It is self-evident that strict sterilization of the operating room and surgical equipment is necessary during any surgical procedure. The strict hygienic and sterilization conditions required in an "operating room," i.e., an operating room or procedure room, require the highest possible sterility of all medical devices and equipment. Part of the sterilization process is the need to sterilize everything that comes into contact with the patient or enters the sterile field, including the imaging device 230 and its accessories and components. It will be understood that the sterile field may be considered a specific area, such as in a tray or on a sterile towel, that is deemed free of microorganisms, or the sterile field may be considered the area immediately surrounding the patient being prepared for the surgical procedure. The sterile field may include properly clothed and hand-washed team members, as well as all equipment and fixtures in the area.

[0101] The wearable sensing system 211 illustrated in FIG. 1 may include one or more sensing systems, such as an HCP sensing system 220 as shown in FIG. 2. The HCP sensing system 220 may include a sensing system that monitors and detects a set of physical and / or physiological conditions of a healthcare professional (HCP). The HCP may generally be a surgeon or one or more healthcare professionals assisting the surgeon or other healthcare provider. In one example, the sensing system 220 may measure a set of biomarkers to monitor the HCP's heart rate. In one example, the sensing system 220 worn on the surgeon's wrist (e.g., a watch or wristband) may use an accelerometer to detect hand movement and / or shaking and determine tremor magnitude and frequency. The sensing system 220 may transmit measurement data associated with the set of biomarkers and data associated with the surgeon's physical condition to the surgical hub 206 for further processing. One or more environmental sensing devices may transmit environmental information to the surgical hub 206. For example, the environmental sensing devices may include a camera 221 for detecting the HCP's hand / body position. The environmental sensing devices may include a microphone 222 for measuring ambient noise within the operating room. Other environmental sensing devices may include devices such as a thermometer for measuring temperature and a hygrometer for measuring ambient humidity within the operating room. The surgical hub 206, alone or in communication with a cloud computing system, may use the surgeon biomarker measurement data and / or environmental sensing information to modify the control algorithms of handheld instruments or the average latency of a robotic interface, for example, to minimize tremor. In one example, the HCP sensing system 220 may measure one or more surgeon biomarkers associated with the HCP and transmit measurement data associated with the surgeon biomarkers to the surgical hub 206.The HCP sensing system 220 may use one or more of RF protocols such as Bluetooth, Bluetooth Low-Energy (BLE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low Power Wireless Personal Area Network (6LoWPAN), Wi-Fi, etc. to communicate with the surgical hub 20006. The surgeon biomarkers may include one or more of stress, heart rate, etc. The environmental measurements from the operating room may include ambient noise levels associated with the surgeon or patient, surgeon and / or staff movement, surgeon and / or staff attention level, etc.

[0102] The surgical hub 206 may use the surgeon biomarker measurement data associated with the HCP to adaptively control one or more surgical instruments 231. For example, the surgical hub 206 may send a control program to the surgical instrument 231 to control its actuators to limit or compensate for fatigue and the use of fine motor skills. The surgical hub 206 may send the control program based on situational awareness and / or context regarding the importance or criticality of the task. The control program may instruct the instrument to change operation to provide more control when control is needed.

[0103] FIG. 3 illustrates an exemplary surgical system 302 having a surgical hub 306. The surgical hub 306 may be paired with a wearable sensing system 311, an environmental sensing system 315, a human interface system 312, a robotic system 313, and an intelligent instrument 314 via a modular control. The hub 306 includes a display 348, an imaging module 349, a generator module 350, a communications module 356, a processor module 357, a storage array 358, and an operating room mapping module 359. In certain embodiments, as illustrated in FIG. 3, the hub 306 further includes a smoke evacuation module 354 and / or a suction / irrigation module 355. The various modules and systems may be connected to the modular control either directly via a router or via the communications module 356. Operating room devices may be coupled to cloud computing resources and data storage via the modular control. The human interface system 312 may include a display subsystem and a notification subsystem.

[0104] The modular controller may be coupled to a non-contact sensor module. The non-contact sensor module may use ultrasonic, laser-type, and / or similar non-contact measurement devices to measure the dimensions of the operating room and generate a map of the operating room. Other distance sensors may be used to determine the boundaries of the operating room. As described under the heading "Surgical Hub Spatial Awareness Within an Operating Room" in U.S. Provisional Patent Application No. 62 / 611,341, filed December 28, 2017, entitled "INTERACTIVE SURGICAL PLATFORM," which is incorporated herein by reference in its entirety, an ultrasound-based non-contact sensor module may scan the operating room by transmitting bursts of ultrasound and receiving echoes as the bursts reflect off the exterior walls of the operating room. The sensor module may be configured to determine the size of the operating room and adjust Bluetooth pairing distance limits. For example, a laser-based non-contact sensor module can scan an operating room by transmitting laser light pulses, receiving the laser light pulses that reflect off the exterior walls of the operating room, and comparing the phase of the transmitted pulses with the received pulses to determine the size of the operating room and adjust the Bluetooth pairing distance limit.

[0105] During surgical procedures, the application of energy to tissue for sealing and / or cutting is generally associated with smoke evacuation, aspiration of excess fluid, and / or irrigation of tissue. Fluid, power, and / or data lines from different sources often become tangled during surgical procedures. Addressing this issue can result in valuable time being lost during a surgical procedure. Untangling the lines may require unplugging them from their corresponding modules, which may require resetting the modules. The hub modular enclosure 360 ​​reduces the frequency of such line tangles by providing a unified environment for managing power, data, and fluid lines. Aspects of the present disclosure present a surgical hub 306 for use in surgical procedures involving the application of energy to tissue at a surgical site.

[0106] The surgical hub 306 includes a hub enclosure 360 ​​and a combination generator module slidably received within a docking station of the hub enclosure 360. The docking station includes data and power contacts. The combination generator module includes two or more of an ultrasonic energy generator component, a bipolar RF energy generator component, and a monopolar RF energy generator component housed within a single unit. In one aspect, the combination generator module also includes a smoke evacuation component, at least one energy delivery cable for connecting the combination generator module to a surgical instrument, at least one smoke evacuation component configured to evacuate smoke, fluid, and / or particulates generated by the application of therapeutic energy to tissue, and a fluid line extending from the remote surgical site to the smoke evacuation component. In one aspect, the fluid line may be a first fluid line and a second fluid line may extend from the remote surgical site to the suction and irrigation module 355, which is slidably received within the hub enclosure 360. In one aspect, the hub enclosure 360 ​​may include a fluid interface.

[0107] Certain surgical procedures may require the application of two or more energy types to tissue. One energy type may be more beneficial for cutting tissue, while another, different energy type may be more beneficial for sealing tissue. For example, a bipolar generator may be used to seal tissue, while an ultrasonic generator may be used to cut the sealed tissue. Aspects of the present disclosure present a solution in which a hub modular enclosure 360 ​​is configured to house different generators and facilitate bidirectional communication between them. The hub modular enclosure 360 ​​may allow for rapid removal and / or replacement of various modules. Aspects of the present disclosure present a modular surgical enclosure for use in surgical procedures involving the application of energy to tissue. The modular surgical enclosure includes a first energy generator module configured to generate a first energy for application to tissue and a first docking station including a first docking port including first data contacts and first power contacts, wherein the first energy generator module is slidably movable into electrical engagement with the power contacts and the data contacts, and the first energy generator module is slidably movable out of electrical engagement with the first power contacts and the first data contacts. In addition to the above, the modular surgical enclosure also includes a second energy generator module configured to generate a second energy different from the first energy for application to tissue and a second docking station including a second docking port including second data contacts and second power contacts, wherein the second energy generator module is slidably movable into electrical engagement with the power contacts and the data contacts, and the second energy generator module is slidably movable out of electrical engagement with the second power contacts and the second data contacts. In addition, the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port configured to facilitate communication between the first energy generator module and the second energy generator module.Referring to FIG. 3 , an aspect of the disclosure is presented regarding a hub modular enclosure 360 ​​that allows for modular integration of a generator module 350, a smoke evacuation module 354, and a suction / irrigation module 355. The hub modular enclosure 360 ​​further facilitates bidirectional communication between the modules 359, 354, and 355. The generator module 350 can comprise integrated monopolar, bipolar, and ultrasonic components supported within a single housing unit that is slidably insertable into the hub modular enclosure 360. The generator module 350 can be configured to connect to a monopolar device 351, a bipolar device 352, and an ultrasonic device 353. Alternatively, the generator module 350 may comprise a series of monopolar, bipolar, and / or ultrasonic generator modules that interact through the hub modular enclosure 360. The hub modular enclosure 360 ​​may be configured to facilitate the insertion of multiple generators and bidirectional communication between generators docked to the hub modular enclosure 360 ​​so that the multiple generators act as a single generator.

[0108] FIG. 4 illustrates a surgical data network having a set of communication hubs configured to connect a set of sensing systems, environmental sensing systems, and other modular devices located in one or more operating rooms, patient recovery rooms, or rooms within a medical facility specially equipped for surgery to a cloud in accordance with at least one embodiment of the present disclosure.

[0109] 4, surgical hub system 460 may include a modular communications hub 465 configured to connect modular devices located within a medical facility to a cloud-based system (e.g., a cloud computing system 464, which may include a remote server 467 coupled to remote storage 468). The modular communications hub 465 and devices may be connected in a room within the medical facility specially equipped for surgical procedures. In one aspect, modular communications hub 465 may include a network hub 461 and / or a network switch 462 in communication with a network router 466. Modular communications hub 465 may be coupled to a local computer system 463 to provide local computer processing and data manipulation.

[0110] The computer system 463 may include a processor and a network interface. The processor may be coupled to a communication module, storage, memory, non-volatile memory, and input / output (I / O) interfaces via a system bus. The system bus can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures, including, but not limited to, a 9-bit bus, an Industrial Standard Architecture (ISA), a Micro-Channel Architecture (MSA), an Extended ISA (EISA), an Intelligent Drive Electronics (IDE), a VESA Local Bus (VLB), a Peripheral Component Interconnect (PCI), a USB, an Advanced Graphics Port (AGP), a Personal Computer Memory Card International Association (PCMCIA), a Small Computer Systems Interface (SCSI), or any other proprietary bus.

[0111] The processor may be any single-core or multi-core processor, such as that known by Texas Instruments under the trade name ARM Cortex. In one aspect, the processor may be, for example, an LM4F230H5QR ARM Cortex-M4F processor core available from Texas Instruments, including on-chip memory of 256 KB of single-cycle flash memory or other non-volatile memory up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, 32 KB of single-cycle serial random access memory (SRAM), internal read-only memory (ROM) loaded with StellarisWare® software, 2 KB of electrically erasable programmable read-only memory (EEPROM), and / or one or more pulse width modulation (PWM) modules, one or more Quadrature Encoder Input (QEI) analog, one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, details of which are available in the product data sheet.

[0112] In one example, the processor may include a safety controller, which includes two controller-based families such as the TMS570 and RM4x, also known under the trade name Hercules ARM Cortex R4, manufactured by Texas Instruments. The safety controller may be specifically configured for IEC 61508 and ISO 26262 safety limit applications, among others, to provide advanced integrated safety mechanisms while offering scalable performance, connectivity, and memory options.

[0113] It should be understood that the computer system 463, in a suitable operating environment, may include software that acts as an intermediary between the described users and the basic computer resources. Such software may include an operating system. The operating system, which may be stored on disk storage, may function to control and allocate resources of the computer system. System applications may take advantage of resource management by the operating system through program modules and program data stored either in system memory or on disk storage. It should be understood that the various components described herein may be implemented with various operating systems or combinations of operating systems.

[0114] A user may input commands or information into the computer system 463 through input devices coupled to the I / O interface. Input devices may include, but are not limited to, pointing devices such as a mouse, trackball, stylus, or touchpad; keyboards; microphones; joysticks; gamepads; satellite dishes; scanners; TV tuner cards; digital cameras; digital video cameras; webcams; and the like. These and other input devices connect to the processor through the system bus via interface ports. Interface ports include, for example, serial ports, parallel ports, game ports, and USB. Output devices use some of the same types of ports as input devices. Thus, for example, a USB port may be used to provide input to the computer system 463 and to output information from the computer system 463 to an output device. Output adapters may be provided to illustrate that there may be several output devices, such as monitors, displays, speakers, and printers, among other output devices that may require special adapters. Output adapters may include, by way of example and not limitation, video and sound cards that provide a means of connection between the output device and the system bus. It should be noted that other devices and / or systems of devices, such as remote computers, may provide both input and output capabilities.

[0115] The computer system 463 may operate in a networked environment using logical connections to one or more remote computers, such as a cloud computer, or a local computer. The remote cloud computer may be a personal computer, a server, a router, a network PC, a workstation, a microprocessor-based appliance, a peer device, or other common network node, but typically includes many or all of the elements described with respect to a computer system. For simplicity, only a memory storage device is illustrated with the remote computer. The remote computer may be logically connected to the computer system through a network interface and subsequently physically connected via a communication connection. The network interface may encompass communication networks such as a local area network (LAN) and a wide area network (WAN). LAN technologies may include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet / IEEE 802.3, Token Ring / IEEE 802.5, etc. WAN technologies may include, but are not limited to, point-to-point links, circuit-switched networks such as Integrated Services Digital Networks (ISDN) and its variants, packet-switched networks, and Digital Subscriber Lines (DSL).

[0116] In various examples, computer system 463 may include an image processor, image processing engine, media processor, or any specialized digital signal processor (DSP) used to process digital images. The image processor may use parallel computing using Single Instruction, Multiple Data (SIMD) or Multiple Instruction, Multiple Data (MIMD) techniques to increase speed and efficiency. The digital image processing engine may perform a variety of tasks. The image processor may be a system on a chip with a multi-core processor architecture.

[0117] The communications connection may refer to the hardware / software used to connect the network interface to the bus. The communications connection is shown internal to computer system 463 for clarity of illustration, but can also be external to computer system 463. For purposes of illustration only, the hardware / software required to connect to the network interface may include internal and external technologies such as modems, including regular telephone-grade modems, cable modems, fiber optic modems, and DSL modems, ISDN adapters, and Ethernet cards. In some examples, the network interface may also be provided using an RF interface.

[0118] The surgical data network associated with the surgical hub system 460 may be configured as passive, intelligent, or switched. A passive surgical data network acts as a conduit for data, allowing it to go from one device (or segment) to another and to cloud computing resources. An intelligent surgical data network includes additional features that allow traffic to pass through the monitored surgical data network and configure each port within the network hub 461 or network switch 462. An intelligent surgical data network may be referred to as a manageable hub or switch. A switching hub reads the destination address of each packet and then forwards the packet to the correct port.

[0119] Modular devices 1a-1n located in an operating room may be coupled to a modular communication hub 465. Network hub 461 and / or network switch 462 may be coupled to a network router 466 to connect devices 1a-1n to a cloud computing system 464 or a local computer system 463. Data associated with devices 1a-1n may be transferred to a cloud-based computer via the router for remote data processing and manipulation. Data associated with devices 1a-1n may also be transferred to local computer system 463 for local data processing and manipulation. Modular devices 2a-2m located in the same operating room may also be coupled to network switch 462. Network switch 462 may be coupled to network hub 461 and / or to network router 466 to connect devices 2a-2m to the cloud 464. Data associated with devices 2a-2m may be transferred to cloud computing system 464 via network router 466 for data processing and manipulation. Data associated with the devices 2a-2m may also be transferred to a local computer system 463 for local data processing and manipulation.

[0120] 4, a computing system such as surgical hub system 460 may include a modular communications hub 465 configured to connect modular devices (e.g., surgical devices) located within a medical facility to a cloud-based system (e.g., a cloud computing system 464 that may include a remote server 467 coupled to remote storage 468). The modular communications hub 465 and devices may be connected in a room within the medical facility specially equipped for surgical procedures. In one aspect, the modular communications hub 465 may include a network hub 461 and / or a network switch 462 in communication with a network router 466. The modular communications hub 465 may be coupled to a local computer system (e.g., a computing device) to provide local computer processing and data manipulation.

[0121] FIG. 5 illustrates a logic diagram of a surgical instrument or surgical tool control system 520 according to one or more embodiments of the present disclosure. The surgical instrument or surgical tool may be configurable. The surgical instrument may include surgical fasteners specific to the procedure at hand, such as imaging devices, surgical staplers, energy devices, endocutter devices, etc. For example, the surgical instrument may include any of a power stapler, a power stapler generator, an energy device, an advanced energy device, an advanced energy jaw device, an endocutter clamp, an energy device generator, an operating room imaging system, a smoke evacuation device, a suction irrigation device, an insufflation system, etc. The system 520 may include control circuitry. The control circuitry may include a microcontroller 521 with a processor 522 and a memory 523. For example, one or more of sensors 525, 526, 527 provide real-time feedback to the processor 522. A motor 530 driven by a motor driver 529 operably couples a longitudinally movable displacement member to drive the I-beam knife element. The tracking system 528 may be configured to determine the position of the longitudinally movable displacement member. The position information may be provided to the processor 522, which may be programmed or configured to determine the position of the longitudinally movable drive member, as well as the positions of the firing member, firing bar, and I-beam knife element. Additional motors may be provided to the tool driver interface to control the firing of the I-beam, the movement of the obturator tube, the rotation of the shaft, and the articulation. The display 524 may display various operating states of the instrument and may also include touch screen functionality for data entry. Information displayed on the display 524 may be overlaid with images acquired via the endoscopic imaging module.

[0122] Microcontroller 521 may be any single-core or multi-core processor, such as those known under the trade name ARM Cortex manufactured by Texas Instruments. In one aspect, main microcontroller 521 may be, for example, an LM4F230H5QR ARM Cortex-M4F Processor Core available from Texas Instruments, with on-chip memory of 256 KB of single-cycle flash memory or other non-volatile memory up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, 32 KB of single-cycle SRAM, internal ROM with StellarisWare® software, 2 KB of EEPROM, one or more PWM modules, one or more QEI analog, and / or one or more 12-bit ADCs with 12 analog input channels, details of which are available in the product datasheet.

[0123] The microcontroller 521 may include a safety controller, which includes two controller-based families such as the TMS570 and RM4x, also known under the trade name Hercules ARM Cortex R4, manufactured by Texas Instruments. The safety controller can be specifically configured for IEC 61508 and ISO 26262 safety limit applications, among others, to provide advanced integrated safety mechanisms while offering scalable performance, connectivity, and memory options.

[0124] The microcontroller 521 may be programmed to perform various functions, such as precise control over the speed and position of the knife and articulation system. In one embodiment, the microcontroller 521 may include a processor 522 and memory 523. The electric motor 530 may be a brushed direct current (DC) motor with a gearbox and mechanical linkage to the articulation or knife system. In one embodiment, the motor driver 529 may be an A3941 available from Allegro Microsystems, Inc. Other motor drivers may be easily substituted for use in the tracking system 528 with an absolute positioning system. A detailed description of the absolute positioning system is provided in U.S. Patent Application Publication No. 2017 / 0296213, entitled "SYSTEMS AND METHODS FOR CONTROLLING A SURGICAL STAPLING AND CUTTING INSTRUMENT," published October 19, 2017, which is incorporated herein by reference in its entirety.

[0125] The microcontroller 521 may be programmed to provide precise control over the velocity and position of the displacement members and articulation system. The microcontroller 521 may be configured to calculate a response in the microcontroller 521 software. The calculated response may be compared to the measured response of the actual system to obtain an "observed" response, which is used for actual feedback decision-making. The observed response may be a suitably adjusted value that balances the smooth, continuous nature of the simulated response with the measured response, which can detect external influences on the system.

[0126] The motor 530 may be controlled by a motor driver 529 and may be used by the surgical instrument or tool firing system. In various forms, the motor 530 may be a brushed DC drive motor having a maximum rotational speed of approximately 25,000 RPM. In some examples, the motor 530 may include a brushless motor, a cordless motor, a synchronous motor, a stepper motor, or any other suitable electric motor. The motor driver 529 may include, for example, an H-bridge driver including field-effect transistors (FETs). The motor 530 may be powered by a power supply assembly releasably attached to the handle assembly or tool housing to provide control power to the surgical instrument or tool. The power supply assembly may include a battery, which may include multiple battery cells connected in series, that may be used as a power source to power the surgical instrument or tool. Under certain circumstances, the battery cells of the power supply assembly may be replaceable and / or rechargeable. In at least one example, the battery cells may be lithium-ion batteries, which may be connectable to and detachable from the power supply assembly.

[0127] The motor driver 529 may be the A3941, available from Allegro Microsystems, Inc. The A3941 may be a full-bridge controller for use with external N-channel power metal-oxide semiconductor field-effect transistors (MOSFETs) specifically designed for inductive loads, such as brushed DC motors. The driver 529 may include an intrinsic charge pump regulator, which provides full (>10V) gate drive for battery voltages up to 7V, allowing the A3941 to operate with reduced gate drive down to 5.5V. A bootstrap capacitor may be used to provide the required battery supply voltage above the N-channel MOSFET. An internal charge pump for the high-side drive allows DC (100% duty cycle) operation. The full-bridge may be driven in fast or slow decay mode using diode or synchronous rectification. In slow decay mode, current recirculation is possible through either the high-side or low-side FET. The power FETs may be protected from shoot-through by a resistor-adjustable dead time. Integrated diagnostics indicate undervoltage, overtemperature, and power bridge faults and can be configured to protect the power MOSFETs under most short circuit conditions. Other motor drivers may be easily substituted for use in tracking system 528 with an absolute positioning system.

[0128] The tracking system 528 may include controlled motor drive circuitry including a position sensor 525 according to one aspect of the present disclosure. The position sensor 525 for the absolute positioning system may provide a unique position signal corresponding to the position of the displacement member. In some examples, the displacement member may represent a longitudinally movable drive member including a rack of drive teeth for meshing engagement with a corresponding drive gear of a gear reducer assembly. In some examples, the displacement member may represent a firing member that may be adapted and configured to include a rack of drive teeth. In some examples, the displacement member may represent a firing bar or an I-beam, each of which may be adapted and configured to include a rack of drive teeth. Thus, as used herein, the term displacement member may be used generally to refer to any movable member of a surgical instrument or tool, such as a drive member, firing member, firing bar, I-beam, or any element that can be displaced. In one aspect, a longitudinally movable drive member may be coupled to a firing member, firing bar, and I-beam. Thus, the absolute positioning system may actually track the linear displacement of the I-beam by tracking the linear displacement of the longitudinally movable drive member. In various aspects, the displacement member may be coupled to any suitable position sensor 525 for measuring linear displacement. Thus, the longitudinally movable drive member, firing member, firing bar, or I-beam, or combinations thereof, may be coupled to any suitable linear displacement sensor. The linear displacement sensor may include a contact displacement sensor or a non-contact displacement sensor.The linear displacement sensor may include a linear variable differential transformer (LVDT), a differential variable reluctance transducer (DVRT), a slide potentiometer, a magnetic sensing system with a movable magnet and a series of linearly arranged Hall effect sensors, a magnetic sensing system with a fixed magnet and a series of movable linearly arranged Hall effect sensors, an optical sensing system with a movable light source and a series of linearly arranged photodiodes or photodetectors, an optical sensing system with a fixed light source and a series of movable linearly arranged photodiodes or photodetectors, or any combination thereof.

[0129] The electric motor 530 may include a rotatable shaft operably interfaced with a gear assembly mounted for meshing engagement with a set of drive teeth, or rack, on the displacement member. The sensor element may be operably coupled to the gear assembly such that one rotation of the position sensor 525 element corresponds to several linear longitudinal translations of the displacement member. The gearing and sensor arrangement may be connected to a linear actuator by a rack-and-pinion arrangement or to a rotary actuator by a spur gear or other connection. A power source may provide power to the absolute positioning system, and an output indicator may display the output of the absolute positioning system. The displacement member may represent a longitudinally movable drive member with a rack of drive teeth formed thereon for meshing engagement with a corresponding drive gear of a gear reducer assembly. The displacement member may represent a longitudinally movable firing member, a firing bar, an I-beam, or a combination thereof.

[0130] One revolution of the sensor element associated with the position sensor 525 may correspond to a linear longitudinal displacement d1 of the displacement member, where d1 is the linear longitudinal distance the displacement member travels from point "a" to point "b" after one revolution of the sensor element coupled to the displacement member. The sensor arrangement may be connected via a gear reduction such that the position sensor 525 completes one or more revolutions for a full stroke of the displacement member. The position sensor 525 may complete multiple revolutions for a full stroke of the displacement member.

[0131] A series of switches (where n is an integer greater than 1) may be used alone or in combination with a gear reduction to provide a unique position signal for two or more revolutions of the position sensor 525. The state of the switches may be fed back to the microcontroller 521, which applies logic to determine a unique position signal corresponding to the longitudinal linear displacement d1+d2+...dn of the displacement member. The output of the position sensor 525 is provided to the microcontroller 521. The position sensor 525 of the sensor arrangement may comprise a magnetic sensor, an analog rotary sensor such as a potentiometer, or an array of analog Hall effect elements that output a unique combination of position signals or values.

[0132] The position sensor 525 may comprise any number of magnetic sensing elements, such as magnetic sensors classified according to whether they measure the total magnetic field or vector components of the magnetic field. The technologies used to produce both types of magnetic sensors may encompass many aspects of physics and electronics. Technologies used to sense magnetic fields may include search coils, fluxgates, optical pumping, nuclear precession, SQUIDs, Hall effect, anisotropic magnetoresistance, giant magnetoresistance, magnetic tunnel junctions, giant magnetoimpedance, magnetostrictive / piezoelectric composites, magnetodiodes, magnetotransistors, optical fiber, magneto-optical, and microelectromechanical systems-based magnetic sensors, among others.

[0133] The position sensor 525 of the tracking system 528 with an absolute positioning system may comprise a magnetic rotary absolute positioning system. The position sensor 525 may be implemented as an AS5055EQFT single-chip magnetic rotary position sensor available from Austria Microsystems, AG. The position sensor 525 interfaces with the microcontroller 521 to provide the absolute positioning system. The position sensor 525 may be a low-voltage, low-power component and may include four Hall-effect elements in an area of ​​the position sensor 525 that may be located above the magnet. A high-resolution ADC and a smart power management controller may also be provided on-chip. A coordinate rotation digital computer (CORDIC) processor, also known as the digit-by-digit method and the Boulder algorithm, may be provided to implement simple, efficient algorithms for calculating hyperbolic and trigonometric functions, requiring only addition, subtraction, bit shifting, and table lookup operations. The angular position, alarm bits, and magnetic field information may be transmitted to the microcontroller 521 through a standard serial communications interface, such as a serial peripheral interface (SPI) interface. The position sensor 525 may provide 12-bit or 14-bit resolution and may be an AS5055 chip provided in a small QFN 16-pin 4x4x0.85mm package.

[0134] The tracking system 528, which comprises an absolute positioning system, may include and / or be programmed to implement a feedback controller, such as a PID, state feedback, and adaptive controller. The power supply converts the signal from the feedback controller into a physical input to the system, in this case a voltage. Other examples include PWM of voltage, current, and force. In addition to the position measured by the position sensor 525, other sensors may be provided to measure physical parameters of the physical system. In some embodiments, other sensors may include sensor arrangements such as those described in U.S. Pat. No. 9,345,481, issued May 24, 2016, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety; U.S. Patent Application Publication No. 2014 / 0263552, published September 18, 2014, entitled "STAPLE CARTRIDGE TISSUE THICKNESS SENSOR SYSTEM," which is incorporated herein by reference in its entirety; and U.S. Patent Application No. 15 / 628,175, filed June 20, 2017, entitled "TECHNIQUES FOR ADAPTIVE CONTROL OF MOTOR VELOCITY OF A SURGICAL STAPLING AND CUTTING INSTRUMENT," which is incorporated herein by reference in its entirety. In a digital signal processing system, the absolute positioning system is coupled to a digital data acquisition system, where the output of the absolute positioning system has a finite resolution and sampling frequency. The absolute positioning system may include comparison and combination circuitry to combine the calculated response with the measured response using algorithms such as weighted averages and theoretical control loops that drive the calculated response towards the measured response. The calculated response of the physical system may take into account properties such as mass, inertia, viscous friction, induced drag, etc., in order to predict what the state and output of the physical system will be given knowledge of the input.

[0135] The absolute positioning system may provide the absolute position of the displacement member upon powering up of the instrument without forcing the displacement member to retract or advance to a reset (zero or home) position, as may be required with conventional rotary encoders that simply count the number of forward or backward steps taken by the motor 530 to infer the position of the device actuator, drive bar, knife, etc.

[0136] Sensor 526, such as a strain gauge or micro-strain gauge, may be configured to measure one or more parameters of the end effector, such as the amplitude of strain exerted on the anvil during clamping, which can be indicative of the closure force applied to the anvil. The measured strain may be converted to a digital signal and provided to processor 522. Alternatively, or in addition to sensor 526, sensor 527, such as a load sensor, can measure the closure force applied to the anvil by the closure drive system. For example, sensor 527, such as a load sensor, can measure the firing force applied to the I-beam during the firing stroke of the surgical instrument or tool. The I-beam is configured to engage a wedge-shaped sled, which is configured to cam the staple driver upward and drive the staples into deforming contact with the anvil. The I-beam may also include a sharp cutting edge that can be used to cut tissue as the I-beam is advanced distally by the firing bar. Alternatively, a current sensor 531 can be used to measure the current drawn by motor 530. The force required to advance the firing member may correspond, for example, to the current drawn by motor 530. The measured force may be converted to a digital signal and provided to processor 522.

[0137] For example, a strain gauge sensor 526 can be used to measure the force applied to tissue by the end effector. A strain gauge can be coupled to the end effector to measure the force applied by the end effector to the tissue being treated. A system for measuring the force applied to grasped tissue by the end effector may include a strain gauge sensor 526, such as, for example, a micro-strain gauge, which can be configured to measure one or more parameters of the end effector. In one aspect, the strain gauge sensor 526 can measure the amplitude or magnitude of strain exerted on the jaw members of the end effector during clamping, which can indicate tissue compression. The measured strain can be converted to a digital signal and provided to the processor 522 of the microcontroller 521. A load sensor 527 can measure the force used to operate the knife element, for example, to cut tissue captured between the anvil and the staple cartridge. A magnetic field sensor can be used to measure the thickness of the captured tissue. The magnetic field sensor's measurements can also be converted to a digital signal and provided to the processor 522.

[0138] Measurements of tissue compression, tissue thickness, and / or force required to close the end effector on the tissue, measured by sensors 526, 527, respectively, may be used by microcontroller 521 to characterize a selected position of the firing member and / or a corresponding value of firing member velocity. In one example, memory 523 may store techniques, equations, and / or look-up tables that may be used by microcontroller 521 during the evaluation.

[0139] The surgical instrument or tool control system 520 may also include wired or wireless communication circuitry for communicating with a surgical hub, such as surgical hub 460, as shown in FIG.

[0140] FIG. 6 illustrates an exemplary surgical system 680 according to the present disclosure and may include a surgical instrument 682 that can communicate with a console 694 or a portable device 696 through a local area network 692 and / or a cloud network 693 via wired and / or wireless connections. The console 694 and the portable device 696 may be any suitable computing devices. The surgical instrument 682 may include a handle 697, an adapter 685, and a loading unit 687. The adapter 685 releasably couples to the handle 697, and the loading unit 687 releasably couples to the adapter 685 such that the adapter 685 transmits force from the drive shaft to the loading unit 687. The adapter 685 or the loading unit 687 may include a force gauge (not explicitly shown) disposed therein to measure force applied to the loading unit 687. The loading unit 687 may include an end effector 689 having a first jaw 691 and a second jaw 690. The loading unit 687 may be an in vivo loading or multi-firing loading unit (MFLU) that allows a clinician to fire multiple fasteners multiple times without having to remove the loading unit 687 from the surgical site to reload it.

[0141] The first jaw 691 and the second jaw 690 may be configured to clamp tissue therebetween, fire fasteners through the clamped tissue, and sever the clamped tissue. The first jaw 691 may be configured to fire at least one fastener multiple times, or may be configured to contain a replaceable multi-fire fastener cartridge containing multiple fasteners (e.g., staples, clips, etc.) that may be fired two or more times before being replaced. The second jaw 690 may include an anvil that deforms or otherwise secures fasteners as they are ejected from the multi-fire fastener cartridge.

[0142] The handle 697 may include a motor coupled to the drive shaft to affect rotation of the drive shaft. The handle 697 may include a control interface for selectively activating the motor. The control interface may include buttons, switches, levers, sliders, a touch screen, and any other suitable input mechanism or user interface that may be engaged by a clinician to activate the motor.

[0143] The control interface of the handle 697 may be in communication with a controller 698 of the handle 697 to selectively activate the motors to affect rotation of the drive shaft. The controller 698 may be disposed within the handle 697 and configured to receive input from the control interface and adapter data from the adapter 685 or loading unit data from the loading unit 687. The controller 698 may analyze the input from the control interface and the data received from the adapter 685 and / or the loading unit 687 to selectively activate the motors. The handle 697 may also include a display viewable by a clinician while using the handle 697. The display may be configured to display a portion of the adapter or loading unit data before, during, or after firing of the instrument 682.

[0144] The adapter 685 may include an adapter identification device 684 disposed therein, and the loading unit 687 may include a loading unit identification device 688 disposed therein. The adapter identification device 684 may be in communication with a controller 698, and the loading unit identification device 688 may be in communication with the controller 698. It will be appreciated that the loading unit identification device 688 may be in communication with the adapter identification device 684, which relays or passes communications from the loading unit identification device 688 to the controller 698.

[0145] The adapter 685 may also include multiple sensors 686 (one shown) disposed about its periphery to detect various conditions of the adapter 685 or the environment (e.g., whether the adapter 685 is connected to the loading unit, whether the adapter 685 is connected to the handle, whether the drive shaft is rotating, the torque of the drive shaft, the strain on the drive shaft, the temperature within the adapter 685, the number of times the adapter 685 has been fired, the peak force of the adapter 685 during firing, the total amount of force applied to the adapter 685, the peak retract force of the adapter 685, the number of times the adapter 685 has been paused during firing, etc.). The multiple sensors 686 may provide input to the adapter identification device 684 in the form of data signals. The data signals of the multiple sensors 686 may be stored in the adapter identification device 684 or may be used to update adapter data stored in the adapter identification device 684. The data signals of the multiple sensors 686 may be analog or digital. The multiple sensors 686 may include a force gauge that measures the force exerted on the loading unit 687 during firing.

[0146] The handle 697 and adapter 685 may be configured to interconnect the adapter identification device 684 and the loading unit identification device 688 with the controller 698 via an electrical interface. The electrical interface may be a direct electrical interface (i.e., including electrical contacts that engage with each other to transmit energy and signals therebetween). Additionally or alternatively, the electrical interface may be a contactless electrical interface for wirelessly transmitting (e.g., inductively transmitting) energy and signals therebetween. It is also contemplated that the adapter identification device 684 and the controller 698 may communicate wirelessly with each other via a wireless connection that is separate from the electrical interface.

[0147] The handle 697 may include a transceiver 683 configured to transmit instrument data from the controller 698 to other components of the system 680 (e.g., the LAN 20292, the cloud 693, the console 694, or the portable device 696). The controller 698 may also transmit instrument data and / or measurement data associated with one or more sensors 686 to the surgical hub. The transceiver 683 may receive data (e.g., cartridge data, loading unit data, adapter data, or other notifications) from the surgical hub 670. The transceiver 683 may also receive data (e.g., cartridge data, loading unit data, or adapter data) from other components of the system 680. For example, the controller 698 may transmit instrument data to the console 694, including the serial number of an attached adapter (e.g., adapter 685) attached to the handle 697, the serial number of a loading unit (e.g., loading unit 687) attached to the adapter 685, and the serial number of a multi-fire fastener cartridge loaded in the loading unit. The console 694 may then return data associated with the attached cartridge, loading unit, and adapter (e.g., cartridge data, loading unit data, or adapter data), respectively, to the controller 698. The controller 698 may display a message on a local instrument display or may send a message via the transceiver 683 to the console 694 or portable device 696, respectively, to display the message on the display 695 or portable device screen.

[0148] 7A illustrates a surgical system 700 that may include a matrix of surgical information. This surgical information may include any discrete atoms of information related to a surgical procedure. Generally described, such surgical information may include information related to the context and scope of the surgical procedure itself (e.g., healthcare information 728). Such information may include data such as, for example, procedure data and patient record data. The procedure data and / or patient record data may be associated with an associated healthcare data system 716 that is in communication with the surgical hub 704.

[0149] The surgical information may include information related to the configuration and / or control of devices being used in the procedure (e.g., device operation information 729). Such device operation information 729 may include information regarding the initial configuration of a surgical device. Device operation information 729 may include information regarding changes to the configuration of a surgical device. Device operation information 729 may include information regarding controls sent from the surgical hub 704 to the device and the information flow associated with such controls.

[0150] Surgical information may include information generated during the surgery itself (e.g., surgical information 727). Such surgical information 727 may include any information generated by a surgical data source 726. The data source 726 may include any device within the surgical context that may generate useful surgical information 727. This surgical information 727 may present itself as an observable quality of the data source 726. The observable quality may include static qualities such as a device's model number, serial number, etc. The observable quality may include dynamic qualities such as the state of a device's configurable settings. Surgical information 727 may present itself, for example, as the result of sensor observations. Sensor observations may include observations from specific sensors in the operating room, sensors for monitoring conditions such as the patient's condition, sensors embedded in surgical devices, etc. Sensor observations may include information used during the surgery, such as video, audio, etc. Surgical information 727 may present itself as device event data. The surgical device may generate notifications and / or log events, and such events may be included in surgical information 727 for communication to the surgical hub 704. Surgical information 727 may present itself, for example, as a result of manual recording. A medical professional may record during a procedure by asking the patient to take notes, capturing still images from the display, etc.

[0151] The surgical data sources 726 may include modular devices (e.g., which may include sensors configured to detect parameters associated with the patient, HCP, and environment, and / or the modular devices themselves), local databases (e.g., a local EMR database containing patient records), patient monitoring devices (e.g., blood pressure (BP) monitors and electrocardiography (EKG) monitors), HCP monitoring devices, environmental monitoring devices, surgical instruments, surgical procedure support equipment, etc.

[0152] The surgical hub 704 can be configured to derive contextual information about the surgical procedure from the data based on, for example, a particular combination of received data or a particular order in which data is received from the data sources 726. The contextual information inferred from the received data can include, for example, the type of surgical procedure being performed, the particular step of the surgical procedure the surgeon is performing, the type of tissue being operated on, or the body cavity that is the target of the procedure. This ability, by some aspects of the surgical hub 704, to derive or infer information about the surgical procedure from the received data can be referred to as “situational awareness.” For example, the surgical hub 704 can incorporate a situational awareness system, which is hardware and / or programming associated with the surgical hub 704 that derives contextual information about the surgical procedure from the received data and / or surgical planning information received from the edge computing system 714 or the healthcare data system 716 (e.g., an enterprise cloud server).

[0153] In operation, this matrix of surgical information may exist as one or more information flows. For example, surgical information may flow from a surgical data source 726 to the surgical hub 704. Surgical information may flow from the surgical hub 704 to a surgical data source 726 (e.g., a surgical device). Surgical information may flow between the surgical hub 704 and one or more healthcare data systems 716. Surgical information may flow between the surgical hub 704 and one or more edge computing devices 714.

[0154] The surgical information as presented in one or more information flows may be used in conjunction with one or more artificial intelligence (AI) systems to further enhance the operation of the surgical system 700. For example, a machine learning system, such as those described herein, may operate on one or more of the information flows to further enhance the operation of the surgical system 700.

[0155] 7B shows an exemplary computer-implemented surgical system 730 having multiple information flows 732. The surgical computing device 704 may communicate with and / or incorporate one or more surgical data sources. For example, the imaging module 733 (and endoscope) may exchange surgical information with the surgical computing device 704. Such information may include information from the imaging module 733 (and endoscope), such as video information, current settings, system status information, etc. The imaging module 733 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (software / firmware, etc.).

[0156] For example, the generator module 734 (and corresponding energy devices) may exchange surgical information with the surgical computing device 704. Such information may include information from the generator module 734 (and corresponding energy devices), such as electrical information (e.g., current, voltage, impedance, frequency, wattage), activity state information, sensor information such as temperature, current settings, system events, active duration, and startup timestamps. The generator module 734 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to the medical professional (e.g., changes in the pitch, duration, and melody of an audible tone), electrical application profiles and / or application logic that may instruct the generator module to provide energy having a defined characteristic curve over the application time, operational updates (e.g., software / firmware), and the like.

[0157] For example, the smoke evacuator 735 may exchange surgical information with the surgical computing device 704. Such information may include information from the smoke evacuator 735 such as operational information (e.g., revolutions per minute), activity status information, sensor information such as temperature, current settings, system events, active duration, and boot timestamps. The smoke evacuator 735 may receive information from the surgical computing device 704 such as control information, configuration information, operational updates (software / firmware, etc.).

[0158] For example, the aspirate / irrigate module 736 may exchange surgical information with the surgical computing device 704. Such information may include information from the aspirate / irrigate module 736, such as operational information (e.g., liters per minute), activity status information, internal sensor information, current settings, system events, active duration, and startup timestamps. The aspirate / irrigate module 736 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (software / firmware, etc.).

[0159] For example, the communications module 739, the processor module 737, and / or the storage array 738 may exchange surgical information with the surgical computing device 704. In one example, the communications module 739, the processor module 737, and / or the storage array 738 may comprise all or part of the computing platform on which the surgical computing device 704 operates. In one example, the communications module 739, the processor module 737, and / or the storage array 738 may provide local computing resources to other devices in the surgical system 730. Information from the communications module 739, the processor module 737, and / or the storage array 738 to the surgical computing device 704 may include logical computing related reports such as processing load, processing power, process identification, CPU %, CPU time, threads, GPU %, GPU time, memory utilization, memory threads, memory ports, energy usage, bandwidth related information, packets in, packets out, data rates, channel utilization, buffer status, packet loss information, system events, and other status information. The communications module 739, processor module 737, and / or storage array 738 may receive information, such as control information, configuration information, operational updates (software / firmware, etc.), etc., from the surgical computing device 704. The communications module 739, processor module 737, and / or storage array 738 may also receive information from the surgical computing device 704 generated by another element or device of the surgical system 730. For example, data source information may be transmitted to and stored in the storage array. For example, the data source information may be processed by the processor module 737.

[0160] For example, the intelligent instrument 740 (with or without a corresponding display) may exchange surgical information with the surgical computing device 704. Such information may include information from the intelligent instrument 740 regarding the operation of the instrument, such as device electrical and / or mechanical information (e.g., current, voltage, impedance, frequency, wattage, torque, force, pressure, etc.), load status information (e.g., information regarding the identity, type, and / or status of reusables such as staple cartridges), clamping force, tissue compression pressure, and / or internal sensor information such as time, system events, active duration, and activation timestamp. The intelligent instrument 740 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of visible and audible notifications to the medical professional (e.g., changes in the pitch, duration, and melody of an audible tone), mechanical application profiles and / or application logic that may instruct the instrument's mechanical components to operate with defined characteristics (e.g., blade / anvil advancement speed, mechanical advantage, firing time, etc.), operational updates (software / firmware, etc.), and the like.

[0161] For example, the sensor module 741 may exchange surgical information with the surgical computing device 704. Such information may include information from the sensor module 741 about its sensor capabilities, such as the sensor results themselves, observation frequency and / or resolution, observation type, device alerts such as alerts for sensor failure, observations exceeding a defined range, observations exceeding an observable range, etc. The sensor module 741 may receive information from the surgical computing device 704, such as control information, configuration information, changes in the nature of the observations (e.g., frequency, resolution, observation type, etc.), triggers defining specific events for observations, on controls, off controls, data buffering, data pre-processing algorithms, operational updates (software / firmware, etc.), etc.

[0162] For example, the visualization system 742 may exchange surgical information with the surgical computing device 704. Such information may include information from the visualization system 742, such visualization data itself (e.g., still images, video, advanced spectral visualization, etc.), visualization metadata (e.g., visualization type, resolution, frame rate, encoding, bandwidth, etc.), etc. The visualization system 742 may receive information from the surgical computing device 704, such as control information, configuration information, changes in video settings (e.g., visualization type, resolution, frame rate, encoding, etc.), visual display overlay data, data buffering sizes, data pre-processing algorithms, operational updates (software / firmware, etc.), etc.

[0163] For example, the surgical robot 743 may exchange surgical information with the surgical computing device 704. Information from the surgical robot 743 may include any of the aforementioned information as it applies to robotic instruments, sensors, and devices. Information from the surgical robot 743 may also include information related to the robotic operation or control of such instruments, such as electrical / mechanical feedback of the robotic articulator, system events, system settings, mechanical resolution, control operation logs, articulator path information, etc. The surgical robot 743 may receive information from the surgical computing device 704, such as control information, configuration information, operation updates (software / firmware, etc.).

[0164] 7C illustrates an exemplary information flow associated with multiple surgical computing systems 704a, 704b within a common environment. As the overall configuration of a computer-implemented surgical system (e.g., computer-implemented surgical system 750) changes (e.g., as data sources are added and / or removed from the surgical computing system), additional surgical information may be generated to reflect the changes. In this example, a second surgical computing system 704b (e.g., a surgical hub) may be added to surgical system 750 (along with a corresponding surgical robot) comprising existing surgical computing system 704a. The messaging flows described herein represent additional surgical information flow 755 (e.g., further integrated, analyzed, and / or processed according to algorithms such as machine learning algorithms) used as disclosed herein.

[0165] Here, two surgical computing systems 704a, 704b request permission from the surgeon for the second surgical computing system 704b (with a corresponding surgical robot 756) to take control of the operating room from the existing surgical computing system 704a. The second surgical computing system 704b presents control of the corresponding surgical robot 756, robotic visualization tower 758, Monohat tool 759, and robotic stapler 749 in the operating room. Permission may be requested through the surgeon interface or console 751. Once permission is granted, the second surgical computing system 704b sends a message to the existing surgical computing system 704a requesting transfer of control of the operating room.

[0166] In one example, the surgical computing systems 704a, 704b can negotiate the nature of their interaction without external input based on previously collected data. For example, the surgical computing systems 704a, 704b may collectively determine that an upcoming surgical task requires the use of a robotic system. Such a determination may cause the existing surgical computing system 704a to autonomously hand over control of the operating room to a second surgical computing system 704b. Upon completion of the surgical task, the second surgical computing system 704b may then autonomously return control of the operating room to the existing surgical computing system 704a.

[0167] As illustrated in Figure 7C, the existing surgical computing system 704a has transferred control to a second surgical computing system 704b, which also assumes control of the surgeon interface 751 and secondary display 752. The second surgical computing system 704b assigns new identification numbers to the newly transferred devices. The existing surgical computing system 704a retains control of the handheld stapler 753, handheld powered dissector 754, and visualization tower 757. In addition, the existing surgical computing system 704a may perform a support role, with the processing and storage capabilities of the existing surgical computing system 704a now available to the second surgical computing system 704b.

[0168] 7D illustrates an exemplary surgical information flow in the context of a surgical procedure and corresponding exemplary uses of the surgical information for predictive modeling. The surgical information disclosed herein may provide data regarding one or more surgical procedures, including surgical tasks, instruments, instrument settings, motion information, procedural variations, and corresponding desirable metrics such as improved patient outcomes, lower costs (e.g., fewer resources utilized, shorter surgical time, etc.). The surgical information disclosed herein (e.g., that disclosed with respect to FIGS. 7A-7C ), in the context of one or more surgical systems and devices disclosed herein, provides a platform upon which certain machine learning algorithms and techniques disclosed herein may be used.

[0169] Surgical information 762 from multiple surgical procedures 764 (e.g., a subset of surgical information from each procedure) may be collected. Surgical information 762 may be collected from multiple surgical procedures 764, for example, by collecting data represented by one or more information flows disclosed herein.

[0170] To illustrate, an exemplary instance of surgical information 766 may be generated from an exemplary procedure 768 (e.g., a lung segmentectomy procedure as shown on timeline 769). Surgical information 766 may be generated during preoperative planning and may include patient record information. Surgical information 766 may be generated from data sources (e.g., data sources 726) during the course of a surgical procedure, including data generated each time medical personnel utilize a modular device paired with the surgical computing system (e.g., surgical computing system 704). The surgical computing system may receive this data from the paired modular device and other data sources. The surgical computing system itself may generate surgical information as part of its operation during a procedure. For example, the surgical computing system may record information related to configuration and control operations. The surgical computing system may record information related to situational awareness activities. For example, the surgical computing system may record recommendations, prompts, and / or other information provided to the medical team (e.g., provided via a display screen) that may be related to the next procedure step. For example, the surgical computing system may record configuration and control changes (e.g., adjustments to modular devices based on context) that may include activating a monitor, adjusting the field of view (FOV) of a medical imaging device, changing the energy level of an ultrasonic surgical instrument or an RF electrosurgical instrument, etc.

[0171] Hospital personnel retrieve the patient's EMR from the hospital's EMR database at 770. Based on the patient data selected in the EMR, the surgical computing system determines that the procedure to be performed is a thoracic procedure.

[0172] At 771, personnel scan incoming medical supplies for a procedure. The surgical computing system may cross-reference the scanned supplies with a list of supplies utilized in various types of procedures. The surgical computing system may verify that the mix of supplies corresponds to a thoracic procedure. Additionally, the surgical computing system may determine that the procedure is not a wedge resection (because the incoming supplies either do not include certain supplies needed for a thoracic wedge resection or are otherwise not compatible with a thoracic wedge resection). The medical personnel may scan a patient band via a scanner communicatively connected to the surgical computing system. The surgical computing system may verify the patient's identity based on the scanned data.

[0173] At 774, medical personnel turn on auxiliary equipment. The auxiliary equipment utilized may vary according to the type of surgical procedure and the technology used by the surgeon. In this example, the auxiliary equipment may include a smoke evacuator, an aspirator, and a medical imaging device. Once activated, the auxiliary equipment may pair with the surgical computing system. The surgical computing system may derive contextual information regarding the surgical procedure based on the paired type. In this example, the surgical computing system determines that the surgical procedure is a VATS procedure based on this particular combination of paired devices. The contextual information regarding the surgical procedure may be ascertained by the surgical computing system via information from the patient's EMR.

[0174] The surgical computing system may retrieve the steps of the procedure to be performed. For example, the steps may be associated with a treatment plan (e.g., a treatment plan specific to this patient's surgery, a treatment plan associated with a particular surgeon, a treatment plan template for the procedure in general, etc.).

[0175] At 775, staff attach EKG electrodes and other patient monitoring devices to the patient. The EKG electrodes and other patient monitoring devices pair with the surgical computing system. The surgical computing system may receive data from the patient monitoring devices.

[0176] At 776, medical personnel induce anesthesia in the patient. The surgical computing system may record information related to this procedure step, such as data from the modular devices and / or patient monitoring devices, including, for example, EKG data, blood pressure data, ventilator data, or a combination thereof.

[0177] At 777, the lung of the patient undergoing surgery is collapsed (and ventilation may be switched to the contralateral lung). The surgical computing system may determine that this procedure step has begun and may collect surgical information accordingly, including, for example, ventilator data, one or more timestamps, etc.

[0178] At 778, a medical imaging device (e.g., a scope) is inserted and video from the medical imaging device is initiated. The surgical computing system may receive medical imaging device data (i.e., video or image data) through a connection to the medical imaging device. The data from the medical imaging device may include imaging data and / or imaging metadata, such as the angle at which the medical imaging device is oriented relative to visualization of the patient's anatomy, the number of medical imaging devices currently active, etc. The surgical computing system may record positioning information for the medical imaging device. For example, one technique for performing a VATS lobectomy places the camera in the anterior-inferior corner of the patient's thoracic cavity above the diaphragm. Another technique for performing a VATS segmentectomy places the camera in an anterior intercostal position relative to the segmental fissure.

[0179] For example, using pattern recognition or machine learning techniques, the surgical computing system may be trained to recognize the positioning of a medical imaging device according to visualization of the patient's anatomy. For example, one technique for performing a VATS lobectomy utilizes a single medical imaging device. Another technique for performing a VATS segmentectomy uses multiple cameras. Yet another technique for performing a VATS segmentectomy uses an infrared light source (which may be communicatively coupled to the surgical computing system as part of the visualization system).

[0180] At 779, the surgical team begins the incision step of the procedure. The surgical computing system may collect data from the RF or ultrasonic generator indicating that the energy instrument is being fired. The surgical computing system may cross-reference the received data with the retrieved steps of the surgical procedure to determine that the energy instrument being fired at this point in the process (i.e., after previously discussed steps of the procedure have been completed) corresponds to the incision step. In one example, the energy instrument may be an energy tool mounted on a robotic arm of a robotic surgical system.

[0181] At 780, the surgical team proceeds to the ligation step of the procedure. The surgical computing system may collect surgical information 766 related to the surgeon ligating the arteries and veins based on receiving data from the surgical stapling and severing instrument indicating that such instrument is being fired. Next, the segmentectomy portion of the procedure is performed. The surgical computing system may collect information related to the surgeon transecting the parenchyma. For example, the surgical computing system may receive surgical information 766 from the surgical stapling and severing instrument, including data related to its cartridge, settings, firing details, etc.

[0182] At 782, the node dissection step is then performed. The surgical computing system may collect surgical information 766 related to the surgical team dissecting the node and performing the leak test. For example, the surgical computing system may collect data received from the generator indicating that an RF or ultrasonic instrument is being fired, including electrical and status information associated with the firing. The surgeon periodically alternates between the surgical stapling / cutting instrument and the surgical energy (i.e., RF or ultrasonic) instrument depending on the particular step in the procedure. The surgical computing system may collect surgical information 766 taking into account the particular sequence in which the stapling / cutting instrument and the surgical energy instrument are used. In one example, a robotic tool may be used for one or more steps in the surgical procedure. The surgeon may, for example, alternate between using the robotic tool and a handheld surgical instrument and / or use the devices simultaneously.

[0183] The incisions are then closed and the post-operative portion of the procedure begins. The patient is de-anesthetized at 784. The surgical computing system may collect surgical information regarding the patient emerging from anesthesia, for example, based on ventilator data (e.g., the patient's breathing rate begins to increase).

[0184] At 785, medical personnel remove various patient monitoring devices from the patient. The surgical computing system may collect information regarding the outcome of the procedure. For example, the surgical computing system may collect information related to the loss of EKG, BP, and other data from the patient monitoring devices.

[0185] The surgical information 762 (including the surgical information 766) may be structured and / or labeled. The surgical computing system may inherently provide such structure and / or labeling in the data collection. For example, the surgical information 762 may be labeled according to particular characteristics, desired results (e.g., efficiency, patient outcome, cost, and / or combinations thereof, etc.), particular surgical techniques, aspects of instrumentation (e.g., surgical instrument selection, timing, and activation, instrument settings, nature of instrument use, etc.), identities of medical professionals involved, particular patient characteristics, etc., each of which may be present in the data collection.

[0186] The surgical information (e.g., surgical information 762 collected over the procedure 764) may be used in connection with one or more artificial intelligence (AI) systems. AI may be used to perform computer cognitive tasks. For example, AI may be used to perform complex tasks based on observation of data. AI may be used to enable computing systems to perform cognitive tasks and solve complex tasks. AI may include using machine learning and machine learning techniques. ML techniques may include, for example, performing complex tasks without being programmed (e.g., explicitly programmed). For example, ML techniques may improve over time based on completing tasks with different inputs. An ML process may train itself, for example, using input data and / or a training dataset.

[0187] Machine learning (ML) techniques may be used, for example, in the medical field. For example, ML may be used on a set of data (e.g., a set of surgical data) to generate output (e.g., reduced surgical data, processed surgical data). In an example, the output of the ML process may include identified trends or relationships in the data input for processing. The output may include verifying results and / or outcomes associated with the input data. In an example, the input to the ML process may include medical data such as surgical images and patient scans. The ML process may output a determined medical condition based on the input surgical images and patient scans. The ML process may be used to diagnose a medical condition, for example, based on the surgical scans.

[0188] An ML process may improve itself, for example, using historical data and / or input data that trained the ML process. Thus, an ML process may continually improve with added inputs and processing. The ML process may update based on the input data. For example, over time, an ML process that generates medical outcomes based on medical data may improve and become more accurate and consistent in medical diagnoses.

[0189] ML processes may be used to solve different complex tasks (e.g., medical tasks). For example, ML processes may be used for data reduction, data preparation, data processing, trend identification, outcome determination, medical diagnosis, and / or the like. For example, an ML process may take surgical data as input and process the data for use in medical analysis. The processed data may be used to determine a medical diagnosis. Finally, an ML process may take raw surgical data and generate useful medical information (e.g., medical trends and / or diagnoses) associated with the raw surgical data.

[0190] ML processes may be combined to perform different discrete tasks on an input data set. For example, an ML process may include testing different combinations of ML subprocesses performing discrete tasks to determine which combination performs best (e.g., competitive use of different process / algorithm types and training to determine the best combination for a data set). For example, an ML process may include subprocess (e.g., algorithm) control and monitoring to refine and / or validate results and / or outcomes (e.g., error bounds).

[0191] An ML process may be initialized and / or set up to perform a task. For example, the ML process may be initialized based on initialization configuration information. The initialized ML process may be an untrained ML process and / or a base ML process for performing the task. An untrained ML process may be inaccurate in performing a specified task. As the ML process is trained, the task may be performed more accurately.

[0192] The initialization configuration information for the ML process may include initial settings and / or parameters. For example, the initial settings and / or parameters may include defined ranges for use by the ML process. The ranges may include manually entered and / or received data ranges. The ranges may include default ranges and / or randomized ranges for unreceived variables that may be used, for example, to complete the dataset for processing. For example, if a dataset lacks a data range, the default data range may be used as a substitute to run the ML process.

[0193] The initialization configuration information for the ML process may include data storage locations. For example, locations or data storage and / or databases associated with data interactions may be included. The databases associated with data interactions may be used to identify trends in the dataset. The databases associated with data interactions may include mappings of data to medical conditions. For example, the databases associated with data interactions may include mappings of heart rate data to arrhythmias, etc.

[0194] The initialization configuration information may include parameters associated with defining the system. The initialization configuration information may include instructions (e.g., methods) associated with displaying, verifying, and / or providing information to a user. For example, the initialization configuration may include instructions for an ML process to output data in a particular format for visualization to a user.

[0195] ML techniques may be used, for example, to perform data reduction. ML techniques for data reduction may include using multiple different data reduction techniques. For example, ML techniques for data reduction may include using one or more of the following: CUR matrix decomposition; decision trees; expectation-maximization (EM) processes (e.g., algorithms); explicit semantic analysis (ESA); exponential smoothing forecasting; generalized linear models; k-means clustering (e.g., nearest neighbor); naive Bayes; neural network processes; multivariate analysis; o-cluster; singular value decomposition; Q-learning; temporal difference (TD); deep adversarial networks; support vector machines (SVM); linear regression; dimensionality reduction; linear discriminant analysis (LDA); adaptive boosting (e.g., AdaBoost); gradient descent (e.g., stochastic gradient descent (SGD)); outlier detection; and / or others.

[0196] ML techniques may be used to perform data reduction, for example, using CUR matrix decomposition. CUR matrix decomposition may include using a matrix decomposition model (e.g., process, algorithm), such as a low-rank matrix decomposition model. For example, CUR matrix decomposition may include a low-rank matrix decomposition process that is expressed (e.g., explicitly expressed) in several (e.g., a small number) columns and / or rows of a data matrix (e.g., the CUR matrix decomposition may be interpretable). CUR matrix decomposition may include selecting columns and / or rows associated with statistical leverage and / or large influence in the data matrix. Using CUR matrix decomposition may enable identifying attributes and / or rows within the data matrix. Simplification of larger datasets (e.g., using CUR matrix decomposition) may enable users to review and interact (e.g., with the data). CUR matrix decomposition may facilitate regression, classification, clustering, and / or other processes.

[0197] ML techniques may be used to perform data reduction using, for example, decision trees (e.g., decision tree models). Decision trees may be used, for example, as a framework for quantifying outcome values ​​and / or the probability of an outcome occurring. Decision trees may be used, for example, to calculate values ​​for uncertain outcome nodes (e.g., in a decision tree). Decision trees may be used, for example, to calculate values ​​for decision nodes (e.g., in a decision tree). Decision trees may be models that enable classification and / or regression (e.g., applicable to classification and / or regression problems). Decision trees may be used to analyze numerical (e.g., continuous) and / or categorical data. Decision trees may be more successful and / or more efficient with large datasets (e.g., compared to other data reduction techniques).

[0198] Decision trees may be used in combination with other decision trees. For example, a random forest may refer to a collection of decision trees (e.g., an ensemble of decision trees). A random forest may include a collection of decision trees whose results may be aggregated into a result. A random forest may be a supervised learning algorithm. A random forest may be trained, for example, using a bagging training process.

[0199] A random decision forest (e.g., random forest) may add randomness (e.g., additional randomness) to a model, for example, while growing a tree. A random forest may be used, for example, to search for the best feature among a random subset of features, rather than searching for the most important feature (e.g., while splitting a node). Searching for the best feature among a random subset of features may result in a wide variety, which may result in a better (e.g., more efficient and / or accurate) model.

[0200] Random forests may include using parallel ensembles. Parallel ensembles may include, for example, fitting (e.g., several) decision tree classifiers in parallel on different dataset subsamples. Parallel ensembles may include using majority voting or averaging on the results or final outcome. Parallel ensembles may be used to minimize overfitting and / or increase prediction accuracy and control. Random forests with multiple decision trees may (e.g., generally) be more accurate than single decision tree-based models. A set of decision trees with controlled variation may be constructed, for example, by combining bootstrap aggregation (e.g., bagging) with random feature selection.

[0201] ML techniques may be used to perform data reduction, for example, using an expectation-maximization (EM) model (e.g., process, algorithm). For example, an EM model may be used to find likelihood (e.g., local maximum likelihood) parameters of a statistical model. An EM model may be used when equations cannot be solved directly. An EM model may consider latent variables and / or unknown parameters and known data observations. For example, an EM model may determine that missing values ​​are present in a dataset. An EM model receives configuration information indicating to assume the presence of missing (e.g., unobserved) data points in the dataset.

[0202] The EM model may use component clustering. For example, component clustering may allow EM components to be grouped into high-level clusters. For example, if component clustering is disabled (e.g., in the EM model), the components may be treated as clustered.

[0203] ML techniques may be used to perform data reduction, for example, using Explicit Semantic Analysis (ESA). ESA may be used at the level of semantics (e.g., meaning) rather than the vocabulary (e.g., surface form vocabulary) of words or documents. ESA may focus on the meaning of a set of text, for example, as a combination of concepts found within the text. ESA may be used for document classification. ESA may be used for semantic relevance computation (e.g., how similar words or fragments of text are to each other). ESA may be used for information retrieval.

[0204] ESAs may be used, for example, for document classification. Document classification may include tagging documents for management and sorting. Tagging documents (e.g., with keywords) may make them easier to search. Keyword tagging (e.g., using only keyword tagging) may limit the accuracy and / or efficiency of document classification. For example, using keyword tagging may reveal (e.g., only) documents that have the keyword, but not documents that have words with similar meanings to the keyword. Semantically classifying text (e.g., using ESAs) may improve a model's understanding of the text. Semantically classifying text may include representing documents as concepts and reducing reliance on specific keywords.

[0205] ML techniques may be used to perform data reduction, for example, using an exponential smoothing forecasting model. Exponential smoothing may be used to smooth time series data, for example, using an exponential window function. For example, in a moving average, past observations may be weighted equally, but using an exponential function, weights may be assigned that decrease exponentially over time.

[0206] ML techniques may be used to perform data reduction, for example, using linear regression. Linear regression may be used to predict continuous outcomes. For example, linear regression may be used to predict the value of a variable (e.g., a dependent variable) based on the values ​​of different variables (e.g., independent variables). Linear regression may apply a linear approach to model the relationship between a scalar response and one or more explanatory variables (e.g., a dependent variable and / or independent variables). Simple linear regression may refer to a linear regression use case associated with one explanatory variable. Multiple linear regression may refer to a linear regression use case associated with two or more explanatory variables. Linear regression may model the relationship, for example, using a linear prediction function. The linear prediction function may estimate unknown model parameters from a dataset.

[0207] For example, linear regression may be used to identify patterns within a training dataset. The identified patterns may relate to groupings of values ​​and / or labels. The model may learn the relationship between (e.g., each) label and expected outcomes. After training, the model may be used on raw data outside the training dataset (e.g., data that does not have a mapped and / or known output). A trained model using linear regression may determine a calculated prediction associated with the raw data, such as identifying seasonal changes in sales data.

[0208] ML techniques may be used to perform data reduction, e.g., generalized linear models (GLMs). GLMs may be used as flexible generalizations of linear regression. GLMs may generalize linear regression, for example, by allowing linear models to relate response variables.

[0209] ML techniques may be used to perform data reduction, for example, using k-means clustering (e.g., nearest neighbor models). K-means clustering may be used in vector quantization. K-means clustering may be used in signal processing. K-means clustering may, for example, aim to divide n observations into k clusters, with each observation falling into the cluster with the closest mean.

[0210] K-means clustering may include K-Nearest Neighbor (KNN) learning. KNN may be instance-based learning (e.g., non-generalized learning, lazy learning). KNN may refrain from building a general internal model. KNN may include storing instances corresponding to training data in an n-dimensional space. KNN may use the data to classify data points, for example, based on a similarity measure (e.g., a Euclidean distance function). Classification may be calculated, for example, based on a majority vote of the k neighbors of a point (e.g., each point). KNN may be robust to noisy training data. Accuracy may depend on data quality (e.g., for KNN). KNN may include selecting the number of neighbors to consider (e.g., an optimal number of neighbors to consider). KNN may be used for classification and / or regression.

[0211] ML techniques may be used to perform data reduction, for example, using a naive Bayes model (e.g., process). For example, a naive Bayes model may be used to build a classifier. Using the naive Bayes model, a class label may be assigned to a problem instance (e.g., represented as a vector of feature values). The class label may be drawn from a set (e.g., a finite set). Different processes (e.g., algorithms) may be used to train the classifier. A family of processes (e.g., a family of algorithms) may be used. A family of processes may be based on the principle that a naive Bayes classifier (e.g., all naive Bayes) classifier assumes that feature values ​​are independent of different feature values ​​(e.g., given a class variable).

[0212] ML techniques may be used to perform data reduction, for example, using neural networks. The neural network may learn (e.g., be trained) by processing examples, for example, to perform other tasks (e.g., similar tasks). The processed examples may include inputs and results (e.g., inputs mapped to results). The neural network may learn by forming probability-weighted associations between inputs and results. The probability-weighted associations may be stored within the neural network's data structure. Training the neural network from a given example may be performed by determining the difference between the network's processed output (e.g., prediction) and a target output. This difference may be an error. The neural network may adjust the weighted associations (e.g., stored weighted associations), for example, according to a learning rule and an error value.

[0213] ML techniques may be used to perform data reduction, for example, using multivariate analysis, which may include performing multivariate state estimation and / or non-negative matrix factorization.

[0214] ML techniques may be used to perform data reduction using, for example, a support vector machine (SVM). SVMs may be used in multidimensional spaces (e.g., high-dimensional spaces, infinite-dimensional spaces). SVMs may be used to construct hyperplanes (e.g., a set of hyperplanes). A hyperplane with the greatest distance (e.g., compared to other constructed hyperplanes) from the nearest training data point within a class (e.g., any class) may achieve strong separation (e.g., generally, the larger the margin, the lower the generalization error of the classifier). SVMs may be effective in high-dimensional spaces. SVMs may behave differently, for example, based on different mathematical functions (e.g., kernels, kernel functions). For example, kernel functions may include one or more of linear, polynomial, radial basis function (RBF), sigmoid, etc. Kernel functions may be used as SVM classifiers. SVMs may be limited, for example, in use cases where the dataset contains a large amount of noise (e.g., overlapping target classes).

[0215] ML techniques may be used to perform data reduction, such as, for example, dimensionality reduction. Reducing the dimensionality of a sample of data (e.g., unlabeled data) may help refine groups and / or clusters. Reducing the number of variables in a model may simplify the data's trends. Simplified data trends may allow for more efficient processing. Dimensionality reduction may be used, for example, when many (e.g., too many) dimensions obscure (e.g., adversely affect) insights, trends, patterns, outcomes, and / or the like.

[0216] Reducing dimensionality may include using principal component analysis (PCA). PCA may be used to establish principal components that govern the relationships between data points. PCA may focus on simplifying (e.g., simplifying only) the principal components. Dimensionality reduction (e.g., PCA) may be used to maintain the diversity of data groupings within a dataset, but rationalize the number of distinct groups.

[0217] ML techniques may be used to perform data reduction, e.g., linear discriminant analysis (LDA). LDA may refer to a linear decision boundary classifier, which may be created, for example, by fitting class conditional densities to data (e.g., and applying Bayes' rule). LDA may include a generalization of Fisher's Linear Discriminant (e.g., projecting a given dataset into a lower-dimensional space to reduce dimensionality, minimize model complexity, and reduce computational cost). An LDA model (e.g., a standard LDA model) may fit classes with Gaussian densities. An LDA model may assume that classes (e.g., all classes) share a covariance matrix. LDA may be similar to an analysis of variance (ANOVA) process and / or regression analysis. For example, LDA may be used to express a dependent variable as a linear combination of other features and / or measurements.

[0218] ML techniques may be used to perform data reduction, such as, for example, adaptive boosting (e.g., AdaBoost). Adaptive boosting may include creating a classifier (e.g., a powerful classifier). Adaptive boosting may include creating a classifier by combining multiple classifiers (e.g., poorly performing classifiers), for example, to obtain a resulting classifier with high accuracy. AdaBoost may be an adaptive classifier that improves the efficiency of a classifier. AdaBoost may trigger overfitting. AdaBoost may be used (e.g., most commonly used) to improve the performance of decision trees, base estimators, binary classification problems, and / or the like. AdaBoost may be sensitive to noisy data and / or outliers.

[0219] ML techniques may be used to perform data reduction, such as stochastic gradient descent (SGD). SGD may include an iterative process used to optimize a function (e.g., an objective function). SGD may be used, for example, to optimize an objective function with specific smoothness properties. Stochastic may refer to random probability. SGD may be used, for example, to reduce computational load in high-dimensional optimization problems. SGD may be used, for example, to enable faster iterations while trading off a slower convergence rate. Gradient may refer, for example, to the slope of a function that calculates the degree of change of a variable in response to a change in another variable. Gradient descent may refer to a convex function that outputs the partial derivative of a set of its input parameters. For example, α may be a learning rate, and J may be the cost of training examples for the i-th iteration. This formula may represent a stochastic gradient descent weight update method for the j-th iteration. In large-scale ML and sparse ML, SGD may be applied to problems in text classification and / or natural language processing (NLP). SGD can be sensitive to feature scaling (e.g., it may be necessary to use a range of hyperparameters, such as the regularization parameter and number of iterations).

[0220] ML techniques may be used to perform data reduction, such as using outlier detection. An outlier may be a data point that contains information (e.g., useful information) about abnormal behavior of the system described by the data. Outlier detection processes may include univariate and multivariate processes.

[0221] The ML process may be trained, for example, using one or more training methods. For example, the ML process may be trained using one or more of the following training techniques: supervised learning; unsupervised learning; semi-supervised learning; reinforcement learning; and / or others.

[0222] Machine learning can be supervised (e.g., supervised learning). Supervised learning algorithms can create a mathematical model from training data sets (e.g., training data). FIG. 8A illustrates an exemplary supervised learning framework 800. Training data (e.g., training examples 802 as shown in FIG. 8) may consist of a set of training examples (e.g., input data mapped to labeled outputs as shown in FIG. 8). Training examples 802 may include one or more inputs and one or more labeled outputs. The labeled outputs may serve as supervisory feedback. In the mathematical model, training examples 802 may be represented by an array or vector, sometimes called a feature vector. Training data may be represented by rows of the feature vector that form a matrix. Through iterative optimization of an objective function (e.g., a cost function), supervised learning algorithms can learn a function (e.g., a prediction function) that can be used to predict outputs associated with one or more new inputs. A suitably trained predictive function (e.g., a trained ML model 808) may determine outputs 804 (e.g., labeled outputs) for one or more inputs 806 that may not be part of the training data (e.g., input data that does not have a mapped, labeled output, as shown in FIG. 8). Exemplary algorithms may include linear regression, logistic regression, neural networks, nearest neighbors, naive Bayes, decision trees, SVMs, and / or others. Exemplary problems that can be solved by supervised learning algorithms may include classification, regression problems, etc.

[0223] Machine learning can be unsupervised (e.g., unsupervised learning). FIG. 8B illustrates an exemplary unsupervised learning framework 810. An unsupervised learning algorithm 814 may train on a dataset that may include input 811 and may find structure 812 in the data (e.g., pattern detection and / or descriptive modeling). The structure 812 in the data may resemble groupings or clusterings of data points. Thus, the algorithm 814 may learn from training data that may be unlabeled. Instead of responding to supervised feedback, the unsupervised learning algorithm may identify commonalities in the training data and may react based on the presence or absence of such commonalities in each training data. For example, training may include operating on training input data to generate a model and / or output with a particular energy (e.g., cost function, etc.), and such energy may be used to further refine the model (e.g., to define a model that minimizes the cost function given the training input data). Exemplary algorithms may include the Apriori algorithm, K-means, K-nearest neighbors (KNN), K-medians, etc. Exemplary problems that can be solved by unsupervised learning algorithms may include clustering problems, anomaly / outlier detection problems, etc.

[0224] Machine learning may be semi-supervised (e.g., semi-supervised learning). Semi-supervised learning algorithms may be used in scenarios where labeling data is costly (e.g., because a skilled expert is required to label the data) and labels for the data are limited. Semi-supervised learning models may take advantage of the idea that while the group membership of unlabeled data is unknown, the data still holds important information about group parameters.

[0225] Machine learning may include reinforcement learning, which may be an area of ​​machine learning that may concern how a software agent can take actions in an environment to maximize some notion of cumulative reward. Reinforcement learning algorithms may not assume knowledge of an exact mathematical model of the environment (e.g., represented by a Markov decision process (MDP)) and may be used when an exact model may not be feasible. Reinforcement learning algorithms may be used in autonomous vehicles or in learning to play games against human opponents. Exemplary algorithms may include Q-learning, temporal difference (TD), deep adversarial networks, and / or others.

[0226] Reinforcement learning may involve an algorithm (e.g., an agent) that continuously learns from an environment in an iterative manner. During the training process, the agent may learn from experience with the environment until the agent has explored the full range of states (e.g., possible states). Reinforcement learning may be defined by the type of problem. Reinforcement learning solutions may be classified as reinforcement learning algorithms. In a problem, an agent may determine an action to select (e.g., a best action) based on the agent's current state. When steps are repeated, the problem may be referred to as an MDP.

[0227] For example, reinforcement learning may include an action step. The action step in reinforcement learning may include an agent observing an input state. The action step in reinforcement learning may include causing an agent to perform an action using a decision-making function. The action step may include an agent receiving a reward and / or reinforcement from the environment (e.g., after an action is performed). The action step in reinforcement learning may include storing state-action pair information related to the reward.

[0228] Machine learning may be part of a technology platform called cognitive computing (CC), which may comprise various fields such as computer science and cognitive science. CC systems may be able to learn at scale, reason purposefully, and interact naturally with humans. Self-teaching algorithms, which may use data mining, visual recognition, and / or natural language processing, may enable CC systems to solve problems and optimize human processes.

[0229] The output of a machine learning training process may be a model for predicting outcomes for new data sets. For example, a linear regression learning algorithm may have a cost function that can minimize the prediction error of a linear prediction function during the training process by adjusting the coefficients and constants of the linear prediction function. If a minimum value can be reached, the linear prediction function with the adjusted coefficients may be considered trained and constitute the model produced by the training process. For example, a neural network (NN) algorithm for classification (e.g., a multilayer perceptron (MLP)) may include a hypothesis function represented by a network of layers of nodes that are assigned biases and interconnected with weighted connections. The hypothesis function may also be a nonlinear function (e.g., a highly nonlinear function) that may include linear and logistic functions nested together, with an outermost layer consisting of one or more logistic functions. The NN algorithm may include a cost function for minimizing classification error by adjusting biases and weights through a process of feedforward propagation and backpropagation. If a global minimum can be reached, the optimized hypothesis function with the adjusted bias and weight layers may be considered trained and constitute the model that the training process produced.

[0230] Data aggregation may be performed for machine learning as a first stage in a machine learning lifecycle. Data aggregation may include steps such as identifying various data sources, collecting data from the data sources, and integrating the data. For example, to train a machine learning model for predicting surgical complications and / or post-surgical recovery rates, data sources including pre-surgical data such as a patient's medical condition and biomarker measurement data may be identified. Such data sources may be a patient's electronic medical record (EMR), a computing system that stores the patient's pre-surgical biomarker measurement data, and / or other similar data stores. Data from such data sources may be retrieved and stored in a central location for further processing in the machine learning lifecycle. Data from such data sources may be linked (e.g., logically linked) and accessed as if they were centrally stored. Surgical data and / or post-surgical data may be similarly identified and collected. Furthermore, the collected data may be integrated. In examples, a patient's pre-surgical medical record data, pre-surgical biomarker measurement data, pre-surgical data, surgical data, and / or post-surgical data may be combined into a patient record, which may be an EMR.

[0231] Data preparation can be performed for machine learning as another stage of the machine learning lifecycle. Data preparation can include data preprocessing steps such as data formatting, data cleaning, and data sampling. For example, collected data may not be in a data format suitable for training a model. Such data records can be converted into a flat file format for model training. Such data can be mapped to numerical values ​​for model training. Such identifying data can be removed before model training. For example, identifying data can be removed for privacy reasons. As another example, data can be removed because there may be more available data than can be used for model training. In such cases, a subset of the available data can be randomly sampled and selected for model training, and the remainder can be discarded.

[0232] Data preparation may include data transformation operations (e.g., after preprocessing), such as scaling and aggregation. For example, the preprocessed data may include data values ​​at various scales. These values ​​may be scaled up or down, e.g., to be between 0 and 1, for model training. For example, the preprocessed data may include data values ​​that become more meaningful when aggregated.

[0233] Model training may be another aspect of the machine learning life cycle. The model training process described herein may depend on the machine learning algorithm used. A model may be considered suitably trained after it has been trained, cross-validated, and tested. Thus, a dataset from the data preparation stage (e.g., an input dataset) may be divided into a training dataset (e.g., 60% of the input dataset), a validation dataset (e.g., 20% of the input dataset), and a test dataset (e.g., 20% of the input dataset). After a model is trained on the training dataset, it may be run on the validation dataset to reduce overfitting. If the model's accuracy is increasing, but decreases when run on the validation dataset, this may indicate an overfitting problem. The test dataset may be used to test the accuracy of the final model to determine whether it is ready for deployment or whether more training may be required.

[0234] Model deployment can be another aspect of the machine learning lifecycle. Models may be deployed as part of a standalone computer program. Models may be deployed as part of a larger computing system. Models may be deployed with model performance parameters. Such performance parameters may monitor model accuracy as it is used to make predictions on a running dataset. For example, such parameters may track false positives and false positives of a classification model. Such parameters may further store false positives and false positives for further processing to improve the accuracy of the model.

[0235] Model updates after deployment may be another aspect of the machine learning cycle. For example, a deployed model may be updated as false positives and / or as false positives are predicted on the production data. In one example, for an MLP model deployed for classification, when a false positive occurs, the deployed MLP model may be updated to increase the probability cutoff for predicting a positive to reduce the false positives. In one example, for an MLP model deployed for classification, when a false negative occurs, the deployed MLP model may be updated to decrease the probability cutoff for predicting a positive to reduce the false negatives. In one example, for an MLP model deployed for classification of surgical complications, when both false positives and false negatives occur, the deployed MLP model may be updated to decrease the probability cutoff for predicting a positive to reduce the false negatives, as predicting a false positive may be less serious than a false negative.

[0236] For example, the deployed model may be updated as more live production data becomes available as training data. In such cases, the deployed model may be further trained, validated, and tested using such additional live production data. In one example, the updated biases and weights of the further trained MLP model may update the biases and weights of the deployed MLP model. Those skilled in the art will recognize that post-deployment model updates may not be a one-time occurrence, but may occur as frequently as is suitable to improve the accuracy of the deployed model.

[0237] ML techniques may be used independently of each other or in combination. Different problems and / or datasets may benefit from using different ML techniques (e.g., combinations of ML techniques). Different training types for models may be more suitable for particular problems and / or datasets. The optimal algorithm (e.g., combinations of ML techniques) and / or training type may be determined for a particular use, problem, and / or dataset. For example, processes may be performed to select one or more of the following: select a data reduction type, select a model and / or algorithm configuration, determine the location of data reduction, determine the efficiency of the reduction and / or results, and / or other.

[0238] For example, an ML technique and / or combination of ML techniques may be determined for a particular problem and / or use case. Multiple data reduction and / or data analysis processes may be performed to determine accuracy, efficiency, and / or compatibility associated with a dataset. For example, a first ML technique (e.g., a first set of combined ML techniques) may be used on a dataset to perform data reduction and / or data analysis. The first ML technique may generate a first output. Similarly, a second ML technique (e.g., a second set of combined ML techniques) may be used on a dataset (e.g., the same dataset) to perform data reduction and / or data analysis. The second ML technique may generate a second output. The first output may be compared to the second output to determine which ML technique produced a more desirable result (e.g., a more efficient result, a more accurate result). Multiple ML techniques may be compared on the same dataset to determine the optimal ML technique to use on future similar datasets and / or problems.

[0239] In an example, in a medical context, a surgeon or medical professional may provide feedback to the ML technique and / or model used on the dataset. The surgeon may input the feedback into the weighted results of the ML model. The feedback may be used as input by the model to determine reduction methods for future analysis.

[0240] In examples, a data analysis method (e.g., an ML technique to be used in the data analysis method) may be determined based on the dataset itself. For example, the origin of the data may influence the type of data analysis method to be used for the dataset. Available system resources may be used to determine the data analysis method to be used for a given dataset. The magnitude of the data may be considered, for example, in determining the data analysis method. For example, the need for an external dataset for local processing levels or magnitude of operational response may be considered (e.g., small device changes may be made using local data, while large device operational changes may require global compilation and validation).

[0241] Such ML techniques may be applied to surgical information (e.g., a combination of the information flow of surgical information in Figure 7) to generate useful ML models.

[0242] 9 is a block diagram of an exemplary surgical system that may enable communication of information between one or more operating rooms 52000, 52010, 52020, a corresponding hospital local network 52030, an edge server 52035, and one or more other entities 52050.

[0243] In one example, each of the operating rooms 52000, 52010, 52020 may include a respective surgical computing device (e.g., surgical hubs 52005, 52015, 52025). The surgical hubs 52005, 52015, 52025 may include, for example, an example of a surgical computing device 704 disclosed herein, as illustrated. For example, the surgical hubs 52005, 52015, 52025 may include an example of a hub described in U.S. Patent Application Publication No. 2019-0200844(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Each surgical hub 52005, 52015, 52025 may be associated with one or more devices used during surgery, such as a surgical generator, intelligent surgical instruments, surgical robots, surgical displays, sensors, etc. Exemplary intelligent surgical instruments can include, for example, those described under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019-0200844(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the entire disclosure of which is incorporated herein by reference. Exemplary robotic systems may include those described in U.S. Patent Application Publication No. 2019-0201137(A1), entitled METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL, filed December 4, 2018 (U.S. Patent Application No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety. Such devices may be used in surgical procedures as part of a surgical system.

[0244] Such devices and corresponding surgical hubs 52005, 52015, 52025 can generate, process, transmit, and / or receive information, such as the surgical information disclosed in FIG. 7A. In one example, the surgical information can include information associated with one or more patient biomarkers (e.g., information disclosed in U.S. Patent Application No. 17 / 156,28, filed November 10, 2021, the disclosure of which is incorporated herein by reference in its entirety). This surgical information can be analyzed. For example, such analyses can include those described in U.S. Patent Application Publication No. 2019-0206569(A1), entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,403), the disclosure of which is incorporated herein by reference in its entirety.

[0245] Each patient may be undergoing a surgical procedure in each of the operating rooms 52000, 52010, 52020. As illustrated, patient A may be undergoing a surgical procedure in operating room A 52000. Patient B may be undergoing a surgical procedure in operating room B 52010. Patient C may be undergoing a surgical procedure in operating room C 52020. Surgical information generated, processed, transmitted, and / or received by each of the hubs 52005, 52015, 52025 may be associated with the patient undergoing surgery in the corresponding operating room. For example, surgical information associated with different patients in a common network and / or networked devices, such as a hospital local network 52030, an edge server 52035, and / or other entities, may pose data privacy issues and may encourage the use of data privacy protection techniques such as those disclosed herein.

[0246] Surgical information, such as patient-specific surgical information, may be communicated via common networks and / or networked devices, such as the hospital local network 52030, edge server 52035, and / or other entities. Illustratively, surgical information associated with a surgical procedure to be performed on patient A in operating room A 52000 may be communicated between the surgical hub device 51005 in operating room A 52000 and the edge server 52035 via the hospital local network 52030. Similarly, surgical information associated with a surgical procedure to be performed on patient B in operating room B 52010 may be communicated between the surgical hub device 51015 in operating room B 52010 and the edge server 52035 via the hospital local network 52030. Similarly, surgical information associated with a surgical procedure to be performed on patient C in operating room C 52020 can be communicated between the surgical hub device 51025 in operating room C 52020 and the edge server 52035 via the hospital local network 52030.

[0247] Such surgical information may have individuality characteristics (e.g., data individuality). Data individuality or data individuality level may represent how likely the surgical information is to be linked to an individual patient. For example, surgical information with a high data individuality level may be more likely to be traced back to a particular patient. For example, surgical information with a low data individuality level may be less likely to be traced back to a particular patient. Surgical information with a medium data individuality level may be more likely to be traced back to a particular patient.

[0248] Data individuality levels can be highly correlated with particular data types. For example, historical data (e.g., patient name, patient ID, surgical procedure date / time, etc.) and / or surgical information tagged with historical data can be associated with high data individuality. Similarly, data types associated with relatively general medical data (e.g., data types having values ​​common to many patients) can have low data individuality levels. For example, a patient's weight can be a data type with a low data individuality level (e.g., because many patients may have the same weight).

[0249] Data individuality levels can be correlated with the specificity of the data taken as a whole. For example, data elements viewed individually may have low data individuality to the extent that any such element taken alone is unlikely to reveal the patient from whom the data arose. However, such data elements, taken together as a whole, may be more likely to reveal the patient from whom the data arose. Such data elements, taken together as a whole, can exhibit high data individuality.

[0250] The data individuality level of a surgical dataset associated with a patient can reflect the patient-specificity of that subset. For example, a surgical dataset may have a high data individuality level because most or all of its subset may contain information that reveals the patient's source. In one example of this, a surgical dataset may have a high data individuality level because a small subset of the data has a relatively high likelihood of revealing the patient's source, and the remaining, larger, complementary subset of the data has a relatively low likelihood of revealing the patient's source.

[0251] The data individuality of surgical information may be altered (e.g., reduced). Anonymization techniques can be used to reduce the data individuality of surgical information. Anonymization techniques can include any logical processing of information that reduces the likelihood of identifying the patient's source. For example, anonymization techniques can include techniques such as editing, randomization, aggregation, and / or averaging. Editing can include excluding subsets of surgical data with high data individuality and retaining subsets of surgical data with low data individuality. In one example, redacting patient names and patient IDs from a dataset can reduce the data individuality of a dataset. Randomization can include altering certain aspects of noisy data to conceal the origin of the data without significantly altering the surgical and / or analytical value of the information. For example, randomizing time information for specific surgical information can help conceal the patient origin of such data without affecting the broader analytical value of the information given a larger population study. Data that averages a collection of common values ​​across similarly situated patients reduces the likelihood that such averages can be traced back to a particular patient.

[0252] In a system, a desired data individuality level may be related to the use and / or location of the data in the system. For example, the data individuality of surgical information being analyzed in an operating room during a patient procedure may remain unchanged. Here, a reduction in data individuality may be undesirable. Here, privacy concerns associated with such high data individuality are minimized because the use and / or location of the data in the system is localized to the patient's surgical procedure and operating room. For example, the data individuality level of surgical information being analyzed in a university and / or educational environment may be reduced. Here, a reduction in data individuality level is desirable because privacy concerns associated with such high data individuality are greater because the use and / or location of the data in the system is away from the patient's surgical procedure and operating room.

[0253] The hierarchy may be used to determine a desired data individuality level in a system. For example, a surgical system may have one or more hierarchical levels. The levels may be, for example, logical levels. The levels may be, for example, physical levels. The levels may be associated with corresponding data individualities, for example, based on their respective positions in the hierarchy. In one example, a use and / or location of surgical data that is more localized to the healthcare of a particular patient may have a level associated with a desired high data individuality. Additionally, a use and / or location of surgical data that is remote from the healthcare of a particular patient may have a level associated with a desired low data individuality level. Illustratively, the use of data and systems in conducting analytical studies across multiple patients and / or multiple surgical procedures may be remote from the healthcare of any one particular patient and, therefore, may be associated with a desired low data individuality.

[0254] The data individuality level may vary based on the location of a processing device in a system hierarchy where the surgical information may be processed or transmitted for processing, as described herein. In one example, if / when the surgical information is transmitted for processing from a local entity located inside the protective boundary to a remote processing device (e.g., a remote enterprise server) located outside the protective boundary, the data individuality level associated with the surgical information may be changed from a high data individuality level to a low data individuality level. The conversion of the individuality level of the surgical information from a high data individuality level to a low data individuality level may be performed using one of the anonymization techniques, as described herein.

[0255] In one example, for example, if surgical information is transmitted from a processing device located inside the protection boundary (e.g., a surgical hub) to a processing device located in an intermediate network with medium protection, the data personality may be converted from high data personality to medium data personality. The intermediate hierarchical level may be located within the medical professional's network but outside the protection boundary, as illustrated in FIG. 10.

[0256] Transforming the surgical information by changing its data individuality level may include anonymizing at least a portion of the surgical information or dataset. Surgical information, surgical dataset, or dataset may be used interchangeably herein. For example, by editing a subset of data points with a high data individuality level, thereby changing the data individuality level from a high data individuality level to a low data individuality level. In one example, changing the data individuality level may include processing the dataset (e.g., aggregating the dataset) into a format in which the data points of the dataset are aggregated or pooled into one overall dataset. Data points in the overall dataset may not be tied to individual datasets.

[0257] Edge processing can balance privacy and comprehensiveness using balancing protocols to package surgical data for sharing within different levels of the system hierarchy. Surgical datasets may experience allometry of data individuality (e.g., portions growing at different rates result in proportional changes). Surgical data allometry (e.g., growth or reduction in size of surgical data or surgical information) may be directly proportional to the level of protection afforded by the system hierarchy level. As a surgical data package (e.g., a surgical dataset) is processed and / or passed through different levels of the system hierarchy, it may change the size of the surgical data and the comprehensiveness of the surgical data. The growth or reduction of surgical data or surgical data portions (e.g., separable surgical data portions) is not linear. In one example, the growth or reduction of surgical data or surgical data portions may be proportional to the level of protection associated with the surgical data, for example, the level of protection afforded by surgical data protection rules (e.g., HIPAA laws) or the level of protection associated with the network on which the surgical data resides. In one example, a higher level of surgical data protection may result in a higher individuality of the surgical data points or surgical data sets.

[0258] The configuration of individual configuration data components may be based on the level of the data within the overall system hierarchy or the protection level of the system. In one example, data and / or algorithms may undergo assimilation and / or aggregation as data is pushed down from a higher level in the system hierarchy (e.g., a remote server) to a lower level in the system hierarchy (e.g., a surgical hub).

[0259] 9, data can maintain the same data individuality level (e.g., a high data individuality level that may include each of the data points in the data set) when transmitted to a processing device, such as an edge server 52035 located within a hospital local network 52030, where the network resides within a protection boundary 52045 (e.g., a Health Insurance Plan Portability and Accountability Act (HIPAA) protection boundary). In such a case, data with a high individuality level may be acceptable because it is less vulnerable to being traced back to the patient.

[0260] In one example, the local processing device may determine that the data should be processed on a processing device located outside the protective boundary of the medical facility's network instead of processing the data at an edge server 52035 located within the protective boundary of the hospital local network 52030. The data individuality level of the surgical information in such a case may be reduced (e.g., from a high data individuality level to a low data individuality level) before the surgical information is transmitted from a processing device located inside the medical facility's protective boundary 52045 (e.g., edge server 52035) to a processing device located outside the medical facility's protective boundary 52045 (e.g., remote server 52040).

[0261] Determining the identity of the data as it passes through different levels of the system hierarchy may be determined based on rule checks (e.g., HIPPA rule checks located in the analytics subsystem of the surgical hub / edge device). The rule checks may be implemented as a check on whether surgical information, or portions of surgical information, are associated with a patient and / or whether the surgical information can be traced back to a patient. In one example, the rule checks may be implemented using a machine learning model that can be trained to generate data identities based on analysis and / or comparison of data points within a surgical dataset. The utilized machine learning technique may be based on a supervised learning framework, for example, as described in FIG. 8A. The training data (e.g., training examples 802 as illustrated in FIG. 8A) can consist of a set of training examples (e.g., input data mapped to labeled outputs, as shown in FIG. 8A). The training data used in training the local machine learning model 52090 may include surgical datasets collected from previous surgical procedures, surgical parameters associated with those surgical procedures and / or simulated surgical procedures. The training data may include resource availability (e.g., memory and / or processing power availability) of various processing devices from previous surgical procedures, control algorithms associated with surgical instruments (e.g., stored locally or received from another entity, e.g., a remote server).

[0262] In one example, machine learning may be unsupervised (e.g., unsupervised learning), as described in FIG. 8B . In unsupervised learning, as illustrated in FIG. 8B , a framework-based machine learning model may be trained on a dataset that may include inputs to find structures or patterns within the data. For example, the inputs may include parameters associated with the dataset to be processed (e.g., the size of the dataset, acceptable latency values, etc.), a rule set (e.g., based on local privacy laws where the surgical procedure will be performed), and parameters associated with various potential processing devices to which the dataset may be sent for processing. The result may be identification of one or more processing devices and / or system hierarchical levels to which the dataset may be sent for processing, and / or a data individuality level that may be applied to the dataset before sending to the selected processing device. The data individuality level may be selected based on where the dataset is sent for processing.

[0263] In one example, a machine learning algorithm may be trained to determine the individuality level of data. For example, a histogram (or other method of estimating a probability distribution) may be generated to calculate the standard deviation of historical data. The deviation of a given data point from the mean value may then be compared to the standard deviation or other predetermined range to classify the data point at a predetermined data individuality level.

[0264] In one example, a machine learning model can assign risk to each of the data points in a dataset based on historical data on which the machine learning model can be trained. The model can, for example, suggest an overall data personality level to be applied to the dataset based on the accumulated risk of the data points in the dataset. This personality can be compared to a local applicable rule set to identify (1) the system hierarchy level and / or processing device to which the dataset may be sent for processing; and (2) the data personality level that may be applied to the dataset (e.g., before sending the dataset out of processing). The rule set can be derived from protection rules (e.g., HIPAA laws) that the medical facility where the surgical procedure is being performed may have to comply with.

[0265] In one example, the surgical hub / edge device may identify a processing device and / or system hierarchical level to which the surgical dataset may be sent for processing. The processing device and / or system hierarchical level may be identified based on, for example, the size of the surgical dataset (e.g., the size of the surgical dataset), the capabilities of the processing server, performance metrics associated with the dataset, etc. Capabilities and characteristics may be used interchangeably herein. In one example, the surgical hub / edge device may determine, for example, based at least on the size of the surgical dataset to be processed, that the surgical dataset should be processed on a remote server having higher processing power than that of the surgical hub or edge server. In such a case, the surgical hub / edge device may send the surgical dataset to the remote server. Based on the identification of the processing device and / or system hierarchical level, the surgical hub / edge device may perform a rule check to determine the data individuality level at which the dataset should be sent to the processing device.

[0266] In one example, the surgical hub / edge device can identify a processing device and / or system hierarchy level based on at least one of the processing device's capabilities, the data size of the surgical data, sensitivity to latency in processing the surgical data, the data individuality level of the surgical data, or the intended use of the surgical data. The identification of the processing device can be performed using one or more lookup tables, which may be combined with an optional ranking between the lookup tables. For example, a lookup table can associate the data size with the processing device's capabilities to identify a processing device suitable for a given data size. Similarly, the intended use of the data can be associated with the processing device's capabilities; for example, if the intended use is for patient treatment, this may be associated with a processing device having lower capabilities, while an intended use is analysis of the data in parallel with other similar data for trend or correlation analysis may be associated with a processing device with higher capabilities. Another lookup table can associate the data individuality level with the location of the processing device. For example, a processing device located inside a protective boundary may have a higher individuality level associated with it than a processing device located outside the protective boundary.

[0267] By combining lookup tables, data having lower capabilities and intended uses associated with lower personality levels can be sent to a higher capability processing device when the magnitude of the data requires. The processing device may be located outside the protective perimeter. The capabilities of the processing device can be increased when moved from the operating room, for example, with an operating room processing device (e.g., a surgical hub) having lower capabilities than the hospital processor, which has lower capabilities than the hospital network processing device, which has lower capabilities than the remote processing device.

[0268] In one example, the performance metric (e.g., along with the rule set) may be considered by the surgical hub / edge device to determine a processing device and / or system hierarchy level to which the data may be sent for processing. Determining the performance metric of the data may include using a simulation, which may output an approximation of the performance metric associated with the data. A simulation framework may be described in U.S. Patent Application No. 17 / 332,593, filed May 27, 2021, entitled "Method for Surgical Simulation," the disclosure of which is incorporated herein by reference in its entirety. In one example, based on a determination of whether the dataset to be processed is sensitive to latency (e.g., processing / transit delay), the dataset may be sent for processing to an edge server located within the healthcare provider's local network and therefore associated with a lower latency level, or to a remote server, as described herein, which may be associated with a higher latency level.

[0269] In one example, a surgical dataset is prepared to be sent to a processing device for processing, and the results may be utilized for a patient's post-operative follow-up, recovery, monitoring, etc. In such cases, the latency or time it takes to process the dataset may not be important. In such cases, the surgical hub / edge device may decide to send the surgical dataset to a remote server for processing based at least in part on the fact that latency is not a factor and / or the benefit of a diverse dataset on the remote server (e.g., a centrally located server).

[0270] In one example, the data magnitude of a surgical dataset may be associated with a data individuality level. The data magnitude may be used in determining a data individuality level that may be applied to the surgical dataset before transmission to a processing device for processing. In one example, a surgical dataset with a high data magnitude may be associated with a high data individuality level, and a low data magnitude may be associated with a low data individuality level.

[0271] Converting the data individuality level from one level to another may include anonymizing (e.g., redacting, randomizing, averaging, etc.) at least a portion of the surgical dataset. Anonymizing the surgical dataset may result in the surgical dataset being less likely or impossible to trace back to an individual patient. In one example, the local hub may determine to send a surgical dataset associated with a surgical procedure to a remote server 52040 based on the remote server 52040 being the best candidate for processing the data, as described herein. Based on this determination, the local hub may anonymize (e.g., redact, randomize, average, etc.) the data. For example, data associated with patient A 52005 may be randomized such that the randomized data cannot be traced back to patient A 52005.

[0272] As described herein, anonymization techniques such as data redaction, summarization, and / or editing may be used on surgical datasets when they are elevated to higher system hierarchy levels (e.g., cloud servers), in which case the level of data privacy protection may be reduced. In one example, when a surgical dataset is prepared to be transmitted to and / or shared with a processing device located at a higher system hierarchy level, data security may be considered, for example, by a machine learning algorithm. In one example, one or more parameters associated with a surgical dataset may be categorized with respect to their relevance or the need to make individual aspects viewable. In such cases, the system may combine certain individual surgical data points of the surgical dataset and average or summarize the surgical data points together within the surgical dataset (e.g., data structure), which may avoid losing trends and prevent individualization of datasets from particular patients. As described herein, portions of the data may be summarized and / or aggregated to generate pools of data that may be blended, homogenized, and / or aggregated, preventing individual components of the surgical dataset from being separated while allowing them to convey the same average results.

[0273] In one example, encryption (e.g., advanced encryption) may be used to protect the surgical data associated with the patent. The level of encryption used may depend on whether the surgical data set is being transmitted for processing to a device located within the protective boundary of the healthcare provider.

[0274] The determination of where to process a surgical dataset associated with a patient and / or medical professional may be based on the degree of benefit that can be obtained from the surgical dataset being processed at a particular hierarchical level. For example, a centrally located remote server 52040 may have access to diverse datasets that may be received from multiple locations of the same or different healthcare providers. The level of data diversity may be proportional to the degree of benefit that can be provided while processing the dataset. In one example, the remote server may be capable of analyzing a particular surgical dataset within a particular time frame. In one example, the determination of where to process a surgical dataset may be based on the speed at which the surgical dataset can be processed on a processing device located at a particular level of the system hierarchy (e.g., data sent to the remote server 52040 may be processed faster than data sent locally).

[0275] The data individuality level may vary based on the anonymization of some or all of the data points in the surgical dataset. Anonymization may include removing or modifying one or more data points from the surgical dataset, as described herein with respect to FIG. 10. The anonymization of surgical data points that may be anonymized may be associated with an assigned high risk, for example, as determined by a machine learning model located in the surgical hub. For example, a distinguishing characteristic data point may be associated with a high risk and therefore may be anonymized from the surgical dataset before transmitting the transformed surgical dataset to a processing device (e.g., a remote server). In one example, the same data point may be included in the surgical dataset when the surgical dataset is transmitted to a processing device (e.g., an edge server 52035) located within the hospital's local network 52030, which is within the protective perimeter 52045.

[0276] In one example, the surgical hub can weight the personalized surgical dataset against the privacy risk associated with the surgical dataset when determining the system hierarchical level at which the surgical dataset may be selected for processing. The privacy risk may be pre-configured and / or part of the machine learning model. In one example, the size of the surgical dataset may be derived based on the level of data individualization applied to the surgical dataset.

[0277] In one example, a surgical dataset generated within a medical facility's network (e.g., locally in a medical facility's operating room) can allow the surgical dataset to be checked based on protection rules (e.g., HIPAA laws). A surgical dataset transmitted from a medical facility's edge network to a remote server (e.g., a cloud server) may combine each of the surgical data points into a single output. In such a case, the transmitted surgical dataset can combine the distribution of surgical data across all patients in a way that cannot be tied to or traced back to a particular patient.

[0278] In one example, during a surgical procedure, a surgical dataset may be collected (e.g., locally within the medical facility for any follow-up, recovery, and / or monitoring) for each of the patient's biometrics, supplies used, complications, and / or outcomes. If the information is transmitted outside the medical facility, the data may be combined into one combined surgical dataset and transmitted to a remote server (e.g., the cloud, or any edge network that may not be part of the medical facility). The information may be transmitted outside the medical facility using distributions, ranges, minimums, and maximums, so that the combined surgical dataset may not be linked back to an individual patient.

[0279] FIG. 10 illustrates an example of determining data individuality levels based on the system hierarchy level to which surgical data may be sent for processing. As shown in FIG. 10 , the surgical data may be associated with patient A 52055 having a surgical procedure being performed on the patient in operating room A 52060, patient B 52065 having a surgical procedure being performed in operating room B 52070, and / or patient C 52075 having a surgical procedure being performed in operating room C 52080. The surgical data may be transmitted (e.g., transmitted via message) to a local surgical hub / edge device 52085. The surgical data may be generated from one or more surgical instruments located in each of the operating rooms. The surgical data may be generated based on measurements taken using sensors, actuators, robotic movements, biomarkers, surgeon biomarkers, visual aids, billing, etc. In one example, the surgical data to be processed may be generated based on a visual tracking system located within each of the operating rooms. For example, the visual tracking system may include a facial recognition system that may generate data related to the status of the patient and / or surgeon during a surgical procedure.

[0280] The surgical data transmitted from surgical instruments in the operating room to individual local surgical hubs may be in raw form (e.g., without any processing performed on it). The raw measurement data may be converted into data points by the local surgical hub. A machine learning model 52090 and / or an analysis subsystem 52095 that are part of the local surgical hub / edge device 52085 may be used to predict the location of a processing device (e.g., a processing device within a system hierarchy level) to which the surgical data may be transmitted for processing. For example, the local hub 52085 may decide to transmit the surgical data to a processing device (e.g., an edge server 52100) located within the hospital's local network. The hospital's local network may be part of the protection boundary 52105. In such a case, the local hub 52085 may transmit surgical data having high data individuality and data magnitude to a server 52100 located within the protection boundary 52105.

[0281] As described with respect to FIG. 10 , data transmitted from the surgical hub / edge device 52085 to the edge server 52100 may be organized into one or more surgical datasets. A surgical dataset may include surgical data points (e.g., parameters associated with a patient, a healthcare provider, and / or a surgical instrument) 1, 2,...N, where N is a finite number. Surgical data points 1-N may be associated with patient A 52055, B 52065, and / or C 52075. In one example, a surgical dataset having surgical data points 1-N may be associated with high data individuality due to a high risk that surgical data points 1 and 2 will be linked back to patient A 52055. As illustrated in FIG. 10 , if the surgical data is being transmitted to a processing device (e.g., edge server 52100) located within the protective boundary 52105, surgical data points 1 and 2 may be included in that surgical dataset. In one example, surgical data points 11-20 may be associated with patient B 52065, and surgical data points 19 and 20 may be of a type that may pose a risk of patient B being tracked. Because the surgical data was transmitted within the protective boundary 52105, the surgical data may be included in the surgical dataset. In one example, surgical data points 30-40 may be associated with patient C 52075. Surgical data points 35 and 36 can be traced to patient C 52075. Because this surgical data was transmitted within the protective boundary 52105, it may be included in a dataset that may be transmitted to the edge server 52100.

[0282] In one example, the local surgical hub / edge device 52085 may determine that surgical datasets, such as those associated with patient A 52055, patient B 52065, and / or patient C 52075, may be transmitted for processing to a processing device (e.g., server 52110) that may be located within an intermediate system hierarchy level. The intermediate system hierarchy level 52110 may be associated with a semi-protected boundary 52115. The server located at the intermediate system hierarchy level 52110 may have moderate processing power when compared to the local server 52100 (e.g., having minimal processing power) and the remote server 52200 (e.g., having the most processing power). In one example, the server may be located within an extended medical facility network. For example, a medical facility may have agreements with several partner medical facilities regarding sharing patient data. In such a case, the network shared by these hospitals may be considered within the semi-protected boundary 52115. The surgical datasets transmitted to a server within this network may adhere to a moderate data individuality level. A surgical dataset with a moderate individuality level may have less individuality than a surgical dataset located within the protected boundary of a medical facility and may have more individuality than a surgical dataset that may be transmitted outside the protected / intermediate boundary. Different individuality levels may be achieved by anonymizing the data (e.g., editing, randomizing, averaging, etc.) as described herein.

[0283] 10 , the surgical data set transmitted to the intermediate system hierarchy level 52110 can include, for example, M of N surgical data points, where M is less than N (e.g., N is the total number of surgical data points generated within the protected boundary 52105 of the medical facility). The excluded or anonymized surgical data points may be surgical data points that may have a high risk of being traced to an individual patient. Surgical data format and surgical data individuality may be used interchangeably herein.

[0284] In one example, as illustrated in FIG. 10, among the surgical data points 1 to N associated with patient A 52055, surgical data points 1 and 2 can have high individuality and can reveal information that can be traced back to patient A 52055. In one example, surgical data point 1 may be, for example, the patient's name, patient ID, identification of the surgical procedure performed on the patient, etc. In such a case, due to the high level of individuality, surgical data point 1 can be edited before the surgical data set of which it is a part is sent for processing to any of the processing devices located outside the protection boundary 52105. In one example, surgical data point 2 can be associated with the patient's physical characteristics, such as height, weight, etc. In such a case, surgical data point 2 can be considered to have a low likelihood of being traced back to patient A 52055 and can be sent in non-anonymized form to a device located at an intermediate level outside the protection boundary 52105, for example, within the network 52115 of healthcare facilities. As illustrated in FIG. 10, in this case, the size M of the data including the number M of surgical data points (which is the number N of data points minus the anonymized data point 1 and thus not available to the processing device for analysis) can be smaller than the number N of the size of the data.

[0285] In one example, the size of the data and / or the data individuality level associated with a hierarchical level can be related to the proportion of algorithms used to process the data at that hierarchical level. For example, in the surgical hub / edge device 52085, the proportion of algorithms used to process the surgical data points 1 to N with higher data individuality can be higher than the proportion of algorithms used to process the surgical data points 1 to M (where M < N) in a server 52115 located within the intermediate level, for example, within the network 52115 of healthcare facilities but outside the protection boundary 52105.

[0286] In one example, the surgical hub / edge device may determine to transmit the surgical dataset to a remote server 52200 located outside the protection boundary 52105 and the intermediate boundary 52115. The local surgical hub / edge device 52085 may identify the processing device using a machine learning model 52090 and / or an analysis subsystem 52095, as described herein. For example, the machine learning model may identify the remote server 52200 based at least on the diversity of the dataset available on the remote server 52200, performance metrics associated with the data, etc., as described herein. In such a case, in addition to anonymizing surgical data point 1, the surgical hub / edge device 52085 may also anonymize surgical data point 2 before transmitting both surgical data points to the remote server 52200 for processing. As illustrated in FIG. 10 , in this case, the size of the data X, including the number of surgical data points X (data point N−2), may be less than the size of the data M(N−1), which may be less than the size of the data N. As illustrated in FIG. 10, as surgical data sets associated with a patient may be transmitted to various processing devices for processing, the size of the data may be large or small based on the level of protection afforded by the hierarchical level in which the processing device is located or the data individuality level associated with that hierarchical level.

[0287] In one example, as described herein, a surgical hub / edge device 52085 may transmit a surgical data set of size M(N-1) to a processing device (server 52110) located at an intermediate hierarchical level and / or associated with an intermediate personality level. The server may transmit the surgical data to a remote server 52200 for further processing. In such a case, the server 52110 may further anonymize the surgical data set, for example, by randomizing data point 2 before transmitting a surgical data set of size X(N-2, where X<M<N) to the remote server 52200. In this case, the proportion of the algorithm used to process the lower data personality surgical data points 1 to X (e.g., at the remote server 52200) may be smaller than, for example, the proportion of the algorithm used to process the surgical data points 1 to M (where X<M<N) at a server 52115 located within the intermediate hierarchical level.

[0288] In one example, surgical data points 1-10 may be associated with patient A 52055, and surgical data point 1 and surgical data point 2 may be traceable back to patient A 52055. In one example, surgical data point 1 may be removed and data point 2 may be anonymized. In one example, these surgical data points may be completely anonymized (e.g., completely redacted, randomized, averaged, etc.) where they cannot be traced back to patient A 52055. In one example, the surgical data points in dataset X may be aggregated to a level where the surgical data cannot be traced back to any of patient A 52055, B 52065, and / or C 52075. In one example, surgical data points 10-20 may be associated with patient B 52065, and surgical data points 19 and 20 may be unique to B 52065 and may be traced back to patient B 52065. Both surgical data points 19 and 20 may be redacted. In one example, surgical data point 20 may be completely anonymized before being sent for processing within a surgical dataset. Surgical data points 30-40 may be associated with patient C 52075. Surgical data points 35 and 36 may be unique to patient C 52075 or may be traced back to patient C 52075. Both data points may be removed (e.g., redacted). In such a case, the transformed surgical data may be associated with low data individuality and low data magnitude.

[0289] In one example, mathematical operations can be used to manipulate the surgical data to change the data identity (e.g., to remove the risk that the surgical data will be associated or linked to a patient). For example, the mean and / or median can be taken among the surgical data points. Some of the surgical data points within the surgical dataset may be manipulated to the point where they cannot be linked back to an individual patient, while other surgical data points within the surgical dataset may remain unchanged. This can reduce the data identity associated with the surgical dataset while allowing the surgical dataset to be transmitted to either the intermediate system hierarchy level 52110 or the remote level 52200.

[0290] 11 illustrates one example of a surgical system in which measurements taken within an operating room are received for processing by one or more respective surgical hub / edge devices. As illustrated in FIG. 11, the surgical hub 52225 may include, among other things, a processor 52235, a memory 52240 (e.g., non-removable memory and / or removable memory), an analysis subsystem 52230, a machine learning model 52220, and / or a storage subsystem 52245. It will be understood that the surgical hub 52225 may include any sub-combination of the foregoing elements / subsystems while remaining consistent with an embodiment.

[0291] The processor 52235 in the surgical hub 52225 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), state machine, etc. The processor 52235 may perform data processing of surgical information that may be received from various surgical devices and instruments attached to the surgical hub. The processor 52235 may perform data processing, authentication, input / output processing, and / or any other function that may enable the surgical hub 52225 to operate in an environment suitable for performing a surgical procedure. The processor 52235 in the surgical hub 52225 may be coupled to a transceiver (not shown). The processor 52235 in the surgical hub 52225 may communicate with other edge servers and / or remote servers using a transceiver, as described with respect to Figures 9 and 10.

[0292] The processor 52235 in the surgical hub 52225 can access information from and store data in any type of suitable memory (e.g., non-removable and / or removable memory). Non-removable memory may include random-access memory (RAM), read-only memory (ROM), a hard disk, a solid-state drive, or any other type of memory storage device. Removable memory may include secure digital memory.

[0293] The processor 52235 in the surgical hub 52225 can access information from and store data in the extended storage 52245 (e.g., non-removable memory and / or removable memory). In one example, the processor 52235 in the surgical hub 52225 can process data points associated with a patient, determine a risk level associated with the data points, and apply a personality level associated with the risk level and / or a hierarchical level to which the data points can be sent for further processing.

[0294] As described with respect to FIG. 10 , the surgical dataset may include multiple surgical data points. The surgical data points may be obtained from measurement data associated with a patient, a medical professional, etc. For example, the surgical data points may be associated with measurements obtained from sensors, actuators, robotic movements, patient biomarkers, surgeon biomarkers, visual aids, etc. Wearable devices may be used for those measurements. Wearable devices or wearables are described in more detail in U.S. Patent Application No. 17 / 156,28, filed November 10, 2021, under the heading "Monitoring Of Adjusting A Surgical Parameter Based On Biomarker Measurements," the entire disclosure of which is incorporated herein by reference. Each surgical data point may have a data individuality level associated with it. The data individuality level may be associated with a risk level. The risk level may indicate whether the surgical data point can be tracked or linked back to a patient. An overall risk level may be attributed to the surgical dataset. The overall risk level may be based, among other things, on an aggregation of the risk levels of each of the surgical data points in the surgical dataset.

[0295] In one example, the measurement may be associated with one of many actuators located in an operating room. For example, the measurement may be generated based on potentiometer readings located on surgical instruments used by surgeons performing surgery on patients, e.g., patient A 52205, patient B 52210, and / or patient C 52215, located in respective operating rooms, as shown in FIG. 11 . The enhancement factor readings received by the local surgical hub / edge device 52225 may then be provided to a machine learning model 52220 located within the local surgical hub / edge device 52225. The machine learning model 52220 may be trained to associate the potentiometer readings with a risk level (e.g., a low risk level). For example, the machine learning model 52220 may determine that the potentiometer readings are unlikely to be linked back to an individual patient and may therefore be associated with a low risk level. Thus, for example, a surgical dataset including potentiometer readings may be associated with an overall low risk level and may be transmitted by the local surgical hub / edge device 52225 to an intermediate system hierarchy level or a remote server for further processing.

[0296] In one example, one of the surgical data points of the surgical dataset may be the patient's cortisol level. The surgical data point may be generated or calculated based on measurements obtained from a wearable that may be worn by the patient during the surgical procedure. For example, the patient may wear a watch that can determine the patient's cortisol level based on sweat readings produced by the patient. The data point may be generated by a surgical instrument or a local surgical hub / edge device 52225. The local surgical hub / edge device 52225 may determine that the cortisol level may uniquely identify the patient and may assign a risk level (e.g., a high risk level) to the surgical data point. The local surgical hub / edge device 52225 may assign a risk level to the surgical data point using a machine learning model 52220. The machine learning model 52220 may recommend removing or anonymizing the cortisol data point before transmitting it to a device that may be located outside the protective boundary. The input to the machine learning model may be a surgical data point that may be generated in an operating room, and the output of the machine learning model may be an identification of a processing device and / or system hierarchy level to which the surgical data point or a surgical dataset including the surgical data point may be sent for processing.

[0297] 12 illustrates an example of a transformation of surgical data parameters associated with a patient based on data personality and system hierarchy level. In 52250, a surgical device (e.g., a surgical hub) may receive multiple surgical data parameters associated with a patient. The multiple surgical data parameters may be of a first data magnitude (e.g., data size) and a first data personality level.

[0298] In 52255, the surgical device may identify a processing device for processing the plurality of surgical data parameters. The processing device may be identified based on one or more of a first surgical data individuality level, a magnitude of the first surgical data, a sensitivity to latency in processing the surgical data parameter, an intended use of the first surgical data parameter, a characteristic or a rule set of the first processing server.

[0299] In 52260, the surgical device can convert the plurality of surgical data parameters into a transformed plurality of surgical data parameters, where the transformed plurality of surgical data parameters are at a second surgical data individuality level and a second surgical data magnitude. In one example, the second surgical data individuality level may be lower than the first surgical data individuality level. The conversion of the first plurality of surgical data parameters can include anonymizing the plurality of surgical data parameters, or anonymizing a subset thereof. The anonymizing can include at least one of editing, randomizing, aggregating, ranging, or averaging.

[0300] At 52265, the transformed plurality of surgical data parameters are transmitted to the processing device identified at 52250 for processing.

[0301] With reference to FIG. 13 , an overview of a surgical system may be presented. Surgical instruments may be used in a surgical procedure as part of the surgical system. A surgical hub / edge device may be configured to coordinate information flow to the surgical instruments (e.g., a display on the surgical instrument). For example, a surgical hub / edge device may be described in U.S. Patent Application Publication No. 2019-0200844(A1), entitled “METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY,” filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Exemplary surgical instruments suitable for use with the surgical system are described, for example, under the heading "Surgical Instrument Hardware" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety.

[0302] FIG. 13 shows an example overview of data transmission to multiple system hierarchy levels. A surgical hub / edge device 52700 may be used to perform a surgical procedure on a patient in a surgical room 52705. A robotic system may be used as part of a surgical system in a surgical procedure. For example, a robotic system may be described in U.S. Patent Application Publication No. 2019-0200844(A1), entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. A robotic hub may be used to process images of the surgical site and then display them to the surgeon through the surgeon's console.

[0303] Other types of robotic systems may be readily adapted for use with the surgical system. Various examples of robotic systems and surgical tools suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0201137(A1), entitled "METHOD OF ROBOTIC HUB COMMUNICATION, DETECTION, AND CONTROL," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,407), the disclosure of which is incorporated herein by reference in its entirety.

[0304] Various examples of cloud-based analytics performed by the cloud and suitable for use with the present disclosure are described in U.S. Patent Application Publication No. 2019-0206569(A1), entitled "METHOD OF CLOUD BASED DATA ANALYTICS FOR USE WITH THE HUB," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,403), the disclosure of which is incorporated herein by reference in its entirety.

[0305] In various aspects, an imaging device may be used in a surgical system and may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, charge-coupled device (CCD) sensors and complementary metal-oxide semiconductor (CMOS) sensors.

[0306] The optics of the imaging device may include one or more illumination sources and / or one or more lenses. The one or more illumination sources may be directed to illuminate a portion of the surgical field. The one or more image sensors may receive light reflected or refracted from the surgical field, including light reflected or refracted from tissue and / or surgical instruments.

[0307] The one or more illumination sources can be configured to emit electromagnetic energy in the visible spectrum as well as the invisible spectrum. The visible spectrum, sometimes referred to as the optical spectrum or luminous spectrum, is the portion of the electromagnetic spectrum that is visible to (e.g., detectable by) the human eye and is sometimes referred to as visible light or simply light. The typical human eye responds to wavelengths in air between about 380 nm and about 750 nm.

[0308] The invisible spectrum (e.g., non-radiative spectrum) is the portion of the electromagnetic spectrum located below and above the visible spectrum (i.e., wavelengths less than about 380 nm and greater than about 750 nm). The invisible spectrum is not detectable by the human eye. Wavelengths greater than about 750 nm are longer than the red visible spectrum, which constitutes invisible infrared (IR), microwave, and radio electromagnetic radiation. Wavelengths less than about 380 nm are shorter than the violet spectrum, which constitutes invisible ultraviolet, x-ray, and gamma-ray electromagnetic radiation.

[0309] In various aspects, the imaging device may be configured for use in minimally invasive procedures. Examples of imaging devices suitable for use with the present disclosure include, but are not limited to, arthroscopes, angioscopes, bronchoscopes, cholangioscopes, colonoscopes, cystoscopes, duodenoscopes, enteroscopes, esophagogastroduodenoscopes (gastroscopes), endoscopes, laryngoscopes, nasopharyngological-nephroscopes, sigmoidoscopes, thoracoscopes, and ureteroscopes.

[0310] The imaging device may use multispectral monitoring to distinguish between topography and underlying structures. Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. Wavelengths can be separated by filters or by using instruments sensitive to specific wavelengths, including frequencies beyond the visible light range, e.g., IR and UV light. Spectral imaging can extract additional information that cannot be captured by the red, green, and blue receptors of the human eye. The use of multispectral imaging is described in more detail under the heading "Advanced Imaging Acquisition Module" in U.S. Patent Application Publication No. 2019-0200844(A1) entitled "METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPLAY," filed December 4, 2018 (U.S. Patent Application No. 16 / 209,385), the disclosure of which is incorporated herein by reference in its entirety. Multispectral monitoring can be a useful tool for repositioning the surgical field after the surgical task is complete to perform one or more of the tests described above on the treated tissue. It is self-evident that strict sterilization of the operating room and surgical equipment is necessary during any surgical procedure. The strict hygiene and sterilization conditions required in an "operating room," i.e., an operating room or procedure room, require the highest possible sterility of all medical devices and equipment. Part of the sterilization process is the need to sterilize everything that comes into contact with the patient or enters the sterile field, including the imaging device and its accessories and components. It will be understood that the sterile field may be considered a specific area, such as in a tray or on a sterile towel, that is deemed free of microorganisms, or the sterile field may be considered the area immediately surrounding the patient being prepared for the surgical procedure. The sterile field may include appropriately clothed and hand-washed team members, as well as all equipment and fixtures in the area.

[0311] As shown in FIG. 13 , a surgical hub / edge device 52700 may be associated with and / or located within a surgical operating room 52705. The operating room 52705, other than the surgical hub, may also include one or more surgical instruments and surgical devices. The surgical instruments and surgical devices may be used (e.g., autonomously or manually by a surgeon) to perform a procedure on a patient. For example, the surgical device may be an endocutter. The surgical device may communicate with the surgical hub / edge device 52700, which may be located in or near the operating room 52705. The surgical hub / edge device 52700 may instruct the surgical device regarding information related to the procedure being performed on the patient. In one example, the surgical hub / edge device 52700 may set configuration parameters for the surgical instrument or surgical device by sending a message to the surgical instrument or surgical device. For example, the surgical hub / edge device 52700 may send surgical device information indicating the firing rate of the endocutter to be set at or during the stage of the procedure. The message may be sent to the surgical instrument in response to the surgical instrument sending a request message for the instrument to the surgical hub / edge device 52700.

[0312] Surgical information related to the procedure may be generated. For example, the information may be based on the performance of a surgical instrument. For example, the data may be associated with physical measurements, physiological measurements, and / or the like. Measurements are described in more detail in U.S. Patent Application No. 17 / 156,28, filed November 10, 2021, under the heading "Monitoring Of Adjusting A Surgical Parameter Based On Biomarker Measurements," the disclosure of which is incorporated herein by reference in its entirety.

[0313] Surgical information associated with a surgical procedure being performed in the operating room may be transmitted to a local surgical hub / edge device 52700. In one example, surgical information associated with measurements taken from a surgical display during a surgical procedure may be transmitted to the surgical hub / edge device 52700, where the surgical information may be further analyzed (e.g., analyzed by the analysis subsystem 52710).

[0314] As shown in FIG. 13 , the surgical hub / edge device 52700 can track the progress of surgical steps in a surgical procedure and can adjust the function of surgical instruments based on such progress indicated by the surgical procedure plan 52715. The surgical hub / edge device 52700 can determine the surgical steps (e.g., surgical steps 1, 2, through K) associated with the surgical procedure plan 52715. In one example, the surgical procedure tracked by the surgical hub / edge device 52700 may be a colectomy. The surgical procedure plan 52715 for a colectomy may include various surgical steps, including, for example, mobilization of the colon. The surgical procedure plan 52715 may be obtained by the surgical hub / edge device or may be manually entered by a healthcare provider, such as a surgeon. The surgical steps associated with the colectomy may be performed by one or more surgical instruments associated with the surgical hub / edge device 52700 and located in the operating room 52705. In one example, each of the surgical instruments may perform a respective task associated with a surgical step. The surgical instruments may perform the surgical step autonomously. How surgical instruments operate autonomously is described in more detail in U.S. Patent Application No. 17 / 747,806, filed May 18, 2022, under the heading "METHOD OF CONTROLLING AUTONOMOUS OPERATIONS IN A SURGICAL SYSTEM," the disclosure of which is incorporated herein by reference in its entirety.

[0315] Surgical instruments involved in the performance of a surgical step may generate surgical data or surgical information associated with the surgical step. The terms data, surgical data, surgical dataset, surgical information, and surgical metric set may be used interchangeably herein. Data or surgical data may include, for example, data associated with the surgical hub / edge device 52700, surgical instruments, data associated with a patient or medical professional, and / or the performance of a surgical step, as described herein. Surgical information or surgical data may be described in more detail in U.S. Patent Application No. 17 / 156,28, filed November 10, 2021, under the heading "Monitoring of adjusting a surgical parameter based on biomarker measurements," the disclosure of which is incorporated herein by reference in its entirety. Surgical data may include data type, data characteristics, and performance metrics. Surgical data characteristics may be associated with how sensitive the data (e.g., data format and / or identity) is (e.g., in other words, what the risk is of the data being traced back to an individual patient). For example, highly sensitive surgical data may be more likely to be tied to an individual patient. Such surgical data may not be transmitted outside the protective boundary 52720.

[0316] Health data is a special category of personal data that, due to its perceived content, receives a higher level of protection (see Article 9 GDPR or HIPPA Privacy Rule), requiring heightened security considerations. A breach of sensitive personal data may result in the accidental or unlawful destruction, loss, alteration, unauthorized disclosure of, or access to, sensitive data, which can have serious personal consequences. For example, the permanent deletion of a person's medical records could potentially have serious and long-lasting consequences for that person's health.

[0317] Surgical data processing may be hybridized based on location, e.g., the location where the data is generated. Data hybridization may include processing portions of the surgical data locally (e.g., on the surgical hub / edge device 52700 or local network processing) using one or more fog computing devices and / or using cloud processing. Cloud processing may include, for example, analysis of surgical datasets that may be larger than the datasets analyzed by the edge server.

[0318] The surgical data may be transmitted (e.g., in a surgical dataset) to entities (e.g., entities having processors) located at different system hierarchy levels, as further described with respect to FIGS. 13 and 14A . The systems and / or subsystems at various hierarchy levels may be divided based on one or more of location (e.g., whether the system or subsystem is inside or outside the protection boundary 52720), processing capability (e.g., processing power), available memory (e.g., memory size and / or type), etc. The various hierarchy levels may include: (1) a surgical hub system; (2) an edge or fog network system; and (3) a cloud enterprise server system. In one example, the surgical hub system and the edge system may be located at the same hierarchy level. A surgical hub / edge device system including a surgical hub / edge device 52700, surgical devices and / or surgical instruments, etc. may be located within an operating room 52705. The edge or fog networking system may include an edge server. The edge or fog networking system may include server systems that may be co-located within a healthcare facility and / or distributed within a network of healthcare facilities. As illustrated in Figure 13, the surgical hub / edge device system and the edge or fog networking system may be located within a protective boundary 52720, for example, a protective boundary under HIPAA law. The enterprise cloud server system 52730 may include one or more enterprise cloud servers.

[0319] The surgical hub / edge device 52700 may determine a processing device within a system hierarchy level that may be suitable for processing the surgical dataset or a portion or sub-block of the surgical dataset. The surgical hub / edge device 52700 may transmit the surgical dataset to the determined processing device. For example, the surgical dataset 52725 may be transmitted locally to an edge server 52735 located within the protection boundary 52720, for example, for processing. In one example, the surgical dataset 52725 may be transmitted to an enterprise cloud server 52730, which may be located outside the protection boundary 52720. In one example, the surgical dataset 52725 may be transmitted to a server located within an intermediate system hierarchy level. For example, the intermediate system hierarchy level may be a location within the hospital network but not within the protection boundary 52720.

[0320] The proportion of surgical data to be processed at various hierarchical levels may be determined using system conditions, parameters associated with the surgical data to be processed, and / or results associated with the surgical data. System conditions, such as specific system conditions or required patterns, may be utilized to determine the location at which surgical data may be processed or transmitted for processing. The system conditions and / or patterns may be utilized to determine the extent to which surgical data should be processed at various hierarchical levels of the system. For example, a radio frequency surgical dataset may be modified (e.g., decimated) to transmit (e.g., only transmit) a portion (e.g., a useful portion) of the surgical data for processing at a different system level in the hierarchy of the system. In one example, a sub-block portion of a surgical dataset may include a calculated impedance band instead of a complete set of voltage and current samples.

[0321] In one example, the parameters of the algorithm (e.g., only the parameters) may be transferred back to the main repository. The parameters may be used by the ML model 52740 to improve tissue characterization and performance. The ML model 52740 may be executed within a smart device (e.g., a smart instrument or smart surgical hub / edge device 52700), edge computing device, or fog computing device (not shown), which may be located within the protected perimeter 52720 of the healthcare facility. In one example, the processing power of the edge or fog computing device may be lower than that of a cloud-based server or enterprise server.

[0322] In one example, an ML model, e.g., a lite version of a local ML model, may be used on the smart surgical hub / edge device 52700 or a fog computing device or edge computing device. The local ML model may be utilized to compute fewer and / or simpler calculations, e.g., using a device with lower processor power than a cloud-based server device. For example, gradient-enhanced Kriging surrogate modeling may be utilized to provide a low-computational-cost mechanism for evaluating processor-intensive functions. Gradient-enhanced Kriging models may be utilized to reduce the number of function evaluations for a desired accuracy when efficient gradient calculations, such as adjoint methods, are available. Such gradient-enhanced Kriging models may be executed on the smart surgical instrument itself to predict outputs. In one example, the gradient-enhanced Kriging model may be executed on the smart surgical hub / edge device 52700 or a fog computing device.

[0323] In one example, the machine learning model and / or the trained machine learning model may be utilized as part of a supervised learning framework. Supervised learning models are described herein in FIG. 8A . The training data (e.g., training examples 802 as illustrated in FIG. 8A ) may consist of a set of training examples (e.g., input data mapped to labeled outputs, as shown in FIG. 8A ). The training data used in training the local machine learning model 52515 may include surgical data collected from previous surgical procedures and / or simulated surgical procedures. The training data may include attributes or parameters associated with patients and / or parameters associated with surgical instruments. In one example, the local ML model as an output may provide a measurable outcome associated with a surgical procedure. For example, the ML model may be utilized to detect low-risk interpretations, including, for example, predicting that hemostasis may be required during a colorectal surgical procedure, predicting postoperative leaks after a surgical procedure (e.g., a colorectal surgical procedure), predicting postoperative air leaks after a thoracic surgical procedure, etc. These predictions may be made based on various surgical data inputs, including, for example, whether the patient received radiation prior to the surgical procedure and / or whether the patient took certain types of medication. One or more of the attributes associated with the patient may be edited before transmitting the surgical data to an enterprise cloud server location for further processing. In one example, the attributes selected for editing may be performed in a manner that has minimal impact on the measured results.

[0324] In one example, the compression parameterization mechanism can be utilized by a system (e.g., a system located at a lower hierarchical level) to filter and compress correlated data or filter data that may not have a high probability of affecting the measured results. A system or device located at a lower hierarchical level can perform compression parameterization of the surgical data, for example, before transmitting to a higher level for further processing. The compression parameterization of the surgical data can be performed based on one or more of the following: limitations of communication, memory storage, processing resources, etc. by one or more higher level systems.

[0325] In one example, surgical data collected at a device or system located at a lower hierarchical level may be reduced before being transferred to the next hierarchical level (e.g., a higher hierarchical level). For example, as illustrated in FIG. 14A , surgical data collected at a surgical instrument 52780 or processed at a surgical computing device 52700 or edge server 52785 located within the protective boundary 52720 may be reduced before being sent to the next hierarchical level (e.g., an enterprise cloud server 52730 located outside the protective boundary 52720).

[0326] Locally compiled parameterization, signal processing and / or data reduction may be performed at a lower (e.g., lowest) branch of the hierarchical tree (e.g., acquisition device or smart instrument). The lower branch may be the smart surgical instrument 52780 or the surgical computing device 52700. Selective data parameterization, signal processing and / or surgical data reduction may be performed based on at least one of the following: processing limitations of the next hierarchical level, importance of the surgical data, surgical data that may have minimal or no impact on the measured outcome or outcome, risk or severity of the surgical data or its related matters, time to event (e.g., failure, technical irregularity, communication problem, etc.).

[0327] In one example, a surgical instrument 52780 or surgical subsystem located at a lower level in the computational hierarchy may perform data decimation before transferring the surgical data to a device or subsystem located at the next or higher level in the computational hierarchy, such as the surgical computing device 52700 or edge server 52785. In one example, decimating the data may include removing every 10th data point in a surgical dataset. In the case of signal processing, decimating by a factor (e.g., a factor of 10) may include saving / retaining every 10th sample. Specialized, dedicated, and / or customized processing units (e.g., application specific integrated circuit (ASIC)-based processing units or reduced instruction set computing (RISC)-based processing units) may be used within such devices (e.g., the end effector, shaft, or handle of an instrument) to decimate the surgical data and / or process / condition signals so that the output from such computing device (e.g., only the output from such device) can be handled by another computing device located at a higher level in the computational hierarchy.

[0328] Intermediate-level devices or systems may reduce and / or limit the data transferred up a computational hierarchy level based on communication parameters, network conditions to the next node in the computational hierarchy (e.g., the next-highest node in the computational hierarchy level), and / or the processing capabilities of systems higher in the computational hierarchy. For example, the surgical computing device 52700 may reduce and / or limit the surgical data transferred to the enterprise cloud server 52730 based on the link conditions between the surgical computing device 52700 and the enterprise cloud server 52730. Data reduction and / or limitation in intermediate-level computational hierarchy systems may provide combined parameter or parameter data by eliminating or limiting surgical data or portions of surgical data that may have minimal or no impact on the measured results. Data reduction and / or limitation may be performed based on the direction of decomposition of data that has major trends but inconclusive results. This may result in the discovery of high-signal patterns or relationships between data (e.g., at the expense of more detailed interactions) to maximize benefits in cost, time, bandwidth, and / or processing resources.

[0329] In one example, edge computing processes running on edge devices 52785 or fog computing devices present in the medical facility's network 52720 may be utilized to provide local edge processing of data, for example, using artificial intelligence. In one example, federated learning may be utilized to enable collaborative training of machine learning models on edge devices. Edge computing may process data away from centralized storage or cloud servers 52730 and may retain information about local portions of the network edge device 52785. Surgical data transmitted to the edge device 52785 or fog computing device may be processed directly on the device without, for example, transmission to a centralized enterprise cloud server 52730. Processing surgical data on the edge server device 52785 or fog computing device may mean minimal or no delay in data processing. Data may be stored at the edge of the network, for example, an Internet of Things (IoT) network, and may be processed immediately.

[0330] In one example, the edge device 52785 or fog computing device may be utilized to perform real-time data analytics on data that the edge device 52785 or fog computing device may receive from a smart device or smart surgical instrument lower in the computational hierarchy or a device or system higher in the computational hierarchy than the edge device 52785 or fog computing device. The edge device 52785 or fog computing device may be utilized to process a significant amount of data, which may be received from a smart computing device or smart surgical instrument lower in the computational hierarchy or a device or system higher in the computational hierarchy than the edge device 52785 or fog computing device. The edge device 52785 or fog computing device may have the capability to process the data immediately.

[0331] In one example, network congestion may be minimal between surgical computing devices 52700 or surgical instruments 52780 that are lower in the computational hierarchy than the enterprise server 52730. Such edge devices 52785 or fog computing devices may be utilized (e.g., utilized first) to process data locally (e.g., at the edge device 52785 or fog computing device) and send the processed data to main storage (e.g., storage at the enterprise server 52730). In one example, various prioritized data types may be sent in order to the edge devices or fog computing devices for processing, for example, based on a priority value associated with each of the data types.

[0332] In one example, a device or surgical instrument 52780 lower in the computational hierarchy than the edge device 52785 or fog computing device (e.g., with limited resources and / or long downtime) can utilize the edge device 52785 or fog device to pre-process or fully process its data. The edge device 52785 or fog computing device can send results (e.g., results in a simpler conclusion format) back to the surgical device or surgical instrument 52780 lower in the computational hierarchy than the edge device 52785 or fog computing device. The edge device 52785 or fog computing device can send the results over a link that may be experiencing network congestion.

[0333] The use of edge devices 52785 or fog computing devices for data management and / or data processing can result in reduced operational costs. Data management, for example, takes less time and computational power because operations can have a single destination instead of circulating from a central location to a local drive.

[0334] A device, e.g., a smart surgical hub / edge device 52700, may determine where to send the surgical data for processing and / or the extent to which to process the surgical data, taking into account one or more of the type of surgical data, the portion of the surgical data to be processed, surgical data characteristics (e.g., surgical data format, size of the surgical data, etc.), performance metrics, processor capabilities, network characteristics (e.g., congestion within the network), etc., as described herein. For example, one of the surgical data characteristics associated with a surgical dataset 52725 may be that the surgical dataset 52725 includes surgical data that is likely to be traceable to an individual patient. In such a case, the surgical data may be processed locally within the protective boundary 52720 and not transmitted to the enterprise cloud server 52730.

[0335] In one example, the smart surgical hub / edge device 52700 may determine, for example, based on processor capabilities and / or processor device capabilities, that the surgical dataset 52725 should be processed on an enterprise cloud server 52730 located outside the protection boundary 52720. If the surgical dataset 52725 includes highly sensitive surgical data, the surgical hub / edge device 52700 may anonymize the surgical dataset 52725 or a portion of the surgical dataset 52725 using, for example, one or more anonymization mechanisms (e.g., redaction, randomization, aggregation, etc.). The surgical dataset 52725 or a portion of the surgical dataset 52725 may be anonymized as described herein to reduce the likelihood that the surgical dataset can be traced back to a patient.

[0336] A mixture of centralized data storage systems and cloud computing may be provided. Computing may be performed on a local network (e.g., while the servers themselves may be distributed). In such cases, for example, portions of the surgical data may be stored locally so that the surgical data may be accessed offline. Fog computing and cloud computing may be provided. Low latency may be associated with fog networks, where large amounts of data may be processed with little or no delay. Because large amounts of data may be stored locally, calculations may be performed faster. Greater data control may be associated with cloud computing. In cloud computing, third-party servers may be completely disconnected from the local network, with little or no control over the data. In fog computing, users may manage surgical information locally and rely on their own security measures. Flexible storage systems may be associated with fog computing. For example, fog computing may not use (e.g., require) constant online access. Data may be stored locally or pulled up from a local drive. Storage may combine online and offline access. Connecting centralized and distributed storage may be described herein. Fog computing can build a bridge between local drives and third-party cloud services, enabling a smooth transition to fully distributed data storage.

[0337] 13 , the location and / or extent of the surgical dataset 52725 to be sent for processing may be determined based on metrics (e.g., performance metrics) associated with the surgical dataset 52725, such as latency, network congestion, etc. In one example, the system may weight the urgency of the need for the results of the surgical data relative to the magnitude of the surgical data and compare that to the capacity within the system's local protection network to determine where and how the data may be sent for processing. For example, a surgical dataset 52725 associated with a low latency metric may indicate its timeliness or importance. Such surgical data may be sent for processing with minimal latency (e.g., to perform the next surgical step in a timely manner). In such a case, the surgical hub / edge device 52700 may decide to send the surgical data locally to a processor or processing device that can process the surgical data in a timely manner with low latency (e.g., rather than sending the data to the enterprise cloud server 52730). For example, the surgical dataset 52725 may be transmitted to an edge network including an edge device 52735 or a fog computing device (not shown). The edge device 52735 or the fog computing device may be located within the protection boundary 52720. Thus, the edge device may process large amounts of data within an acceptable time interval. In one example, if the surgical dataset 52725 is associated with a high latency performance metric, the server hub 52700 may transmit the surgical data for processing to an enterprise cloud server 52730 located outside the protection boundary 52720. The surgical data, or a portion of the surgical data, may be anonymized before being transmitted to the enterprise cloud server for processing, as described herein.

[0338] In one example, the edge device 52275 may perform analysis on the surgical data, then further anonymize the surgical data (e.g., as described in FIG. 10 ) and transmit it to an enterprise server 52730 located outside the protection boundary 52720 for further comprehensive processing.

[0339] In one example, results and / or outcomes associated with surgical data acquired within a local medical facility network may be transmitted to an enterprise cloud server for further processing. Portions of the surgical data may be transmitted to the enterprise cloud server in clear and / or edited form as described herein. The surgical data, such as results and / or outcomes associated with other portions of the surgical data, may be utilized to determine a relationship with one or more measured outcomes. For example, in a chronic air leak (PAL), a portion of the lung is removed. Following a surgical procedure, there may be an air leak that may stop within a few days. Lung collapse may occur when the chest cavity fills. PAL may depend on one or more of the following surgical data: the transection device used during the lobectomy, the location of the lobe removed, patient artifacts (e.g., the state and / or stage of the lung calcifying disease, whether the patient was exposed to radiation or received chemotherapy prior to the surgical procedure, and / or whether the patient was taking any medications that may cause air leaks or promote healing), the type of surgical procedure, and / or the risks associated with the surgical procedure (e.g., whether small or large pieces of lung were removed). When surgical data associated with a thoracic surgical procedure is transmitted, for example, from a device within the protective boundary to an enterprise cloud server, a portion of the surgical data associated with the patient may be anonymized before transmission to the enterprise cloud server. The portion of the surgical data may include, for example, the stage of the calcified lung disease, whether the patient was exposed to radiation or received chemotherapy prior to the surgical procedure, and / or whether the patient was taking any medications. Other portions of the surgical data may be transmitted in a non-anonymized form.

[0340] In one example, the measured outcome may be a characterization of a disease state. Such a measured outcome may be determined by excluding portions of personal data associated with the patient. The selection of the data portions to be excluded or redacted may be based on the relevance of the data portions in determining the measured outcome.

[0341] Analysis of variance may be performed, for example, to compare actual outcomes of a surgical procedure with expected or standard outcomes. Differences may be investigated, for example, to address performance inefficiencies. In one example, analysis of variance may be performed using a decision model. Variances that are statistically significant and require further investigation may be identified.

[0342] In one example, surgical data associated with a surgical procedure may be transferred (e.g., automatically transferred) from the surgical hub / edge device 52700 to an enterprise cloud server 52730 (e.g., an enterprise server). The enterprise server may collect the surgical data from various medical facilities in various geographic locations. In one example, the surgical hub / edge device 52700 may periodically transmit the surgical data to the enterprise cloud server 52730. In one example, the surgical data may be transmitted aperiodically, for example, based on the surgical hub / edge device 52700 receiving a request from the enterprise cloud server 52730.

[0343] In one example, the surgical hub / edge device 52700 may determine a system hierarchy level to which the surgical data may be sent for processing. The system hierarchy level to which the surgical data may be sent for processing may be determined by using a machine learning model 52740 (e.g., which may be located within the surgical hub / edge device 52700). In one example, the machine learning model and / or trained machine learning model may be utilized as part of a supervised learning framework, for example, as described in FIG. 8A herein. The training data (e.g., training examples 802 as illustrated in FIG. 8A) may include a set of training examples (e.g., input data mapped to labeled outputs, as shown in FIG. 8A). The training data used in training the local machine learning model 52515 may include at least one of a data type, characteristic, and performance metric associated with the surgical data, processor capabilities associated with a particular target processing device to which the surgical data may be sent for processing, etc. The output may include a hierarchy level that may be suitable for processing the surgical data. The output may also include an identification of a server and / or server location to which the surgical data may be sent for processing.

[0344] As described with respect to FIGS. 13 and 14A , the surgical dataset 52725 may be divided into data chunk portions or sub-blocks and transmitted to different levels of the system hierarchy. Surgical data chunks, surgical data portions, or surgical data sub-blocks may be used interchangeably herein. The surgical data sub-blocks may be transmitted to various processing devices in parallel at the same time interval or sequentially at different time intervals. In one example, a machine learning model 52740 may be used to predict how the surgical dataset 52725 may be divided into data sub-blocks. The machine learning model 52740 may also be used to predict where and when the divided data sub-blocks (e.g., each of the data sub-blocks) may be transmitted for processing. In one example, the machine learning model 52740 may predict how to divide the surgical dataset 52725 in a manner that results in data sub-blocks that do not contain sensitive data. The machine learning model 52740 may predict where each of the data sub-blocks should be transmitted for further processing. For example, the machine learning model 52740 may predict that a first data sub-block containing non-sensitive data may be sent for processing to the enterprise cloud server 52730. The machine learning model 52740 may also predict that a data sub-block containing sensitive data may be sent locally to an edge server located within the protection boundary 52720.

[0345] The surgical hub / edge device 52700 may consider the potential benefit of sending data to a particular system hierarchy level when determining where to send data. For example, the surgical hub 52700 may evaluate that the surgical dataset 52725 could benefit from being processed at the enterprise cloud server 52730 rather than locally (e.g., based on the enterprise cloud server 52730 having access to a more diverse pool of data than the local edge server). The surgical hub / edge device 52700 may decide to send the surgical dataset 52725 to the enterprise cloud server 52730. Thus, the surgical hub / edge device 52705 may send the surgical dataset 52725 to the enterprise cloud server 52730 or to the local edge server.

[0346] In one example, the surgical hub / edge device 52700 may consider the capabilities of processors located at different system hierarchy levels when determining where to transmit the surgical dataset 52725. For example, the enterprise cloud server 52730 may be capable of having higher processing power compared to the processing power of the local surgical hub or even edge server. The surgical dataset 52725 may have a high data scale (e.g., included in the data characteristics). In such a case, the surgical hub / edge device 52700 may determine that the surgical dataset 52725 should be processed on the enterprise cloud server 52730 with more power. In an example, the surgical dataset 52725 may have a smaller data scale. In such a case, the surgical hub / edge device 52700 may transmit the surgical dataset 52725 locally (e.g., to one of the local servers with less processing power than the enterprise cloud server).

[0347] In one example, the surgical hub / edge device 52700 (e.g., via the machine learning model 52740) may consider surgical data granularity (e.g., included in data characteristics) when determining where to send the surgical dataset 52725. Surgical data granularity may be associated with a measure or degree of comprehensiveness of the surgical dataset 52725 (e.g., all of the relevant data points versus a subset of the relevant data points). The surgical hub / edge device 52700 may determine that for a particular surgical dataset 52725, data granularity may be more important than data diversity. In such cases, the surgical hub / edge device 52700 may send the surgical data or a portion of the surgical data to a local server for processing (e.g., if none of the data points in the surgical dataset require anonymization, such as redaction, resulting in a surgical dataset 52725 with greater data granularity). In one example, the surgical hub / edge device 52700 may determine that data diversity may be given higher importance than data granularity for a particular surgical dataset 52725. In such a case, the surgical hub / edge device 52700 may transmit the surgical dataset 52725 to an enterprise cloud server 52730 located outside the protection boundary 52720 (e.g., where the data granularity (e.g., the amount of data that may be included with a request) is lower and the data diversity is higher than the data granularity of the surgical hub / local edge device).

[0348] 13 , the surgical hub / edge device 52700 may transmit surgical datasets 3 and 4 to an enterprise cloud server 52730. These surgical datasets may have lower granularity than surgical datasets 1, 2, and / or K. The surgical hub / edge device 52700 may transmit datasets 1, 2, and / or K to a local server 52735 located within the protection boundary 52720.

[0349] A feedback mechanism can be used to evaluate the predictions or decisions of the machine learning model. For example, a score can be generated based on the performance of the surgical instrument, such as when the machine learning model selects the local server 52735 over the enterprise cloud server 52730 for data processing. The score can be used to improve the predictions or decisions of the machine learning model when determining where to send the surgical dataset 52735 for processing.

[0350] As described herein, the capabilities of the processors (e.g., each of the processors) may be considered by the surgical hub / edge device 52700 when determining where to send the surgical data set 52725 for processing. Data individuality levels may also be considered by the surgical hub / edge device 52700, as described herein. For example, the surgical hub / edge device 52700 may be aware of the capabilities of the processors (e.g., each of the processors). The surgical hub / edge device 52700 may be configured with the capabilities, for example, as part of the surgical procedure plan 52715 or before starting the surgical procedure. For example, the surgical hub / edge device 52700 may determine that the processing power of the remote cloud server 52730 is higher than the processing power of the surgical hub 52700 or the local edge server 52735. The surgical hub / edge device 52700 may also consider data individuality levels associated with devices to which the surgical data 52725 may be sent for processing. These factors may be used as input by the machine learning model 52740 in determining where to send the surgical dataset 52725 for processing. In one example, the capabilities of various devices (e.g., an edge server located inside a protective perimeter, an edge server located within a medical facility's network, or an enterprise cloud server located centrally at a global or regional level) may be determined by exchanging discovery request / response messages.

[0351] Network traffic may be taken into consideration when determining where to send the surgical dataset 52725. For example, the surgical hub / edge device 52700 may send a test signal over the network to each of the processors that are part of servers or devices located at different system hierarchy levels. The test signal may be utilized to request an acknowledgement message (e.g., an ACK message). Based on the latency of the ACK message, the surgical hub / edge device 52700 may determine and assign a network quality score to each of the processing devices located across the various system hierarchy levels. The network quality score may then be utilized by the machine learning model 52740 in predicting where to send the surgical dataset for processing.

[0352] In one example, a simulation may be generated by the surgical hub / edge device 52700. The simulation may be used (e.g., in combination with the machine learning model 52740) to determine a device or processor associated with a device to which the surgical dataset 52725 may be sent for processing. The simulation may be used to determine a threshold value (e.g., an ideal threshold value). A simulation framework may be described in U.S. Patent Application No. 17 / 332,593, filed May 27, 2021, entitled "Method for Surgical Simulation," the disclosure of which is incorporated herein by reference in its entirety. The simulation may output a score associated with sending the data to each of the processing servers. The surgical hub / edge device 52700 may select a processing device for surgical data processing based on the simulation to maximize the score. Simulations with scores lower than the determined threshold may be excluded from consideration as candidates for selecting a processing device.

[0353] In one example, characteristics of the controlled surgical dataset may be considered by the machine learning model 52740 when determining where to send the surgical data for processing. For example, if the surgical dataset 52725 is sent to an enterprise cloud server 52730, the surgical hub / edge device 52700 may have little or no control over the management of the data. The surgical hub / edge device 52700 may be able to manage and control the surgical data if the surgical dataset 52725 is sent to a local server 52735, for example.

[0354] FIG. 14B shows an example of a surgical hub / edge device 52745 dividing a surgical dataset 52755 into various surgical data subsets and transmitting the divided surgical data subsets to different system hierarchy levels. In one example, the surgical dataset 52755 may be conditioned using a machine learning model 52750 before transmitting it for processing. For example, the surgical hub / edge device 52745 may determine that a given surgical dataset 52755, such as surgical dataset N, should be conditioned and / or manipulated and divided into data sub-blocks 52760 (e.g., as shown in FIG. 14B) before transmitting it for processing. The surgical hub / edge device 52745 may run simulations involving various combinations of dividing the surgical dataset 52755 into data sub-blocks. The surgical hub / edge device 52745 may then obtain how the surgical dataset 52755 may be divided and determine how the surgical dataset 52755 may be divided.

[0355] In one example, the machine learning model 52750 may be trained to take the surgical dataset 52755 as input and a combination of multiple data sub-blocks 52760 as output. In one example, the machine learning model and / or the trained machine learning model may be utilized as part of a supervised learning framework, for example, as described in FIG. 8A herein. The training data (e.g., training examples 802 as illustrated in FIG. 8A) may include a set of training examples (e.g., input data mapped to labeled outputs, e.g., as shown in FIG. 8A). The training data used in training the local machine learning model 52515 may include the surgical dataset. The output may include data sub-blocks and instructions for where, when, and to what extent the data sub-blocks should be processed or sent for processing.

[0356] In one example, machine learning model 52750 may predict and indicate that surgical dataset N 52755 will be split into surgical data subsets 1, 2, M (e.g., each of the surgical datasets may include some data points originally present in dataset N). As illustrated in FIG. 14B , machine learning model 52750 may be utilized to indicate that at time T equals 1, surgical data subset 1 52770 and surgical data subset 2 52775 should be processed locally, while surgical data subset 3 52778 should be sent remotely to an enterprise cloud server.

[0357] In one example, processing of surgical data subset 1 52770, surgical data subset 2 52775, and surgical data subset 3 52778 may occur in parallel. In such a case, the surgical data subsets may be transmitted in parallel, e.g., at the same time interval, to various processors or processing devices for processing.

[0358] In one example, a machine learning model may be used to predict sending various surgical data subsets or sub-blocks (e.g., sub-blocks associated with a surgical data set) to the same processor, such as a local processor. The machine learning model may also predict time intervals (e.g., different time intervals) during which the data subsets or sub-blocks may be processed by the processor or processing device.

[0359] 14B , surgical hub / edge device 52745 may determine that due to confidentiality associated with at least surgical data subsets 52770 and 52775, they may not be sent to an enterprise cloud server for processing. In such a case, surgical hub / edge device 52745 (e.g., using machine learning model 52750) after dividing or partitioning surgical data set 52755 into multiple surgical data subsets or sub-blocks may be processed locally by surgical hub / edge device 52745 or may be sent to edge servers 52772 and 52776, or at least one fog computing device (not shown in FIG. 14B) for processing. Surgical hub / edge device 52745, edge servers 52772 and 52776, and fog computing devices may be located within protection boundary 52746. In one example, two surgical data subsets or sub-blocks may be sent to the same edge server or fog computing device located within protection boundary 52746 for processing.

[0360] In one example, the surgical hub / edge device 52745 may determine how the surgical data sub-blocks may be processed based at least on performance metrics associated with the surgical data subsets or surgical data sub-blocks. For example, surgical data subset 1 52770 may be associated with low latency and surgical data subset 3 52778 may be associated with high latency. In such a case, surgical data subset 1 52770 may be sent to a local server that can process the data with low latency, while surgical data subset 3 52778 may be sent to an enterprise cloud server 52779.

[0361] In one example, the location of a device or processor (e.g., a level in a computational hierarchy) to which surgical data may be sent for processing may be determined based on various surgical data characteristics, such as the intended use of the results associated with the surgical data or the type of metadata associated with the surgical data. For example, a local device (e.g., a surgical hub / edge device 52745 or a smart surgical instrument) may be utilized for interactive or iterative access, update, or aggregate surgical data processing. In such cases, surgical data may be repeatedly added or extracted. Accordingly, the outcome or result may be updated (e.g., periodically updated). The outcome or result may be updated, for example, after each surgical data addition or extraction. A portion of the surgical data processing algorithm that processes such iterative operations may reside on a device or smart surgical instrument located within the protective perimeter or medical facility grounds or network.

[0362] In one example, the surgical hub / edge device 52745 may use metadata, or a portion of the metadata, associated with the surgical data to determine a location where the surgical data may be transmitted for processing, storage, and / or utilization. The metadata, or a portion of the metadata, may indicate a network (e.g., a hospital network-level microcloud network) where the data was collected or stored. The network may retain control of sensitive patient information. Patient-specific information may be utilized to train new control algorithms. Training of the control algorithms may be performed from a base surgical dataset (e.g., serving as a seed surgical dataset) or using data collected within the hospital network.

[0363] In one example, the metadata or a portion of the metadata may include a surgical data sensitivity flag or identifier that specifies the sensitivity level of the surgical data, such as a data sensitivity level. Such metadata or a portion of the metadata may be used to determine or control the level of surgical data processing.

[0364] In one example, the surgical hub / edge device 52745 may use the amount of redaction of the surgical data as a factor to control the level within the system at which certain types of analysis of the surgical data can be performed. For example, low-level analysis that can benefit from all of the interrelated yet identifiable individual surgical data may be performed within the protective boundary of the healthcare provider network 52746. In one example, high-level analysis that may be performed using a portion of the underlying de-identified surgical data may be performed by an enterprise cloud server 52779 located outside the protective boundary 52746.

[0365] In one example, higher level aggregation of regional or global surgical procedure outcome and / or surgical procedure step data may be performed on an enterprise cloud server. The enterprise cloud server may be located outside the protected medical facility network. Such an enterprise cloud server may have the capacity to process large amounts of data. The data that may be processed in the enterprise cloud server may be of a type where personal biomarker data may not be required. The processed data may be edited before transferring it from the protected network to another storage location.

[0366] In one example, one or more of the resources available within the processing device, system or network, the risk associated with the surgical data, and the need to process the surgical data within a protected network may be utilized to determine the priority, processing depth and / or storage of the data and / or algorithm results.

[0367] 15, priorities may be assigned to surgical data subsets or sub-blocks (e.g., each of the surgical data sub-blocks) to determine the time and / or resources that may be used to process a particular surgical data subset or sub-block. The availability of at least one resource may be used in determining the resources that may be used for a particular data subset or sub-block of a particular priority level, as described herein.

[0368] 15 illustrates compartmentalization of surgical data and / or algorithms. Machine learning (ML) models 52790 may be utilized to process surgical data on a local device / system and / or cloud server. Compartmentalization (e.g., selective compartmentalization) of ML algorithm processing of local surgical data may be performed.

[0369] In one example, the adjustment / scaling of the breadth, depth, and / or reduction of local surgical data may be performed on a local surgical computing device (e.g., surgical hub) or edge server based on local available resource-time dependencies. The adjustment / scaling may include adjusting / scaling one or more of the following: the amount of data or variables that can be processed, the frequency or precision level of the surgical data, the algorithm type, the algorithm tolerance, the algorithm stacking level, or the validation (e.g., verification and / or checking) of the measured outcome or result.

[0370] 15, various resources of a processing device (e.g., a surgical hub or edge server device) may have different availability. For example, resources 1 and 2 may only be available two of three time slots (e.g., time slots 2 and 3 for resource 1, and time slots 1 and 2 for resource 2), while resource 3 may be available in all three slots. In such a case, compartmentalization and scaling may be performed such that the width and / or depth of the surgical data subsets or sub-blocks processed by resources 1 and 2 may require fewer resources (e.g., a smaller amount of data, fewer variables, etc.) than the surgical data subsets or sub-blocks processed by resource 3.

[0371] 15 , local surgical data (e.g., surgical dataset 52795) may be adjusted and / or scaled 52800 based on at least one of the following: timeliness of required results, available processing and memory, network bandwidth or communication parameters (e.g., throughput, latency, etc.), risk level of functioning without an answer, importance of the data or task, availability of other surgical data to be used instead. In one example, if the surgical dataset 52795 being analyzed is associated with timeliness of required results, compartmentalization may be performed such that the time-sensitive surgical data is scaled to be processed by resources 2 and 3 (e.g., all three time slots are available for processing) rather than resource 1 (the first time slot is not available for processing immediately or within slot 1). In one example, scaling the breadth, depth, and / or reduction of the local surgical data may be performed to balance the level of results achieved within a time interval with the resources that may be required.

[0372] In one example, as illustrated in FIG. 15, the surgical data or ML algorithm may be compartmentalized or clustered (52927). The ML algorithm may be partitioned into smaller portions based on, for example, the magnitude or level of processing. In one example, the complexity of the ML algorithm may be determined based on the level of available local computing resources. The ML algorithm may be utilized between the cloud and an edge processing network. The algorithm pre-processing component may use the system and its resources in which it resides as a means to determine the following: the diversity of the surgical dataset, the level of compartmentalization of the surgical data, and / or factors to be considered for analysis of the surgical data.

[0373] The ML algorithm used to analyze the surgical data may be scaled based on the computing resources (e.g., computing power, size of memory of the computing resources) associated with the surgical system 52785 on which the ML algorithm is running, the competing processing needs associated with various processes running on the surgical system 52785, and / or the width of the surgical dataset 52795. The computing resources associated with the surgical system, the competing processing needs of the various processes running on the system, and / or the width of the surgical dataset may change based on time, as illustrated in FIG.

[0374] In one example, in a surgical computing device (e.g., a surgical hub) where computing resources are utilized to process and / or analyze surgical data received from various surgical devices (e.g., including video feeds from various cameras within an operating room), the surgical device may scale ML algorithms based on the level of computing resources available (e.g., available during a time slot).

[0375] In one example, in a surgical computing device, the availability of computing resources on the surgical computing device utilized to process surgical data received from various surgical devices may vary over time, and the scaling of the ML algorithm may change dynamically (e.g., dynamically change over time) based on the resources available on the surgical computing device on which the ML algorithm resides and / or executes.

[0376] 15, in time slot 1, only computing resource 1 and computing resource 2 may be available for utilization by the ML algorithm, while in time slot 2, all three computing resources 1, 2, and 3 may be available for utilization only for the time slot. The surgical device may scale the ML algorithm accordingly based on the availability of computing resources. For example, in time slot 1, the ML algorithm may be scaled down (e.g., simplified by ignoring, removing, or combining certain surgical data aspects). This may be done, for example, to accommodate the unavailability of computing resource 1, which may process a significant portion of the surgical data. As an example, in time slot 2, when all three resources are available, the ML algorithm may be scaled up (e.g., by using a more comprehensive surgical dataset and / or performing a more complex and comprehensive analysis of the surgical dataset).

[0377] As described herein, various types of ML algorithms may include supervised algorithms, unsupervised learning algorithms, semi-supervised learning algorithms, reinforcement learning algorithms, etc. Some specified types of ML algorithms may include linear regression, logistic regression, decision trees, SVM algorithms, naive Bayes algorithms, KNN algorithms, K-means, etc. A respective algorithmic complexity level may be associated with each of the ML algorithms. For example, a KNN algorithm may be more computationally complex and therefore may have a higher algorithmic complexity level than a decision tree algorithm.

[0378] The complexity level of an ML algorithm may be related to available computing resources. In one example, an ML algorithm with higher computational complexity may be utilized on an edge processing device or a cloud-based enterprise server with higher computational / processing power and / or memory resources. In another example, an ML algorithm with lower computational complexity may be utilized on a device (e.g., a surgical hub) with lower computational / processing power and / or memory resources.

[0379] One or more of the scaling of the complexity of the ML algorithm, the ML algorithmic method or processing method to be applied, and / or the size of the dataset to which the ML algorithm is applied may be determined based on the resources (e.g., computational resources, network resources, etc.) available on the surgical system or surgical computing device on which the ML algorithm may reside and / or one or more attributes of the dataset. The attributes of the dataset may include the size of the dataset, the complexity of the dataset, the depth to which the dataset may be processed, etc.

[0380] In one example, an ML algorithm may be partitioned into various portions that may be processed on an edge processing device (e.g., an edge processing device within the protected network) and a cloud-based enterprise server (e.g., an enterprise server located outside the protected network). In one example, an algorithm on a computing device (e.g., a pre-processing component of the algorithm) may at least consider resources associated with the computing device to determine factors that may be used to obtain the size of the dataset that may be analyzed by the ML algorithm. Resources associated with the computing device may include computational / processing power and / or memory resources.

[0381] In one example, ML algorithm scaling on a surgical computing system or surgical computing device may be based on at least one of the total amount of surgical data analyzed, the depth to which the computing system compiles the surgical data, the serialization of different processing stages (which may, for example, provide an indication of how long it may take to process the surgical data), or the simplicity of the surgical data or surgical data compilation. Scaling may ignore surgical data aspects, remove or combine categories, or aggregate datasets before removing individual pairwise comparisons.

[0382] In one example, scaling the ML algorithm may result in a simplification of the analysis performed on the surgical data, for example, by excluding certain surgical data aspects or by anonymizing, removing, or combining certain surgical data categories.

[0383] In one example, scaling of local analysis may be performed. As illustrated in Figure 15, additional processing of the surgical data or surgical data subsets or sub-blocks (e.g., data subset M) may be performed on one or more enterprise cloud servers 52787. The enterprise cloud servers 52787 may be co-located or geographically separated. In another example, the additional surgical data processing may be performed later in time or in combination with the current surgical data processing.

[0384] The device or system may be configured to prioritize local sub-processing. The local sub-processing may process portions of the surgical data that may be personalized data. The non-personalized portions of the surgical data may be processed on a remote system or server. The non-personalized portions of the surgical data may be processed simultaneously or sequentially with the local processing of the personalized portions of the surgical data.

[0385] In one example, a device or system (e.g., a system located within a medical facility) can scale analysis associated with time-sensitive aspects of surgical data that may require immediate results within a surgical procedure. The surgical device can perform more complete or in-depth processing of the complete surgical data set 52795 offline to the procedure. The offline processing can be performed by the device or system or by a remote cloud-based server or service.

[0386] In one example, dynamic reallocation of ML compartments may be performed. For example, in the case of a dynamically disconnected device or element of the computing chain, reallocation of ML compartments may be performed based on reallocation of processing resources. For example, if a communication channel is interrupted due to a failure in the chain (e.g., a power outage, a disconnected or damaged instrument or cable during surgery, or other hardware / software failure), one of the other surgical computing devices or computing elements may be configured to share the load associated with the failed surgical computing device or computing element. A notification indicating the failure of the device or computing element and / or a notification, e.g., a warning notification, may be sent to the healthcare provider or user indicating that processing of surgical data may be slowed down.

[0387] In one example, the compartmentalization of ML algorithms can be dynamically scaled or adjusted with resource availability. One or more ML compartments can be designated as related. In one example, such relationships can be dynamic and updated (e.g., periodically updated). In one example, such relationships can be defined prior to surgical data processing, allowing the system to combine or separate related ML situations as needed.

[0388] In one example, the width and / or depth of the surgical data on the surgical computing device (e.g., the surgical hub) may be modified or reduced, and at least one surgical data attribute analyzed by the ML algorithm may be scaled or adjusted. The modification or reduction of the surgical data and the adjustment / scaling of the at least one surgical data attribute analyzed by the ML algorithm may be based on the availability of resources-time availability of the surgical computing device.

[0389] The availability of resource-time relationships for a surgical computing device may be determined based on at least one of the timeliness of the required results, the level of computational processing associated with the surgical computing device or the computational memory associated with the surgical computing device, the network bandwidth between the surgical computing device and the location where the required results are sent, one or more communication parameters (e.g., the throughput rate at the surgical computing device or the latency experienced by the surgical computing device), the risk level of functioning without obtaining the required results, the level of importance of the surgical data or surgical task associated with the surgical task, and / or the availability of other data that may be used as a substitute.

[0390] The modification or reduction of the surgical data and the adjustment / scaling of at least one surgical data attribute may be performed on the surgical computing device, for example, as described herein, to balance the level of results achieved within a time slot with the resources that the surgical computing device may make available, for example, within a time slot.

[0391] The surgical computing device may scale at least one attribute associated with the ML algorithm based on a balance of the level of result required, the time associated with the result required, and the availability of computing resources within the time associated with the result required. The at least one attribute may include a size of the surgical data, a number of surgical data variables, a frequency associated with the surgical data, a precision level associated with the surgical data, an ML algorithm type, a tolerance associated with the ML algorithm, a number of stacking levels associated with the ML algorithm, and / or a validation or check of the results.

[0392] In one example, the ML algorithm on the surgical computing device may be compartmentalized or clustered into multiple portions or parts, and the size and / or level of processing required may be determined for each smaller portion or part of the ML algorithm.

[0393] Figure 16 illustrates an example of connectivity between a surgical computing device / edge computing device 52805 and an enterprise cloud server 52810 (e.g., an enterprise cloud server). As illustrated in Figure 16, the surgical computing device / edge computing device 52805 may include, among other things, a processor 52812, a memory 52814 (e.g., non-removable memory and / or removable memory), an analysis subsystem 52816, a local machine learning model 52818, and / or a local storage subsystem 52820. It will be understood that the surgical computing device / edge computing device 52805 may include any sub-combination of the foregoing elements / subsystems while remaining consistent with an embodiment.

[0394] The processor 52812 in the surgical computing device / edge computing device 52805 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (SIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 52812 may perform data processing, authentication, input / output processing, and / or any other function that may enable the surgical computing device / edge computing device 52805 to operate in an environment suitable for performing a surgical procedure. The processor 52812 may be coupled to a transceiver (not shown). The processor 52812 may use the transceiver (not shown in the drawings) to communicate with the enterprise cloud server 52810.

[0395] The memory 52814 in the surgical hub / edge device 52805 may be used to store where data has been sent. For example, the memory may be used to recall that data has been sent to the enterprise cloud server 52810. The memory may include a database and / or lookup table. The memory may include virtual memory that may be linked to a server located within the protected network.

[0396] The processor 52812 in the surgical computing device / edge computing device 52805 may access information from and store data in any type of suitable memory (e.g., non-removable and / or removable memory). Non-removable memory may include random-access memory (RAM), read-only memory (ROM), a hard disk, a solid-state drive, or any other type of memory storage device. Removable memory may include secure digital memory.

[0397] The processor 52812 in the surgical computing device / edge computing device 52805 can access information from and store data in extended storage 52820 (e.g., non-removable memory and / or removable memory). In one example, the processor 52812 may access information from and store data in memory that is not physically located on the surgical computing device / edge computing device 52805, such as on a server or secondary edge computing system (not shown).

[0398] The enterprise cloud server 52810 may include, among other things, a processor, memory (e.g., non-removable memory and / or removable memory), and / or a storage subsystem. It will be understood that the enterprise cloud server 52810 may include any sub-combination of the foregoing elements / subsystems while remaining consistent with an embodiment.

[0399] The analytics module 52816 in the surgical hub / edge device 52805 can be used to determine when and where to send the surgical data for processing, as described herein with respect to Figures 13, 14A, 14B, and 15. The analytics module 52816 can be used to determine when and how to perform compartmentalization of the surgical data and ML algorithms, as described herein with respect to Figure 16.

[0400] The storage 52820 used within the surgical hub / edge device 52805 may be used to archive the results of what happened when data was sent to a particular processor. The storage 52820 may be a module included in the surgical hub / edge device 52805. In one example, the storage may be hardware (e.g., off-disk storage) accessible by the surgical hub / edge device 52805.

[0401] A local machine learning model 52818 in the surgical hub / edge device 52805 can be trained to determine where (e.g., to which processor) to send data for processing and / or how to partition the data, as described with respect to Figures 13, 14A, and 15.

[0402] 16, the surgical hub / edge device 52805 may transmit data to and / or receive surgical data from an enterprise cloud server 52810. The surgical data may be based on measurements taken from sensors, actuators, robotic movements, biomarkers, surgeon biomarkers, visual aids, and / or others. Wearables are described in more detail in U.S. Patent Application No. 17 / 156,28, filed November 10, 2021, under the heading "Monitoring of adjusting a surgical parameter based on biomarker measurements," the disclosure of which is incorporated herein by reference in its entirety.

[0403] The measurements may be associated with one of many actuators located in the operating room. For example, measurements may be generated based on the readings of a potentiometer located on a surgical instrument used as described with respect to FIG. 14B . The surgical data may be related to the surgeon's cortisol levels. Surgical data may be collected based on these measurements and may help define the power, force, functional operation, or behavior of a surgical instrument, such as a smart handheld stapler, as may be described in more detail under the heading "Techniques for adaptive control of motor velocity of a surgical stapling and cutting instrument" in U.S. Patent No. 10,881,399, filed June 20, 2017, the disclosure of which is incorporated herein by reference in its entirety. The data may be used to provide situational awareness to smart instruments, such as smart energy devices, as may be described in more detail under the heading "Method for smart energy device infrastructure" in U.S. Patent No. 16 / 209,458, filed December 4, 2018, the disclosure of which is incorporated herein by reference in its entirety.

[0404] For example, a surgeon may wear a sensing device (e.g., a watch) that can determine the surgeon's cortisol levels based on readings of sweat produced by the surgeon. Such data may be anonymized (e.g., redacted, randomized, summarized, averaged, etc.) from being transmitted to the removal server.

[0405] Smart interconnected systems may be provided to define their relationships, cooperative behavior, or monitoring / storing of data detailing the treatments described herein, which may be aggregated to develop better algorithms, trends, or treatment responses based on a comparison of outcomes and options. Such techniques may be described in more detail under the heading "Method of hub communication, processing, display, and cloud analytics" in U.S. Patent No. 16 / 209,416, filed December 4, 2018, the entire disclosure of which is incorporated herein by reference.

[0406] 17 shows an example of a flowchart for determining where surgical data may be sent for processing. At 52825, a device (e.g., a surgical hub, an edge server, a fog computing device, etc.) may obtain surgical data associated with a surgical task. The surgical data may be a surgical data magnitude and a surgical data individuality level. The surgical data magnitude may be the extent to which the surgical data may be processed. The surgical data individuality level may be the surgical data individuality level at which the surgical data is processed.

[0407] At 52830, the surgical hub / edge device may determine a set of parameters associated with a first surgical data sub-block of surgical data and a second surgical sub-block of surgical data. For example, the surgical hub / edge device may determine a first set of parameters associated with the first surgical data sub-block of surgical data and a second set of parameters associated with the second surgical data sub-block of surgical data.

[0408] In 52835, the surgical hub / edge device may determine a processing level to be used to process each of the first sub-block of surgical data and the second sub-block of surgical data. For example, the surgical hub / edge device may determine a first processing level to be used to process the first surgical data sub-block. The first processing level may be obtained based on a first capability associated with a first processing device located at a first computing hierarchy level of the healthcare provider's network. The surgical hub / edge device may also determine a second processing level to be used to process the second surgical data sub-block. The second processing level may be obtained based on a second capability associated with a second processing device located at a second computing hierarchy level of the healthcare provider's network.

[0409] In 52840, the surgical hub / edge device may transmit the first surgical data sub-block to the first pro...

Claims

1. 1. A processor-implemented method, the method comprising: acquiring first surgical data associated with the surgical procedure; identifying a first processing device for processing the surgical data based on at least one of a capability of the first processing device, a data size of the first surgical data, a sensitivity to latency in processing the first surgical data, a data individuality level of the first surgical data, and an intended use of the first surgical data; A method comprising:

2. The method comprises: receiving surgical data associated with the surgical procedure, the surgical data including the first surgical data and second surgical data; identifying a second processing device for processing the second surgical data based on at least one of a capability of the second processing device, a data size of the second surgical data, a sensitivity to latency in processing the second surgical data, a data individuality level of the second surgical data, and an intended use of the second surgical data; The method of claim 1 further comprising:

3. The method comprises: transmitting the first surgical data to the first processing device; transmitting the second surgical data to the second processing device; The method of claim 2 further comprising:

4. The method of claim 3 , wherein the first processing device is different from the second processing device.

5. The method of claim 2 , further comprising dividing the acquired surgical data into the first surgical data and the second surgical data.

6. 3. The method of claim 2, wherein the data partitioning is based on at least one of the following: a data size of the first surgical data and the second surgical data, a data granularity of the first surgical data and the second surgical data, a timeliness of results associated with the first surgical data and the second surgical data, an importance level associated with the first surgical data and the second surgical data, the first data individuality level and the second data individuality level, a frequency or accuracy level associated with the first surgical data and the second surgical data, resource-time availability of the first processing device and the second processing device, network bandwidth, and availability of other surgical data to be used in place of the first surgical data and the second surgical data.

7. 3. The method of claim 2, wherein the at least one capability associated with the first processing device and / or the second processing device comprises at least one of processing power, memory size, or time taken to process the first surgical data and / or the second surgical data.

8. the first surgical data is first health data associated with a patient, the first health data having a first personality level, and the method further comprises: determining whether to transform the first health data based on a privacy protection level associated with the first processing device or a location of the first processing device and, optionally, the first personality level; If conversion is determined to be necessary, converting the first health data such that the converted first health data has a second data individuality level, the second data individuality level being lower than the first data individuality level; transmitting the first health data or the converted first health data to the first processing device depending on whether conversion is required; The method of claim 1 further comprising:

9. 10. The method of claim 8, further comprising determining the first data individuality level of the first health data.

10. The method of claim 8 , wherein the first data individuality level is determined using a machine learning model.

11. The method of claim 8 , wherein the identification of the first processing device is performed using machine learning.

12. 10. The method of claim 1, wherein the first health data includes a plurality of data points, each data point associated with a level of personality, and the first level of personality of the first health data is based on an aggregation of the levels of personality of the data points.

13. 13. The method of claim 12, wherein transforming the first health data includes anonymizing one or more of the plurality of data points.

14. The method of claim 8 , wherein the capabilities of the first processing device may include a diversity of data sets available to the first processing device and / or a processing power of the first processing device.

15. The method comprises: receiving second health data associated with a second patient and having a third data individuality level and a third data magnitude; identifying a second processing device for processing the second health data based on at least one of a capability of the second processing device, a size of the third data, a sensitivity to latency in processing the second health data, and an intended use of the second health data; determining whether to transform the second health data based on a privacy protection level associated with the second processing device or a location of the second processing device and, optionally, the third data individuality level; If it is determined that conversion is necessary, converting the second health data so that the converted second health data has a fourth data individuality level and a fourth data magnitude, wherein the fourth data individuality level is lower than the third data individuality level; transmitting the second health data or the converted second health data to the second processing device depending on whether conversion is required; The method of claim 8 further comprising:

16. 16. The method of claim 15, wherein the first processing device is located outside a protected network and the second processing device is located inside the protected network.

17. The method of claim 16 , wherein the privacy protection level associated with the first processing device is higher than the privacy protection level associated with the second processing device.

18. 16. The method of claim 15, wherein the protected network is a Health Insurance Portability and Accountability Act (HIPAA)-based network that includes at least one medical facility.

19. 14. The method of claim 13, wherein the one or more data points are anonymized by one or more of: editing the data points, randomizing the data points, aggregating the data points, summarizing the data points, averaging the data points, or expressing the data points by a range.

20. The method of claim 8 , wherein the first processing device is identified based on an ability of the first processing device to process data associated with a given data magnitude.

21. A device, A processor configured to carry out the method of any one of claims 1 to 20. A device comprising:

22. A computing program which, when executed by a processor, causes the processor to carry out the method of any one of claims 1 to 20.

23. a processor, the processor comprising: partitioning surgical data into a first surgical data sub-block and a second surgical data sub-block, the first surgical data sub-block associated with a first resource-time availability of a processing device and the second surgical data sub-block associated with a second resource-time availability of the processing device; processing the first surgical data sub-block using a first ML algorithm sub-block according to the first resource-time availability of the processing device; processing the second surgical data sub-block using a second ML algorithm sub-block according to the first resource-time availability of the processing device; A device that is configured to:

24. 24. The device of claim 23, wherein the processor is further configured to divide an ML algorithm into the first ML algorithm sub-block and the second ML algorithm sub-block.

25. 24. The device of claim 23, wherein the processor is further configured to divide an ML algorithm into the first ML algorithm sub-block and the second ML algorithm sub-block based on a magnitude or level of processing associated with the first ML algorithm sub-block and the second ML algorithm sub-block.

26. 24. The device of claim 23, wherein the surgical data is divided into the first surgical data sub-block and the second surgical data sub-block based on one or more of timeliness of results, network bandwidth, at least one communication parameter, processing power, risk level, or one of task importance associated with each of the first surgical data sub-block and the second surgical data sub-block, an importance level, or availability of other surgical data to be used in place of the first surgical data sub-block or the second surgical data sub-block.

27. 24. The device of claim 23, wherein the processor is further configured to divide the ML algorithm into the first ML algorithm sub-block and the second ML algorithm sub-block based on one or more of: timeliness of results, network bandwidth, at least one communication parameter, processing power, and one of risk level.

28. 24. The device of claim 23, wherein the surgical data is divided based on one or more of the amount of surgical data or number of variables processed, a frequency or precision level associated with the surgical data.

29. 24. The device of claim 23, wherein the processor is further configured to divide an ML algorithm into the first ML algorithm sub-block and the second ML algorithm sub-block based on at least one of an algorithm type, a tolerance of the ML algorithm, a stacking level of the ML algorithm, and a verification or check of a result.

Images