Systems and methods for prediction of physiological trajectory

Abstract

Systems and methods in accordance with the present disclosure can include one or more machine learning models configured to generate real-time, intraoperative predictions of events of a subject during a procedure being performed on the subject. The machine learning models can be used to predict events during targeted forecast windows to allow for effective response to the events. The machine learning models can be trained to operate in multi-model voting (MMV) architectures. The machine learning models can be trained using features or signals selected to provide high predictive accuracy for a given condition. The machine learning models can be trained to have various forecast windows, including various predetermined and / or adjustable forecast windows, to provide for more effective guidance on onset and response to the events.

Description

SYSTEMS AND METHODS FOR PREDICTION OF PHYSIOLOGICALTRAJECTORYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 607,739, filed on December 8, 2023, and U.S. Provisional Patent Application No. 63 / 698,320, filed on September 24, 2024, the entireties of which are incorporated by reference herein.TECHNICAL FIELD

[0002] The present disclosure relates to prediction of physiological trajectory of a subject.BACKGROUND

[0003] Patients and other subjects undergoing procedures can be susceptible to significant conditions as a result of the procedures. Due to various factors including both pre-procedure physiological considerations as well as real-time factors or complications during the procedure, it can be challenging for these conditions to be detected in a sufficiently accurate and timely manner to allow for timely responses to the conditions.SUMMARY

[0004] At least one aspect of the present disclosure is directed to a method for multimodel voting prediction. The method can include monitoring, by one or more processors, one or more features of a subject during a first period during a procedure performed on the subject. The method can include applying, by the one or more processors, as input to a plurality of machine learning models, the one or more features to cause the plurality of machine learning models to each generate a candidate prediction of an event of the subject during a second period subsequent to the first period. The plurality of machine learning models can be configured using training data that includes example data of the one or more features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period. The method can include determining, by the one or more processors, an output prediction of the event based on each candidate prediction generated by each machine learning model of the plurality of machine learning models. The methodcan include outputting, by the one or more processors, an indication of the output prediction of the event.

[0005] Another aspect of the present disclosure is directed to a system for multimodel voting prediction. The system can include one or more processors. The one or more processors can monitor one or more features of a subject during a first period during a procedure performed on the subject. The one or more processors can apply, as input to a plurality of machine learning models, the one or more features to cause the plurality of machine learning models to each generate a candidate prediction of an event of the subject during a second period subsequent to the first period. The plurality of machine learning models can be configured using training data that includes example data of the one or more features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period. The one or more processors can determine an output prediction of the event based on each candidate prediction generated by each machine learning model of the plurality of machine learning models. The one or more processors can output an indication of the output prediction of the event.

[0006] Another aspect of the present disclosure is directed to a method for training data for multi-model voting prediction. The method can include identifying, by one or more processors, training data that includes one or more features of a plurality of example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period. The method can include updating, by the one or more processors, a plurality of machine learning models by applying the training data as input to the plurality of machine learning models.

[0007] Another aspect of the present disclosure is directed to a system for training data for multi-model voting prediction. The system can include one or more processors. The one or more processors can identify training data that includes one or more features of a plurality of example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period. The one or more processors can update a plurality of machine learning models by applying the training data as input to the plurality of machine learning models.

[0008] Another aspect of the present disclosure is directed to a method for multivariate prediction. The method can include monitoring, by one or more processors, aplurality of features of a subject during a first period during a procedure performed on the subject. At least one feature of the plurality of features can be selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject. The method can include applying, by the one or more processors, as input to one or more machine learning models, the plurality of features to cause the one or more machine learning models to generate a prediction of the event during a second period subsequent to the first period. The one or more machine learning models can be configured using training data that includes example data of the plurality of features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period.

[0009] Another aspect of the present disclosure is directed to a system for multivariate prediction. The system can include one or more processors. The one or more processors can monitor a plurality of features of a subject during a first period during a procedure performed on the subject. At least one feature of the plurality of features can be selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject. The one or more processors can apply, as input to one or more machine learning models, the plurality of features to cause the one or more machine learning models to generate a prediction of the event during a second period subsequent to the first period. The one or more machine learning models can be configured using training data that includes example data of the plurality of features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period. The one or more processors can output an indication of the prediction of the event.

[0010] Another aspect of the present disclosure is directed to a method for training data for multivariate prediction. The method can include identifying, by one or more processors, training data that includes a plurality of features of a plurality of example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period. The method can include updating, by the one or more processors, one or more machine learning models by applying the training data as input to the one or more machine learning models.

[0011] Another aspect of the present disclosure is directed to a system for training data for multivariate prediction. The system can include one or more processors. The one or more processors can identify training data that includes a plurality of features of a pluralityof example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period. The one or more processors can update one or more machine learning models by applying the training data as input to the one or more machine learning models.

[0012] Another aspect of the present disclosure is directed to a method for multiple forecasters. The method can include monitoring, by one or more processors, one or more features of a subject during one or more data periods during a procedure performed on the subject. At least one feature of the one or more features can be selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject. The method can include applying, by the one or more processors, as input to one or more machine learning models, the one or more features to cause the one or more machine learning models to generate a plurality of predictions of the event during a plurality of forecast periods each corresponding to the one or more data periods. Each of the plurality of forecast periods can be subsequent to the corresponding one or more data periods. The method can include outputting, by the one or more processors, a plurality of indications of the plurality of predictions of the event.

[0013] Another aspect of the present disclosure is directed to a system for multiple forecasters. The system can include one or more processors. The one or more processors can monitor one or more features of a subject during one or more data periods during a procedure performed on the subject. At least one feature of the one or more features can be selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject. The one or more processors can apply, as input to one or more machine learning models, the one or more features to cause the one or more machine learning models to generate a plurality of predictions of the event during a plurality of forecast periods each corresponding to the one or more data periods. Each of the plurality of forecast periods can be subsequent to the corresponding one or more data periods. The one or more processors can output a plurality of indications of the plurality of predictions of the event.

[0014] Another aspect of the present disclosure is directed to a method for training data for multiple forecasters. The method can include identifying, by one or more processors, training data that includes one or more features of a plurality of example subjects during one or more data periods and of a presence of an event for the plurality of example subjects during a plurality of forecast periods each corresponding to the one ormore data periods. Each of the plurality of forecast periods can be subsequent to the corresponding one or more data periods. The method can include updating, by the one or more processors, one or more machine learning models by applying the training data as input to the one or more machine learning models.

[0015] Another aspect of the present disclosure is directed to a system for training data for multiple forecasters. The system can include one or more processors. The one or more processors can identify training data that includes one or more features of a plurality of example subjects during one or more data periods and of a presence of an event for the plurality of example subjects during a plurality of forecast periods each corresponding to the one or more data periods. Each of the plurality of forecast periods can be subsequent to the corresponding one or more data periods. The one or more processors can_update one or more machine learning models by applying the training data as input to the one or more machine learning models.

[0016] Those skilled in the art will appreciate that the summary is illustrative only and is not intended to be in any way limiting. Other aspects, inventive features, and advantages of the devices and / or processes described herein, as defined solely by the claims, will become apparent in the detailed description set forth herein and taken in conjunction with the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0018] FIG. 1 illustrates an example of a system for real-time prediction of physiological states of subjects, according to an example implementation.

[0019] FIG. 2A illustrates a plot of a positive episode, according to an example implementation.

[0020] FIG. 2B illustrates a plot of a positive episode, according to an example implementation.

[0021] FIG. 3 A illustrates a plot of an extended period of negative data, according to an example implementation.

[0022] FIG. 3B illustrates a plot of a negative episode, according to an example implementation.

[0023] FIGS. 4A-4D illustrate a study cohort, according to an example implementation.

[0024] FIGS. 5 A and 5B illustrate the performance of a univariate model, according to an example implementation.

[0025] FIG. 6A illustrates a plot of performance for prediction of an event, according to an example implementation.

[0026] FIG. 6B illustrates a plot of performance for prediction of an event, according to an example implementation.

[0027] FIGS. 7 A and 7B illustrate the performance of a multivariate model in a static scenario, according to an example implementation.

[0028] FIG. 7C illustrates a plot of an ablation study, according to an example implementation.

[0029] FIGS. 8 A and 8B illustrate the performance of a multivariate model in a dynamic scenario, according to an example implementation.

[0030] FIGS. 9A-9G illustrate the performance of a multi-model voting approach, according to an example implementation.

[0031] FIGS. 10A and 10B illustrate the performance of a multi-model voting approach, according to an example implementation.

[0032] FIG. 11 illustrates a method to generate a prediction of a physiological event, according to an example implementation.

[0033] FIG. 12 illustrates a method to train data for generating a prediction of a physiological event, according to an example implementation.

[0034] FIG. 13 illustrates a method to generate a prediction of a physiological event, according to an example implementation.

[0035] FIG. 14 illustrates a method to train data for generating a prediction of a physiological event, according to an example implementation.

[0036] FIG. 15 illustrates a method to generate a prediction of a physiological event, according to an example implementation.

[0037] FIG. 16 illustrates a method to train data for generating a prediction of a physiological event, according to an example implementation.

[0038] FIG. 17 is a block diagram of an example computing system suitable for use in the various arrangements described herein, according to an example implementation.

[0039] Like reference numbers and designations in the various drawings indicate like elements.DETAILED DESCRIPTION

[0040] Following below are more detailed descriptions of various concepts related to, and implementations of, methods, apparatuses, and systems for prediction of physiological trajectory. The various concepts introduced above and discussed in greater detail below may be implemented in any of a number of ways, as the described concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided primarily for illustrative purposes.SYSTEMS AND METHODS FOR PREDICTION OF PHYSIOLOGICAL TRAJECTORYI. Overview

[0041] During procedures such as surgeries being performed on subjects (e.g., patients), subjects can be susceptible to various conditions that can be useful to detect in order to modify the procedures, or otherwise mitigate the conditions. These conditions can include changes in physiological status, health status, intraoperative, and / or postoperative events. For example, it can be useful to detect events (e.g., hypotensive events), which can be related to cardiac issues even in non-cardiac surgeries; as an example, patients developing myocardial injury after non-cardiac surgery (MINS) have high mortality one year after surgery. Except for biochemical determination of cardiac injury, there are typically no early signs that can predict MINS, underscoring the usefulness of a computational process to monitor signals regarding a subject during a procedure in order to accurately predict such events in sufficient time prior to their occurrence to allow for intervention.

[0042] Systems and methods in accordance with the present disclosure can enable an intraoperative early warning system, such as a machine learning model-based predictionsystem, for prediction of a subject event (e.g., changes in physiological status, health status, and or intraoperative events) during a procedure being performed on a subject, such as during surgery, including when the subject is being provided anesthesia. This can allow for vigilance and / or intervention before the changes / events occur. As compared to systems that are limited to a general indication if an event may occur during a procedure, systems and methods in accordance with the present disclosure can more accurately and precisely predict the timing of such events (e.g., predict whether and when an adverse event is likely to occur) to allow for a real-time prediction system. Systems and methods in accordance with the present disclosure can more effectively provide such predictions in the context of cancer patients, for which estimation and / or prediction models are to account for unique physiological status of such patients given complex, multifactorial aspects of the disease and concurrent therapies, such as chemotherapy, immunotherapy, and radiation therapy, among others, that can substantially alter physiological status and response.

[0043] The machine learning models described herein can achieve high accuracy, precision, and recall with respect to correct, timely prediction of events. The machine learning models can be configured (e.g., trained, fine-tuned, updated) using data regarding subjects as well as real-time intraoperative data from one or more sensors targeted to the subject. The machine learning models can be configured using a voting-based architecture, such as multi-model voting (MMV). This can limit the occurrence of false positive predictions of subject events during a procedure. The machine learning models can be configured using selected types of data, such as subsets of subject data and / or sensor data, validated to allow for improved performance regarding prediction of specific events. The machine learning models can be configured to have multiple, controllable forecasting windows for durations of time in which the predictions are generated, providing greater flexibility in targeting predictions during procedures.II. Machine Learning Model-Based Intraoperative Prediction Systems

[0044] FIG. 1 depicts an example of a system 100. The system 100 can be an intraoperative prediction system, such as to perform functions including predicting specific events expected to occur to a subject 110 during particular period(s) of time in a procedure. The techniques described in the present disclosure may apply to any of a variety of procedures or therapies on subjects, including, for example, open liver lobectomy, minimally invasive liver lobectomy, open colorectal surgery, and / or minimally invasive colorectal surgery. The system 100 can be operated during the procedure, such as to receivesensor data regarding the subject 110 during the procedure in order to predict occurrence of one or more events regarding the subject 110 during the procedure. Various aspects of the system 100 can be implemented using one or more components of the computing system 1700 described with reference to FIG. 17.

[0045] The subject 110 can be, for example, a patient undergoing the procedure. The subject 110 can be positioned in detection range of sensors 115 described further below. The subject 110 can undergo various operations as part of the procedure; for example, the procedure can be a surgery in which the subject is provided with anesthesia. The procedure can include any of various operations described herein, for example and without limitation, surgical, minimally-invasive surgical, robotic, diagnostic, therapeutic, and / or imaging operations, or various combinations thereof.

[0046] The system 100 can include or be coupled with one or more sensors 115. The sensors 115 can include any of various sensors capable of detecting information regarding the subject. For example, the sensors 115 can include one or more imaging sensors (e.g., CT, MRI, X-ray, ultrasound, camera, 3D camera, etc.), physiological and / or vital signs sensors (e.g., heart rate, pulse rate, heart rhythm, blood pressure, body temperature, respiration rate, respiration volume, neurological monitoring), or various combinations thereof. The sensors 115 can provide sensor data having values at one or more time points.

[0047] The system 100 can include a data processing system 105. The data processing system can include one or more features of computing system 1700 described with reference to FIG. 17. The data processing system 105 can be used to perform any of various functions described herein, including but not limited to operations for generating predictions of events and / or onset of events for the subject 110, as well as configuring (e.g., training, updating, fine-tuning, etc.) machine learning models 125 described further herein. The data processing system 105 can be implemented on a single computing device, or can have one or more components or operations implemented in a distributed architecture, such as to allow for model training, updating, and / or execution be performed on a first device, and user interface operations be performed on a second device.

[0048] The system 100 can include or be coupled with data input and / or output devices, such as a display interface 130 and / or a user input interface 135, which can incorporate features of various components of the computing system 1200. The data processing system 105 can receive inputs form the user input interface 135 such as text,speech, audio, image, and / or video data indicative of information regarding the subject, such as information provided by one or more users performing the procedure. For example, the data processing system 105 can receive unstructured data regarding a status of the procedure. The display interface 130 can be used to present an indication of the event predicted by the procedure monitor 120, which can include, for example, a type of the event, the likelihood of the event occurring within a particular time interval, an expected time for the event or of onset of the event, an anatomical feature associated with the event, or various combinations thereof.

[0049] Referring further to FIG. 1, the system 100 (e.g., the data processing system 105) can include at least one procedure monitor 120. The procedure monitor 120 can include one or more machine learning models, neural networks, algorithms, computerexecutable code, rules, heuristics, logic, data structures, algorithms, functions, or various combinations thereof to perform functions including receiving data regarding the subject 110 and using one or more machine learning models 125 to process the received data to predict one or more events regarding the subject.

[0050] The procedure monitor 120 can include at least one machine learning model 125, which can include any of various supervised and / or unsupervised machine learning models (e.g., models configured to classify data representative of events, such as to identify data that maps to whether certain events are or occur or not occur). The machine learning models 125 can include any one or more of decision trees, graph networks, random forest models, Bayesian models, regressions, support vector machines, gradient-boosted trees, or various combinations thereof. For example, the machine learning models 125 can include a plurality of random forest network models, which can be structured as decision trees that are configured (e.g., trained) using a feature bagging process, for example. The machine learning model 125 can include one or more neural networks, including, for example and without limitation, any one or more artificial neural networks, deep learning networks, convolutional neural networks, recurrent neural networks; various such neural networks can be useful for real-time processing of time-series data such as periodically detected and / or received data regarding a subject undergoing a procedure. The neural network can include a plurality of node arranged in one or more layers, such as an input layer, an output layer, and / or one or more intermediate layers. The data processing system 105 can configure the neural network by modifying or updating one or more parameters, such as weights and / orbiases, of various nodes of the neural network responsive to evaluating outputs of the neural network. The machine learning model 125 can be based on a random forest approach.

[0051] In some implementations, the machine learning model 125 used to predict events regarding the subject 110 is trained using data indicative of events. For example, the data processing system 105 can include or receive training data that includes a plurality of training data elements, each training data element including (1) at least one of example subject data regarding one or more example subjects or example sensor data (e.g., values of sensor data at one or more points in time) detected during a procedure performed on the one or more example subjects and (2) an indication of whether an event occurred for the example subject (e.g., whether the event occurred and a time of occurrence). Using various processes described herein, including but not limited to MMV, selected multivariate, and / or multiple forecast window-based training and architectures for the machine learning models 125, the machine learning model(s) 125 can be trained to predict events based on the training data. For example, the example subject data and / or example sensor data of the training data elements can be applied as input to the machine learning models 125 to cause the machine learning models 125 to generate an estimated output responsive to the input, and a comparison can be performed (e.g., using a loss function and / or optimizer) to determine a difference between the estimated output and the indication of whether the event occurred in order to update the machine learning models 125 according to the comparison, such as to modify one or more weights and / or biases of the machine learning models 125 until a convergence condition is satisfied. One or more outputs of the machine learning models 125 can be provided as one or more inputs to the display interface 130, as depicted in FIG. 1; in various implementations, the outputs can be provided to any one or more devices communicatively coupled with the procedure monitor 120. As described in further detail below with respect to FIGS. 11-16, the training of the machine learning models 125 can be performed in a manner such that the sensor data corresponds to a first period of time during the procedure, and the indication of whether the event occurred corresponds to a second, later period of time during the procedure (which may be immediately after the first period or may be separated by a period between the first and second periods), allowing the machine learning models 125 to be trained to effectively predict events / states of subject before onset of such events / states.

[0052] The system 100 can implement the machine learning models 125 trained using data from one type of procedure to update and / or inform the machine learning models125 for the same type of procedure or a different type of procedure. The machine learning models 125 can receive as input one or more features that include at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, heart rate, age, sex, body mass index, height, weight, medication information, fluid information, bleeding information.

[0053] The system 100 can be used for detecting multiple events. For example, the system 100 can determine an output prediction of a second event subsequent to the occurrence of a first event. The system 100 can use the occurrence of the first event as input to the machine learning models 125 to output an indication of the output prediction of the second event. The machine learning models 125 can receive input indicating type and dosage of medication used to treat the patient in response to the first event to the output prediction of the second event. For example, the system 100 can periodically input features regarding a patient during a procedure to the one or more machine learning models 125, e.g., at predetermined and / or varying intervals of time; the inputted features can include whether one or more prior events have occurred (and times of the prior events and / or data associated with the prior events) along with medication administered to the patient subsequent to the occurrence of the one or more prior events. The machine learning models 125 can be configured (e.g., trained) based on training data that indicates multiple events as well as medication information; the effects of prior events as well as medication on patient physiology can thus be used to more accurately predict potential events.

[0054] The data processing system 105 can receive input indicating a selection of a procedure being performed. For example, a member of a medical team can indicate (e.g., select) the procedure being performed in user input interface 135. Based on the input, the machine learning models 125 can be configured to be specific to the selected procedure. The machine learning models 125 can include procedure-specific models trained on procedure specific data and / or a composite model trained on data across multiple procedures. For example, training data elements can include a label indicating a type of procedure. At inference, the data processing system 105 can select a specific model corresponding to the selection, or can provide the selection to the composite model as a input or feature.

[0055] The procedure monitor 120 can be coupled with at least one subject parameter database 140. The subject parameter database 140 can include subject data such as age, height, weight, medical history records, medical image data for the subject, previous procedure data, etc. The subject data can be analogous to the data used to train the machine learning models 125 and / or can be used to update the machine learning models.

[0056] Referring further to FIG. 1, the procedure monitor 120 can receive input including at least one of the subject data regarding the subject or sensor data from one or more sensors 115, during a first period of the procedure. The procedure monitor 120 can apply the input to the machine learning model(s) 125 at any one or more points in time during the first period to cause the machine learning models 125 to generate a prediction of whether an event is likely to occur, during a second period subsequent to the first period, for the subject 110 and / or to generate a prediction of a state of the subject 110 (e.g., state and / or change of state). For example, the procedure monitor 120 can periodically (e.g., at a sampling rate of the sensors 115; every second; every ten seconds; etc.) use the machine learning models 125 to generate the prediction(s), and use the display interface 130 to present the indication of the prediction(s).III. Examples

[0057] While certain implementations have been illustrated and described, it should be understood that changes and modifications can be made therein in accordance with ordinary skill in the art without departing from the technology in its broader aspects as defined in the claims. Various examples of implementations of the technology are provided below.

[0058] Worldwide, more than 300 million surgeries are performed annually, a number expected to increase with the aging population and the consequent rise in comorbid conditions. In patients having non-cardiac surgery, intraoperative hypotension (IOH) under general anaesthesia is a frequent physiological derangement. In those undergoing liver resections, IOH has been reported in nearly half of the cases because of their complexity, positioning, substantial blood loss, and fluid shifts. Brief episodes of IOH, ranging from moderate to profound, have been shown to carry clinically significant consequences. IOH can be associated with a higher risk of postoperative morbidity due to acute renal failure, delirium, myocardial injury after non-cardiac surgery. Therefore, avoiding IOH can be important in procedures such as liver resections.

[0059] Intraoperative hypotension can be a relatively common adverse event that is associated with severe postoperative complications to renal, cardiovascular, and / or neurovascular systems. Intraoperative hypotension during surgery can be common in patients having non-cardiac surgery under general anesthesia. Intraoperative hypotension can be a frequent event associated with increased perioperative morbidity and mortality. Even a short duration of hypotension (e.g., 3-5 min) can lead to severe postoperative complications. Intraoperative hypotension can be multifactorial and can be associated with major post-operative complications, such as acute kidney injury, myocardial injury, and death. There can be multiple factors that affect the possibility of a patient experiencing a hypotensive episode. These factors can include age, BMI, systolic blood pressure, and other complex etiological factors such as health status, the type of anesthetic drugs administered, and dosage. Given such a complex array of factors that may be associated with IOH, it may be difficult to predict hypotension in advance of surgery. Predicting IOH can be clinically impactful to millions of patients.

[0060] The systems and methods of the present disclosure can predict the time of occurrence and / or provide a sentinel early warning that an IOH event is about to occur. The systems and methods of the present disclosure can use intraoperative data to predict an imminent IOH event throughout the course of surgery. The systems and methods of the present disclosure can be directed to a machine learning (ML) model to predict IOH with a “dial” on sensitivity and specificity to predict a first IOH episode during liver resection surgeries. The systems can include an intraoperative physiological monitoring system for respiratory, cardiac, and cognitive monitoring. The methods can include a machine learning algorithm that combines disparate physiological signals during surgery with preoperative data (e.g., information from the electronic medical record) and provides them as inputs to predict IOH within minutes of its occurrence. These systems and methods can provide an early warning system of IOH, which can permit anesthesiologists to exercise increased vigilance and / or to intervene before such an event occurs. The systems and methods can show robust performance in forecasting IOH during hepatobiliary surgery. These systems and methods can be extended to other intraoperative conditions, such as hypothermia or hypoxia.

[0061] Prediction and avoidance of intraoperative hypotension (IOH) can lead to less postoperative morbidity. Machine learning can be applied to predict IOH. Incorporating demographic and physiological features in an ML model can improve the performance ofIOH prediction. In addition, a “dial” feature can be added to alter prediction performance. The ML prediction model can be based on a multivariate random forest (RF) trained algorithm using physiologic time series data (e.g., a plurality of time physiologic time series data, 13 physiologic time series data) and patient demographic data (age, sex, and BMI) for adult patients undergoing hepatobiliary surgery. An implementation can include an adjustable, multi-model voting approach (e.g., multi-model voting algorithm) to improve performance in the challenging context of a dynamic, sliding window for which the propensity of data is normal (e.g., negative for IOH).

[0062] In one example, the study cohort included 85% of subjects exhibiting at least one IOH event. Males constituted 70% of the cohort, median age was 55.8 years, and median BMI was 27.7. The multivariate model yielded average AUC = 0.97 in the static context of a single prediction made up to 8 min before a possible IOH event, and it outperformed a univariate model based on MAP-only (average AUC = 0.83). The MMV model demonstrated AUC = 0.96, PPV = 0.89, and NPV = 0.98 within the challenging context of a dynamic sliding window across 40 min prior to a possible IOH event.

[0063] Machine learning in healthcare can be as a tool for prognostication, clinical decision support, and predicting complications. Machine learning strategies can provide a framework from which to estimate intraoperative events of interest from observed timeseries data (e.g., blood pressure or oxygen saturation). Machine learning can be applied in the complex interplay between surgery, patient demographics, anaesthetic factors, and their relationship with perioperative outcomes.

[0064] ML approaches can be applied to predict IOH events. Potential limitations in other work with the Hypotension Prediction Index (HPI) can include the need for high- frequency arterial waveform data (which can add cost to patient care and most anaesthetics delivered do not use), selection bias, selection of prediction-outcome pairs, reduced performance between backwards and forward case analysis, and complicated or inaccurate treatment recommendations. The HPI can overestimate the prediction of IOH. Another notable gap in other work using HPI is that liver resections were excluded in some HPI analyses due to possibly lower IOH prediction performance in that context.

[0065] Furthermore, in the clinical setting, anaesthesiologists can routinely rely on additional factors to predict the individual thresholds of hypotension in the operating room. For example, the amount of anaesthetic being administered could affect the patient’s bloodpressure. Several factors can be considered to determine the optimal blood pressure range, such as age and amount of anaesthetic being delivered.

[0066] To more closely mirror the clinician’s complex decision-making process at the bedside, the performance of a prediction algorithm that incorporates multiple dimensions of physiological monitoring data (e.g., cardiac, respiratory, and neuro monitoring) in combination with patient-specific factors (e.g., age, sex, and BMI) to forecast the occurrence of IOH can be investigated. Initial results can include the challenging scenario of major liver surgeries, which are hemodynamically complex and carry strong potential for substantial blood loss and fluid shifts that expose patients to the risk of IOH. The multivariate ML algorithm of the present disclosure can yield high performance for IOH prediction in liver surgery. The systems and methods of the present disclosure can include: (i) the incorporation of multivariate physiological time series data and patient-specific demographic information, (ii) a multi-model approach that includes an ensemble of ensembles to improve performance and practicality, and / or (iii) the ability to control the sensitivity and specificity of the prediction in real-time.

[0067] The study can draw from retrospective data for patients undergoing hepatobiliary surgery. Data were extracted from a database of physiological monitoring variables. With respect to guidelines reported in the multidisciplinary review of ML-based predictive models, the work was retrospective in nature with internal validation. Inclusion criteria were open partial or total liver lobectomy, age > 18 years at the time of surgery, availability of demographic data, and intraoperative physiological signals. Patients undergoing emergency or minimally invasive surgeries and those having multiple surgical procedures utilizing other surgical specialties were excluded.

[0068] The systems and methods of the present disclosure can include the combination of demographic and multivariate physiological time-series data incorporated into the prediction model. Demographic data can include the subject age at the time of surgery, body mass index (BMI), and sex. Physiological data drawn from anaesthesia monitoring can include 13-time series signals recorded at 1-minute intervals: diastolic arterial line (Art Line D), systolic arterial line (Art Line S), mean arterial pressure (MAP), heart rate as measured by pulse oximeter (HR, Oximeter), inspired oxygen fraction (FiO?), delivered oxygen flow (02 Flow), bispectral index (BIS Monitor), inspired desflurane, expired desflurane, minute volume ventilation (Minute Volume), respiratory rate (Resp), peak inspiratory pressure (PIP), and tidal volume (Vt). The saturated haemoglobinconcentration from the pulse oximeter reading (SpO2) can be included. Analysis of feature importance showed little influence of SpO2 on IOH prediction.

[0069] Time series data were considered 30 min after arterial line placement, when physiological signals mentioned above were present, and were low-pass filtered with a 3- minute averaging window width to reduce noise. Signals exhibiting missing data of 1-2 min were median filtered to impute minor gaps in the data. Training cases were classified as positive for IOH if they exhibited a drop in MAP below 65 mm Hg sustained for at least 2 min. The first IOH event was considered for each case, beyond which the subject was assumed to be under medical intervention and was not further considered in training or testing. Negative cases were those for which MAP was above 65 mm Hg throughout the observation period. Selection bias was minimized by ensuring forward analysis of physiological variables and future IOH (up to 8 min in advance) and not by the way the data was assembled.

[0070] For data extraction, the MD Anderson Anesthesia Universe was employed as a data repository for intraoperative physiological signals and other patient data (age, sex, etc.). Data was curated and extracted for patients who had undergone liver lobectomy surgeries. Data was extracted from 916 patients, all of whom had their surgeries at the UT MD Anderson Cancer Center between 2018-2023. Of the 916 subjects, 635 patients underwent partial liver lobectomy, 123 patients had complete right liver lobectomy, and 157 patients had complete left liver lobectomy.

[0071] Retrospective data from adult patients undergoing hepatobiliary surgery can be used. A random forest model can predict IOH events, which can be defined as an occurrence of the mean arterial pressure reaching a value of less than 65 mmHg, up to eight minutes before incidence of the event. Independent variables can include demographic features (e.g., age, body mass index, and sex), hemodynamic features (e.g., systolic and diastolic invasive blood pressure, mean arterial pressure, and heart rate), respiratory features (e.g., respiratory rate, minute volume ventilation, tidal volume, peak inspiratory pressure, inspired oxygen, and oxygen delivery flow), and / or anesthetic use (e.g., bispectral index and expired desflurane). A 10-minute sliding window (e.g., data interval) can be used to forecast IOH occurring within 8 minutes (e.g., forecast window) throughout a procedure. Metrics of model performance can include accuracy, precision, and recall (e.g., sensitivity). The contribution of each independent variable can be evaluated via feature ablation in the multidimensional model.

[0072] As summarized in Table 1 regarding data filtering and segment extraction, physiological series extracted for model building included the following signals at 1 -minute intervals: arterial line diastolic pressure, arterial line systolic pressure, mean arterial pressure (MAP), inspired desflurane, expired desflurane, BIS Monitor, heart rate (oximeter), O2 Flow, fraction of inspired oxygen (FiCh), peak inspiratory pressure (PIP), tidal volume, respiratory rate (resp), and minute volume. From the EHR, data regarding the patient’s sex, age at encounter, and body mass index (BMI) were extracted. To reduce noise, all of the time series were low-pass filtered. Instances of brief and / or spuriously missing data (e.g., a small number of 1 to 2 minute gaps) were filled by interpolation.

[0073] Table 1 : Time series signals extracted from one or more clinical data repositories and one or more electronic medical records. Demographic features such as sex, age, and BMI can be extracted either from one or more clinical data repositories or from one or more electronic medical records.

[0074] Intraoperative hypertension can be defined as a drop in MAP below 65 mmHg. The prediction model analyzed a 10-minute interval (e.g., data interval 205) of data sampled at 1-minute intervals from 13 time series signals (Table 1) along with sex, age, and BMI. For positive patients, only the 10-minute data prior to the first hypotensive episode was used (since the patient’s physiology is expected to be substantially perturbed in subsequent IOH episodes due to intervention). Subjects with less than 2 minutes of sustained hypotension episodes (e.g., a 1-minute “blip”) were excluded. Subjects with missing data of greater than 2 minutes in any signal were not included. Only IOH events after the first 30 min of arterial intubation were included. Moreover, subjects with either MAP less than 65 or MAP greater than 150 in the data interval 205 were excluded. Therefore, “positive” cases were taken as those for which the first hypotensive episode occurred after at least 30 minutes post arterial intubation. Following these exclusion criteria, 3 patients were excluded because of missing data, 76 patients were excluded because of missing signals and / or data (among the 13 in Table 1), and 54 patients were excluded because of hypotension occurring within 30 minutes of intubation. Negative data series were drawn from the 172 subjects who did not exhibit an episode of hypotension throughout their surgery.

[0075] As summarized in Table 2, these inclusion and exclusion criteria for model training and testing yielded a total of 783 cases for prediction. Of these, 611 subjects experienced a (first) hypotensive event 30 minutes after arterial line intubation and 172 subjects did not have an IOH event (e.g., event 215) throughout their surgery.

[0076] Table 2 illustrates a summary of data for training and testing of the initial embodiment of the invention for prediction of IOH. Total cases represent the total numberof cases that were used after 133 / 916 datasets were removed per the exclusion criteria. Positive cases are the number of cases in which the first case of hypotension occurred greater than 30 minutes after arterial incubation. Negative Cases are the number of cases that never had a hypotension episode throughout their surgery. Positive datasets are the same as positive cases, with a single 10-minute data interval 205 extracted for each positive patient. Negative datasets are the number of datasets extracted from negative patients. To improve class balance, 3 or 4 data intervals 205 were extracted from each negative data set.

[0077] FIG. 2A illustrates a plot 200 of a positive episode. As illustrated in FIG. 2A, for each positive case, a 10-minute data interval 205 was extracted (for each physiological signal in Table 1) occurring 5 minutes prior to the first hypotensive event (e.g., event 215). For example, the first signal in the data interval 205 is 15 minutes prior to the IOH event, and the last signal in the data interval 205 is 5 minutes prior to the IOH event. The data interval 205 can be a sliding (e.g., shifting, moving) window. The data interval 205 can slide along minute-by-minute and offer a prediction each minute. The 5-minute interval (e.g., forecast window 210) can represent the time window within which the anesthesiologist could choose to intervene, given an “early warning” from the system. In various implementations, the system 100 can operate in accordance with any of various intervals for the forecast window 210, including but not limited to 2 minute, 3 minute, 4 minute, 6 minute, 10 minute windows, or various intervals within such timings, for example and without limitation. The forecast window 210 can be a sliding (e.g., shifting, moving) window. The forecast window 210 can be a variable forecast window (e.g., variable forecast interval). The system 100 can include one or more forecast windows 210. Multiple forecast windows 210 are described below with reference to FIGS. 15 and 16. FIG. 2A illustrates the data interval 205, the forecast interval (e.g., forecast window 210), and the IOH event (e.g., event 215) for a positive case. The various physiological signals from Table 1 are plotted. A minimum threshold of 65 mmHg is marked by the horizontal dashed line, for which a drop in MAP below this level for more than 1 minute constitutes an IOH event.

[0078] In another example, 841 patients were included in training and testing: 684 positive for at least one IOH event, 157 negative for IOH throughout the procedure. 684 positive data intervals were extracted up to 8 minutes prior to IOH along with 684 negative data intervals, achieving class balance via random sampling of negative intervals. Training and test data were parsed without overlap in 90: 10 proportion with 11 -fold cross validation. The multi-dimensional RF model (FIG. 2B) showed overall median accuracy of 96% inpredicting a drop in MAP below 65 mmHg 8 minutes in advance of the event. Median precision and recall were 94% and 89%, respectively. As shown in Table 3, body mass index and age can be the more important demographic variables, and arterial line systole, diastole, and MAP can be the most important physiological signals, followed by tidal volume, respiratory rate, and BIS monitor signal.

[0079] FIG. 2B illustrates a plot 250 of a positive episode. FIG. 2B illustrates physiological time series data. The data can include the data interval 205. The data interval can be a 10-minute data interval. The data can include a forecast interval (e.g., forecast window 210). The forecast interval can be a 3 minute to 8 minute-forecast interval. The data can show a positive case of hypotension (e.g., hypotensive event, event 215). The data can be time-shifted to time t = 16 minutes. The legend denotes 13 physiological signals incorporated in the model, which also included BMI, age, and sex as demographic features.

[0080] FIG. 2B illustrates the segmentation of time series data into the data intervals 205 (e.g., the input data from which a prediction of IOH is made) and the forecast intervals (e.g., the time period between the data interval and a possible IOH event, forecast window 210). For illustration purposes, the time stamp in FIG. 2B denotes time-shifted series relative to the start of the data interval and does not describe the actual time during surgery. The data interval 205 can be taken to be a 10-minute interval prior to a possible IOH event. The focus can be on a relatively short forecast interval up to 8 min in duration, selected in part due to the relatively fast hemodynamic swings that have been described in liver surgery. Thus, the proposed interval along with the sliding window approach can be fairly realistic with respect to clinical decision making in the operating room and could in turn provide flexibility to monitor, react, and promptly intervene to prevent IOH. Shorter (~5 min) and longer (~15 min) intervals are possible. The forecast interval duration can be randomly varied from 3 min to 8 min according to a uniform distribution, and as shown in FIG. 2B, for a data interval spanning t = 1-10 min, the IOH event could therefore occur anywhere from t = 13 min to t = 18 min.

[0081] A number of model architectures can be considered for investigation of IOH prediction, including artificial neural networks (ANN), gradient-boosted trees (GBT), and random forest (RF) classifiers. While ANNs are a potentially powerful approach, they can use large training sets and can challenge interpretability. Simpler ML approaches such as GBTs and RFs can yield a reasonable degree of predictive performance, a high degree of interpretability and explainability, and have widely available shared software libraries thatfacilitate implementation and reproducibility (e.g., the sktime ML Python library for time series data). For these reasons, the initial studies can be based on RF supervised classifiers with 100 decision trees combined using the ColumnEnsembleClassifier() function, although alternative architectures can be used.

[0082] Several variations in model training and testing can be investigated to rigorously assess the potential for bias and to evaluate performance under a broad variety of conditions. The differences in accuracy can be analyzed using a non-parametric Mann- Whitney U test, with p-value < 0.05 interpreted as evidence of statistical significance. Models can be trained with and without Z-score signal normalization. Two variations in traimtest proportion can be considered (90: 10 and 80:20) to investigate the trade-offs between a larger training set and fewer test samples, hypothesizing slight improvement for the former due to a larger volume of training data. As an alternative to the 8-minute forecast interval (randomly varied from 3-8 min), models with a fixed 5-minute interval can be investigated. Finally, three variations in class balance can be trained and tested: (i) the natural imbalance of the data (as shown, ~6x in favor of positive cases); (ii) balancing negative and positive datasets by sampling three negative datasets (non-overlapping, separated by at least 20 min) from each negative case plus three negative datasets (similarly non-overlapping) from positive cases sampled at least 30 min prior to the IOH event, referred to as “3x sampling”); and (iii) balancing achieved by sampling six non-overlapping datasets from negative cases (referred to as “6x sampling”). Models (ii) and (iii) therefore involved patient-level combined with segment-level splitting to achieve class balance and is recognized to carry potential bias. The bias can be partly mitigated by sampling nonoverlapping data intervals separated by at least 20 min and assumed to be independent, recognizing that age, sex, and BMI are common at the level of segment and therefore not strictly partitioned.

[0083] Two application scenarios can be considered for analysis of model predictions. The first can include a relatively simple “static” scenario in which a single 10- minute data interval was taken as input for each case, occurring up to 8 min prior to a possible IOH event, as illustrated in FIG. 2B. The static scenario can represent an optimistic upper bound on model performance under the idealized condition that half of the validation data were positive or negative for IOH.

[0084] Table 3 illustrates an analysis of rank-order importance of features incorporated in the model, including demographic variables (e.g., BMI, age, and sex) and physiological time series signals. Each row corresponds to a model trained and tested without the (ablated) feature listed. “NONE” refers to the base model incorporating all features. “Median Accuracy” therefore refers to the accuracy of the “ablated” model (e.g., the model without the respective feature). BMI and age can be the more important demographic variables. Arterial line systole, diastole, and MAP can be the most importantphysiological signals, followed by tidal volume, respiratory rate, and BIS monitor signal. Incorporation of multiple feature inputs (upwards of 14-16 features listed) can be important to robust predictive performance.

[0085] FIG. 3 A illustrates a plot 300 of an extended period of negative data prior to a possible IOH event. A scenario can be considered in which the 10-minute data interval advanced in 1 -minute steps through an extended period of truly negative data prior to a possible IOH event. With the forecast interval up to 8 min and a possible IOH event timeshifted to t = 40 min, the duration of the truly negative period ranged from 32 min (for an 8- minute forecast interval) to 37 min (for a 3-minute forecast interval). Sliding the 10-minute data window in 1 -minute steps through a preponderance of truly negative instances and forming a prediction at each step can be referred to as the dynamic sliding window scenario, illustrated in FIG. 3A. Model performance can be challenged (e.g., a reduction in PPV) in the dynamic scenario due to the high prevalence of negative instances. This can be analogous to the more realistic clinical scenario of an early warning system for which most of the samples are truly negative, and a high level of PPV with few false alarms is required for the system to be useful. Testing in the sliding window scenario involved 20 cases (10 positive + 10 negative) that were held out and previously unseen in training at either the patient-level or segment-level.

[0086] Three main variations in the predictive model can be developed and tested. The first variation can include an RF model trained as detailed above but with only MAP as an input feature - referred to as the univariate “MAP-Only” model.

[0087] To evaluate the importance of multi-dimensional inputs that more closely reflect the actual clinical considerations of an anaesthesiologist, a multivariate RF model can be developed that takes the 3 demographic variables and 13 physiologic time series signals described above as input - referred to as the single multivariate model (in contrast to multi-model). The importance of individual features contributing to multivariate model predictions can be evaluated via ablation (e.g., removing a given feature, retraining, and retesting in the absence of the ablated feature). The reduction in model performance can yield a surrogate for feature importance, and the features can be rank ordered accordingly.

[0088] In light of the challenge presented by a large prevalence of truly negative instances in the dynamic scenario, a multi-model voting approach can be developed that runs multiple, separately trained RFs in parallel, each contributing a vote. Whereas thesingle multivariate model can represent a single RF resulting from an ensemble of decision trees, the MMV approach can include multiple RFs. Given that an RF is itself an ensemble approach, the MMV approach can be considered an “ensemble of ensembles.” With MMV, each of the 11 RF models from 11 -fold cross validation described above contributes a prediction that is taken as a vote in forming a final prediction, e.g., by simple majority (Nvotes > 6). FIG. 3 A illustrates the MMV approach for a single case in the dynamic scenario, with the number of votes (Nvotes) shown on the right axis and True-Negative (TN), True-Positive (TP), False-Negative (FN), and False-Positive (FP) predictions marked. The MMV approach can be further investigated with Nvotes taken as an adjustable parameter, allowing the anaesthesiologist to control the sensitivity and specificity of the model by dialling the Nvotes threshold lower (for greater sensitivity) or higher (for greater specificity).

[0089] Predictive performance can be evaluated in terms of binary hypothesistesting metrics of TP predictions (correctly predicting an IOH event within the forecast interval), TN predictions (correctly predicting the absence of an IOH event), FP predictions (incorrectly predicting an IOH event), and FN predictions (incorrectly predicting that an IOH event will not occur). Corresponding metrics of Accuracy, Sensitivity, Specificity, positive predictive value (PPV), and negative predictive value (NPV) can be computed. Receiver operating characteristic (ROC) curves were evaluated by varying the probability threshold (from 0 to 1) in model predictions to be considered positive, and the area under the ROC curve (AUC) can be evaluated by numerical integration. The resulting distributions can be analyzed using a non-parametric Mann-Whitney U test computed in Python to compare the observed difference in mean value between distributions, with p- value < 0.05 taken as statistically significant.

[0090] FIG. 3B illustrates a plot 350 of a negative episode. As illustrated in FIG. 3B, for the negative cases, three or four 10-minute data intervals 205 sampled throughout each case (non-overlapping and otherwise following the inclusion / exclusion criteria detailed above) were extracted. The multiple intervals extracted for each negative case yielded a form of class balance (e.g., an equal number of positive and negative datasets), as summarized in Table 2. The data interval 205 can be a sliding (e.g., shifting, moving) window. The data interval 205 can slide along minute-by-minute and offer a prediction each minute. The forecast window 210 can be a sliding (e.g., shifting, moving) window. The forecast window 210 can be a variable forecast window (e.g., variable forecast interval). The system 100 can include one or more forecast windows 210. Multiple forecast windows210 are described below with reference to FIGS. 15 and 16. FIG. 3B illustrates the data interval 205, the forecast interval (e.g., forecast window 210), and the absence of an IOH event (e.g., non-event 305) for a negative case. The various physiological signals from Table 1 are plotted. Because there is no hypotensive event, the phase of data illustrated here is illustrative of the data considered for intervals absent of IOH.

[0091] The predictive model was based on a random forest algorithm implemented using the sktime library in Python. Random Forest can include a machine learning model that samples the training datasets randomly to create a large number of decision trees. It then uses the majority vote of these decision trees to yield a prediction. The individual decision trees work in tandem to predict the value of a target variable based on the several provided input values. Random forests can be a general approach that can provide a high degree of accuracy from generating a large number of decision trees in deciding a final result based on the majority. The machine learning models can include at least one of k-nearest neighbor, logistic regression, artificial neural network (ANN), random forest, support vector machines, or gradient-boosted trees.

[0092] The systems and method of the present disclosure can predict an IOH event within a given forecast window 210. This can constitute an early warning against an event that is about to occur; the prior art mentioned above either aims to predict IOH occurring strictly within the first 10 minutes of intubation (early stage of the procedure only) or at any point during the procedure (amounting to a risk predictor). The invention could be implemented as a continuous monitor to alert the clinical team of imminent IOH throughout the procedure.

[0093] A TimeSeriesForestClassifier from the sktime library can be used, since the library can handle both time series signals (Table 1) as well as static variables (age, sex, BMI). ColumnConcatenator from sktime can be used to combine all the individual decision trees produced by TimeSeriesForestClassifier. 10-fold cross validation in training and testing can be employed. The 10-fold cross validation divides the data into 10 parts (9 training sets and 1 testing set) and repeats the training and / or testing separately 10 times, each time producing a differently trained model and a distinct test set. Given the data described above, each model therefore had 90% (1100 datasets) for training 10% (122 datasets) for testing.

[0094] The performance of the initial embodiment was evaluated using metrics: accuracy (the percentage of correctly predicted outcomes out of the total number of predictions); precision (also called positive predicted value, PPV, and relating the percentage of positive prediction that were truly positive); recall (also called sensitivity, the total percentage of true positives out of the number that should have been predicted positive); and F-l score (the harmonic mean of Precision and Recall). The formulae for each metric are given below:where TP is the number of True Positive values, FP is the number of False Positive values, TN is the number of True Negative values, and FN is the number of False Negative values.

[0095] FIGS. 4A-4D illustrate the study cohort. Inclusion criteria yielded a total of 918 cases undergoing open liver lobectomy. Of these, 723 were partial lobectomy, 98 were left lobectomy, and 97 were right lobectomy. Median age at the time of surgery was 55.8 years [range 20 - 90, FIG. 4A], Median BMI was 27.7 [range 20.0 - 35.2, FIG. 4B], and males constituted 70% of cases [642 / 918, FIG. 4C], At least one IOH event was observed (at least 30 min after arterial line placement and sustained for at least 2 min) in 85% of cases [783 / 918, FIG. 4D], and the remaining 15% (125 / 918) exhibited MAP >65 mm Hg for the duration of their case. Among the 783 positive cases, 73% (570 / 783) were male, approximately consistent with the male:female proportion in cases overall.

[0096] Among the basic model variations investigated, there was no statistically significant difference in performance with and without signal normalization (p=0.132), consistent with scale invariance of the underlying RF approach; therefore, models were trained without signal normalization. A statistically significant improvement can be observed in performance between 90:10 and 80:20 split in traimtest datasets (p=0.002) due to a somewhat larger training set, and the former was used throughout. There was no evidence of a statistically significant difference in performance between the 8-min forecast interval (e.g., variable 3-8 min) and the (fixed) 5-minute interval (p=0.242), and the formerwas used throughout in the interest of increased variety in the training set. The three variations in class balance resulted in: (i) 6x imbalance in favor of positive cases (783) vs negative cases (135); (ii) balanced datasets via 3x sampling of 135 negative cases (giving 270 negative datasets) plus 513 negative datasets drawn from positive cases sampled at least 30 min prior to the IOH event; and (iii) balanced datasets via 6x sampling of negative cases to yield 783 negative and positive datasets. The imbalanced dataset exhibited a statistically significant reduction in performance compared to the 3x and 6x sampling (average AUC = 0.84 compared to 0.91 (p=0.002) and 0.97 (p=0.003), respectively); the 6x sampling dataset can be used throughout to achieve class balance while mitigating bias in multiple sampling of negative datasets assumed to be independent.

[0097] As a starting point, the performance of the MAP-only univariate model (e.g., a single RF model with MAP as the sole input feature in the relatively simple static scenario), is shown in FIGS. 5A and 5B. Performance overall is modest, exhibiting average AUC = 0.83 (over 11 folds) and median Accuracy = 0.73, Sensitivity = 0.69, Specificity = 0.79, PPV = 0.77, and NPV = 0.71.

[0098] FIG. 6A illustrates a plot 600 of performance for prediction of an event, which includes a summary performance of an embodiment for prediction of IOH. Embodiments were tested using the datasets described above with 10-fold cross validation for prediction of IOH. As summarized in FIG. 6A, the median accuracy was 94.21%, median precision was 92.47%, median recall was 90.16%, and median Fl score was 93.02%. The performance can correspond with the study including 783 patients.

[0099] FIG. 6B illustrates a plot 650 of performance for prediction of an event, which includes a summary performance of an embodiment for prediction of IOH. Overall performance for prediction of hypotension using the multi-dimensional random forest model shows an overall accuracy of 96.0% in predicting a drop in MAP below 65 mm Hg 10 minutes in advance of the event. The performance can correspond with the study including 841 patients.

[0100] FIGS. 7A and 7B illustrate the performance of a multivariate model (e.g., multivariate RF model) in a static scenario. Performance overall is markedly improved, demonstrating AUC = 0.97 (average over 11 folds) and median Accuracy = 0.95, Sensitivity = 0.86, Specificity = 0.93, PPV = 0.94, and NPV = 0.88.

[0101] The importance of individual features in the model is shown in FIG. 7C, where the horizontal axis denotes the ablated feature. Age and BMI showed the greatest importance in the prediction, followed by a combination of hemodynamic features (Art Line S, Art Line D, and MAP), respiratory features (Tidal Volume and Resp), and anaesthetic delivery (Inspired Desflurane). Other features individually exhibited less influence on the model overall, but were maintained in the model, since they may contribute to the aggregate. FIG. 7C illustrates a plot 700 of an ablation study. The ablation study is designed to examine the importance of features underlying the prediction of IOH. The ablated feature is shown on the horizontal axis, and models exhibiting the largest reduction in performance imply a higher level of importance for the ablated feature. “NONE” refers to the model with all features included. To gain insight on model explainability and the factors and / or signals that are likely more or most important in the prediction, an “ablation study” (e.g., knockout study) was performed in which a single feature (e.g., any of the 13 signals in Table 1, age, sex, or BMI) was eliminated from the data. Models trained in the absence of any single feature are called “ablation models.” Accordingly, the observation that an ablation model exhibiting a large drop in performance suggests that the ablated feature carries high importance. As illustrated in FIG. 7C (RIGHT), the most important features underlying the prediction of IOH can include age, BMI, arterial line diastolic pressure, tidal volume, and MAP.

[0102] The overall accuracy of 94% is quite good for a basic machine learning model. However, even that level of accuracy may not be good enough for implementation as a real-time monitor, since it may imply a high frequency of false positive (false alarms) in real use. For example, in a case spanning 300 minutes (5 hours), and with the system analyzing data every 1 minute, a system with -90% accuracy would report -30 false alarms during the case. The challenge is the enormous “class imbalance” between negative instances (almost always) and positive instances (e.g., first event, first hypotensive event).

[0103] FIGS. 8 A and 8B illustrate the performance of a multivariate model in a dynamic scenario. Deployment of an IOH prediction model as a real-time early-warning system may need to operate with a high degree of PPV to avoid false alarms. Performance of the multivariate RF model in the more challenging, dynamic sliding window scenario is shown in FIGS. 8 A and 8B, where testing spanned a prolonged period, t = 1-40 min, within which the first 30 min was truly negative, prior to a forecast interval up to 8 min and possible instance of IOH at t = 40 min. Analysis was performed on 10 positive test cases,amounting to 11x30 = 330 negative samples and 11x10 = 110 positive samples). A substantial drop in overall performance is evident: average AUC dropped from 0.97 to 0.84, median Accuracy from 0.95 to 0.86, Specificity from 0.93 to 0.88, and PPV from 0.94 to 0.63. The large drop in PPV in the dynamic scenario particularly motivated development of the MMV approach.

[0104] FIGS. 9A-9G illustrate the performance of a multi-model voting approach. The MMV approach can run 11 separately trained RF models in parallel and evaluates the resulting 11 votes to yield a prediction. Moreover, the sensitivity and specificity of the MMV approach can be controlled by adjusting (e.g., dialing) the Nvotes threshold for positive prediction. FIGS. 9A-9G illustrate the MMV performance and influence of the adjustable threshold in the dynamic sliding window scenario. The 11 ROC curves in FIG. 9A show the dependence on Nvotes, which is largely unaffected over the range Nvotes = 1-5 and decreasing for Nvotes > 6, illustrated further in terms of AUC in FIG. 9B. FIGS. 9C and 9D show the anticipated trade-offs in sensitivity and specificity with adjustment of Nvotes. In the dynamic scenario for which negative samples outnumber positives by at least a factor of 3, FIGS. 9F and 8G show PPV and NPV to be optimal in the range Nvotes 6-8.

[0105] FIGS. 10A and 10B illustrate the performance of a multi-model voting approach. The performance of the MMV approach can be evaluated in the dynamic scenario with a nominal value of Nvotes = 6 out of 11 (e.g., a simple majority), as summarized in FIGS. 10A and 10B. Compared to FIGS. 7A and 7B, this scenario presents a more challenging, realistic context in which the propensity of instances is truly negative. Compared to the single-model approach in FIG. 8A and 8B, MMV demonstrated improved AUC (0.96) and median Accuracy (0.98), Sensitivity (1.0), Specificity (0.96), PPV (0.89), and NPV (0.98). The improvements are statistically significant (p < 0.05) compared to the single-model predictions.

[0106] Liver resections can have a high incidence of hemodynamic disturbances, including IOH, likely related to large fluid shifts, extreme positioning, and high use of vasopressors. Machine learning methods can be employed to predict IOH in various clinical settings outside of liver surgery. The systems and methods of the present disclosure can use multivariate ML models using forward analysis of variables to predict, with the aid of invasive blood pressure monitoring, the first episode of IOH during open liver resections by integrating demographic data (e.g., age, BMI, and sex) with 13 physiological time series signals recorded at 1 -minute intervals.

[0107] The multivariate model can outperform a univariate (MAP-only) model, and a fairly wide variety of hemodynamic and respiratory time-series signals can be important to reliable prediction. This finding contrasts with a study indicating that MAP may perform as good as the HPI at set thresholds of 72 or 73 mmHg. The dynamic sliding window scenario presented the real-world prevalence challenge of sequential data for which a preponderance of the data is truly negative for IOH, and an MMV technique can maintain PPV under such conditions. The MMV approach can present the capability for the anaesthesiologist to dial the sensitivity and specificity of the model according to their judgment with respect to factors related to the patient or phase of the procedure.

[0108] Based on AUC, the HPI in the internal validation cohort has shown the highest performance at 5 min, with the lowest being in the Jacquet-Lagre'ze’s model. Compared to those models, the present MMV strategy demonstrated a relatively high performance (AUC = 0.96).

[0109] The systems and methods of the present disclosure can use only data relating to an intraoperative context, whereas other methods can use a mixed population of operative and ICU patients. Other methods can include data from invasive and non-invasive arterial waveform analysis using HPI, whereas systems and methods of the present disclosure can use only recording from invasive arterial blood pressure without waveform patterns. Other methods can observe in their forward approach without a gray zone a low PPV (0.52), whereas the MMV approach can maintains high PPV. This suggests that the systems and methods of the present disclosure could be more clinically actionable in predicting whether a particular patient will truly develop IOH.

[0110] Additionally, other methods can use intraoperative arterial waveform analysis, such as the HPI model, to predict IOH. Other methods can use local trends of arterial blood pressures to show how certain waveform shapes were associated with IOH. The study can show good predictive performance based on the reported AUC (>0.9). The rationale for using arterial waveform contour lies in identifying abnormal compensatory mechanisms before IOH develops via changes in the waveform. However, pulse contour analysis technology needs frequent calibration in patients with low systemic vascular resistance or after changes in vasopressor dosage. The systems and methods of the present disclosure may not use waveform contour analysis. Rather, the multivariate ML model described above can incorporate demographic data in combination with arterial bloodpressure values, respiratory parameters, and anaesthetic time series data for prediction instead of a single physiological parameter.[OHl] Other methods can use a deep learning convolutional neural network to construct several prediction models of IOH in a conglomerate of non-cardiac surgical patients employing (a) an arterial-pressure-only model, (b) an invasive multichannel model (arterial blood pressure, electrocardiograph, photoplethysmography, and capnography), (c) a photoplethysmography-only-model, (d) a non-invasive-multichannel-model and (e) a non- invasive hybrid model. The invasive multichannel model can share some of the physiological parameters used in the prediction model reported above, demonstrating AUC = 0.91, sensitivity = 0.86, and specificity = 0.86. The MMV model reported here yielded a somewhat higher AUC (0.96), recognizing the need for external validation. Other models can include an interpretable predictor and interpretable methods. The model can show high performance in the internal validation phase, which was somewhat reduced in external validation. The other methods can use a mixed population of patients, waveform shape analysis and Fourier transform for each blood pressure cycle, and use of a “gray zone” between MAP = 65 mmHg and 75 mmHg.

[0112] Feature ablation analysis can shed light on the importance of the multiple intraoperative variables contributing to the model developed. The three highest-ranking parameters in the prediction model were age, BMI, and diastolic pressure. While age and BMI variables can provide value in predicting postinduction hypotension, they do not lend themselves to real-time monitoring / early warning systems in the operating room.

[0113] Accurate alerts on potentially imminent IOH can allow anaesthesiologists to intervene and improve postsurgical clinical outcomes. It could also be of value to report the relative uncertainty associated with the prediction (e.g., a score from 1 to 11 corresponding to Nvotes in the MMV approach). The methods can include the ability to control the sensitivity and specificity of the model by adjusting the Nvotes threshold. A clinical application of such an approach could allow the anaesthesiologist to “dial up” the number of votes in a relatively healthy patient for surgery, indicating that some level of tolerability of hypotension while “dialing down” for a patient with multiple co-morbidities and a lower threshold to intervene early.

[0114] The systems and methods of the present disclosure can be characterized by (a) retrospective study design, which may lead to predictive bias, (b) the arbitrary but well-accepted definition of IOH (<65 mm Hg sustained for at least 2 min), (c) ML methods that can be subject to bias introduced in training data, (d) the prediction of the first IOH event (only), and (e) limitation to internal validation, with testing on external datasets recognized as an essential area of future work. Class imbalance intrinsic to the data (6x in favor of positive cases, consistent with prevalence of IOH in these surgeries) necessitated a combination of patient-level and segment-level splitting of the data, with 6x sampling of negative data intervals (non-overlapping, separated by at least 20 min) necessary to achieve class balance. Segment-level splitting in training for the “static” scenario can carry potential bias. Larger datasets can enable a larger volume of training data balanced at the level of the patient. However, the “sliding window” scenario involved testing with strict patient-level splitting in 20 unseen cases, confirming performance from the “static” scenario and suggesting that possible leakage effects were minimal.

[0115] Work on perioperative outcomes can suggest an interaction between IOH, deep levels of hypnosis, and low minimum alveolar concentration (MAC). The present disclosure can include “anaesthetic dose” and depth of hypnosis as variables; however, MAC data was not available in the data registry. Furthermore, patients receiving desflurane as the main volatile anaesthetic for the maintenance of hypnosis were included. In addition, despite including heart rate as an input variable in the model, the algorithm may not differentiate between endotypes of IOH.

[0116] The present model has shown that it is capable of predicting an IOH event up to 8 minutes before the event, which can be a somewhat shorter window overall than that used by the HPI (5, 10, and 15 minutes). Recognizing the flexibility of the MMV approach described above, the model can include short, medium, and long range predictors (e.g., with short-range models optimized for sensitivity (reduced Nvotes threshold) and long-range models optimized for specificity (higher Nvotes threshold)). The strategy can be adapted to multiple instances of IOH and in response to treatment and specific patient populations.

[0117] In conclusion, the rate of IOH is high in patients undergoing liver resection surgery (e.g., 85% in this cohort). The overall performance of the MMV predictive model was high, achieving AUC = 0.96, median PPV = 0.89, and median NPV = 0.98 within the challenging context of a dynamic sliding window for which the propensity of data is normal (truly negative for IOH).SYSTEMS AND METHODS OF MULTI-MODEL VOTING PREDICTION OFPHYSIOLOGICAL STATES OF A SUBJECT

[0118] Systems and methods in accordance with the present disclosure can incorporate multi-model voting (MMV) to allow for more accurate prediction of physiological states of a subject, including real-time prediction during a procedure performed on the subject. In the MMV approach, a plurality of models can be independently trained (e.g., 11 models trained via 11 -fold cross validation). At various instances during the procedure, such as periodic and / or user-trigger instances, the models can compute a prediction (e.g., in parallel), and a vote can be determined according to the computed predictions. The vote can include, for example, determining whether at least a threshold number of models predict a same state and / or event for the subject. At any given instance, for example, 9 models might predict “negative” for IOH, and 1 model might predict “positive.” The threshold (e.g., majority; plurality; threshold that may be less than half the total number of models) vote can be taken as the MMV output. This can provide a strong filter against false alarms. In initial studies run on cases negative for IOH (e.g., as described above with reference to FIG. 3B), MMV can reduce the number of false alarms from about 30-50 false alarms to about 3 false alarms.

[0119] Various filtering strategies can be envisioned for implementation of the systems and methods of the present disclosure for real-time monitoring. Examples can include low-pass filtering of outputs, recursive filters, voting by a single model over a small interval (e.g., 3 minutes), a sensitivity dial, an ordinal / numerical scale, and combinations of these with the MMV method.

[0120] FIG. 11 illustrates a method 1100 to generate a prediction of a physiological event. The method 1100 can be implemented by any of various systems and devices described herein, including but not limited to the system 100 and / or the procedure monitor 120 described with reference to FIG. 1. The method 1100 can be implemented in any of various modalities, including but not limited to real-time, intraoperative techniques in which one or more aspects of the method 1100 are executed responsive to real-time sensor data detected regarding a subject on which a procedure is being performed, as well as any of various asynchronous implementations (e.g., for testing or validation, or updating of machine learning models of the procedure monitor 120). The method 1100 can be implemented using a plurality of machine learning models configured to perform MMV.

[0121] The method 1100 can be performed, for example, in conjunction with signals regarding the subject, such as at least one of record data regarding the subject or sensor data from one or more sensors that monitor the subject during the procedure. For example, the subject (e.g., patient, user) can undergo a procedure such as a surgical procedure. The subject can be at risk of any of various intraoperative events, such as intraoperative hypotension, which can include an event associated with severe postoperative complications to renal, cardiovascular, and / or neurovascular systems. The method 1100 can assist the medical team (e.g., surgeon, anesthesiologist, nurse, staff) by providing timely predictions (e.g., alarms, alerts, warnings) of an adverse event (e.g., imminent adverse event) such as intraoperative hypotension. Based on the existence and / or timing of the warning, this can allow the medical team to decide how to respond to the potential adverse event.

[0122] In brief summary, the method 1100 can include monitoring one or more features of a subject (1105). The method 1100 can include applying, as input to a plurality of machine learning models, the one or more features to cause the plurality of machine learning models to each generate a candidate prediction of an event of the subject (1110). The method 1100 can include determining an output prediction based on each candidate prediction (1115). The method 1100 can include outputting an indication of the output prediction (1120).

[0123] Referring to FIG. 11 in further detail, the method 1100 can include monitoring one or more features (e.g., first features) of a subject (1105). The one or more features can be monitored by, for example, any of various procedure monitors 120 described with reference to FIG. 1. The one or more features can include features monitored in real-time. Data related to the one or more features can be received by the procedure monitors as a stream of data (e.g., data stream). The one or more features can include physiological features. The one or more features can include at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate. The one or more features can include signs (e.g., vital signs, multivariate vital signs). The one or more features can include physiological signals. The one or more features can include at least one of body temperature, pulse rate, respiration rate, or blood pressure.

[0124] The one or more features can be monitored using one or more sensors. The one or more sensors can include, for example, any of various one or more sensors 115described with reference to FIG. 1. Data from the one or more features can be received and / or monitored by a data processing system. The data processing system can include, for example, any of various data processing systems 105 described with reference to FIG. 1. The data can be obtained from or stored in, for example, any of various subject parameter databases 140 described with reference to FIG. 1. The data from one or more features can be sampled at various intervals. For example, the one or more features can be sample at one second intervals, 30 second intervals, 1 -minute intervals, 2-minute intervals, or 5 -minute intervals. The intervals can be frequent enough to sample enough data for predictive capabilities. The data can be obtained from the one or more sensors. For example, a heart rate monitor coupled to the subject can sample the subject’s heart rate at a specified time interval. For an intraoperative early warning system to predict changes in physiological status during surgery, the one or more features can be indicative of an event, such as an intraoperative hypotension event. By monitoring the one or more features, the method 1100 can give an indication of an imminent adverse event and permit intervention before the adverse event occurs.

[0125] The one or more features of the subject can be monitored during one or more periods relating to the procedure, including during a first period (e.g., a monitoring period). The first period can be a period during a procedure performed on the subject. The first period can occur prior to the event to be predicted (e.g., the event may not have yet occurred during the first period in which the features are monitored, such as to occur or be predicted to occur during a second period subsequent to the first period as described further below). The procedure can include a surgery (e.g., bariatric surgery, breast surgery, colon and rectal surgery, endocrine surgery, general surgery, gynecological surgery, hand surgery, head and neck surgery, hernia surgery, minimally invasive surgery, neurosurgery, orthopedic surgery, ophthalmological surgery, outpatient surgery, pediatric surgery, plastic and reconstructive surgery, robotic surgery, thoracic surgery, trauma surgery, urologic surgery, or vascular surgery). The event can include a hypotensive event. For example, the event can include an intraoperative hypotension. The first period can include the data interval 205. The length of the first period can be selected randomly during training of the plurality of machine learning models.

[0126] The first period can include a period of time during which the one or more features is monitored. The first period can have a duration that represents a data interval. The first period can include a period of time during which data relating to the one or morefeatures is collected from the subject. The first period can be in a range of 1 minute to 20 minutes. For example, the first period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. Similar to the sampling rate of retrieving data from the sensors, the data interval for the first period can be long enough to sample sufficient data for predictive capabilities; the data interval can be a configurable setting and / or selected by the machine learning models (e.g., based on testing or optimization of the machine learning models). The first period can be a fixed amount of time or a variable amount of time before the event.

[0127] The method 1100 can include applying, as input to a plurality of machine learning models, the one or more features (1110). The plurality of machine learning models can include, for example, any of various machine learning models 125 described with reference to FIG. 1. Applying the one or more features as input to the plurality of machine learning models can cause the plurality of machine learning models to each generate a candidate prediction of the event of the subject. The candidate prediction can include at least one of an indication that the event is expected to occur, an ordinal value representative of a likelihood of the event, or a confidence score associated with the indication (e.g., a binary indication of whether the event is expected to occur; a percentage indication of a likelihood of the event being expected to occur; the binary indication can be determined responsive to comparing the confidence score with a threshold).

[0128] The event can be predicted to occur during a second period. The second period can be a future period relative to when the plurality of machine learning models are caused to generate the candidate predictions of the event (e.g., the plurality of machine learning models can generate the candidate predictions during the first period, where the candidate predictions are for whether the event is likely to occur during the second period).The second period can be a period that is during the procedure performed on the subject. The second period can be subsequent to the first period. The first period can be before the second period. The second period can be a fixed amount of time or a variable amount of time before the event. The second period can be a fixed amount of time or a variable amount of time after the first period. The event can occur during or after the second period.

[0129] The second period can have a duration that represents a forecast window, such as to represent a period of time in which the event is predicted to occur. The second period can be in a range of 1 minute to 20 minutes. For example, the second period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The forecast window (e.g., forecast interval) can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. The second period of time can include an amount of time that the medical team has after receiving a warning of a potential adverse event until the adverse event occurs. A longer second period can give the medical team more time to respond to a potential adverse event. For example, the anesthesiologist can intervene during the second period of time, given an early warning based on the data gathered during the first period. The second period can include the forecast window 210. The length of the second period can be selected randomly during training of the plurality of machine learning models.

[0130] The method 1100 can include applying, as input to the plurality of machine learning models, one or more second features of the subject. The one or more second features can include data from the subject’s electronic medical record (EMR). For example, in addition to the sensor data of the one or more first features, the one or more secondfeatures can include record data, such as personal information, historical information, and / or demographic information. The one or more second features can include available medical information. The one or more second features can include at least one of age, sex, body mass index, height, or weight. Age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject. For example, age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure (e.g., as described above with reference to FIG. 7C). The one or more first features and the one or more second features can include one or more features with greater predictive capability (compared to other features) for a given event. For example, the one or more features can be a subset of the one or more first features and the one or more second features. These one or more features can be obtained from an ablation study (e.g., as described above with reference to FIG. 7C). The one or more features can include cardiac data regarding the subject and pulmonary data regarding the subject.

[0131] The plurality of machine learning models can be configured using training data. The training data can include example data of the one or more features of a plurality of example subjects during a first example period. The training data can include example data of a presence of the event for the plurality of example subjects during a second example period. The second example period can be subsequent to the first example period. The first example period can be before the second example period. The plurality of machine learning models can include a composite model configured using training data comprising data labeled with the procedure. The plurality of machine learning models can be selected as a procedure-specific model using training data that includes data corresponding to the procedure.

[0132] The plurality of machine learning models can be derived from the same underlying architecture. For example, the plurality of machine learning models can be derived from the same underlying architecture trained in separate training sets (e.g., folds). Each of the plurality of machine learning models can be from a different fold. The predictions from the plurality of machine learning models can be pooled. For example, thepredictions from the plurality of machine learning models can be pooled for voting. This can be in contrast to ensemble methods, which can assemble different models and pick one.

[0133] The method 1100 can include determining an output prediction (1115). The output prediction of the event can be based on each candidate prediction generated by each machine learning model of the plurality of machine learning models. The candidate prediction can include at least one of an indication that the event is expected to occur, an ordinal value representative of a likelihood of the event, or a confidence score associated with the indication (e.g., a binary indication of whether the event is expected to occur; a percentage indication of a likelihood of the event being expected to occur; the binary indication can be determined responsive to comparing the confidence score with a threshold). Determining the output prediction can include performing a voting operation using each candidate prediction. Due to the manner in which the machine learning models are configured and / or operated, the output prediction can have an accuracy of greater than 80%. For example, the accuracy can be greater than 80%, greater than 85%, greater than 90%, greater than 91%, greater than 92%, greater than 93%, greater than 94%, or greater than 95% (e.g., as described above with reference to FIGS. 6A and 6B).

[0134] The method 1100 can include outputting an indication of the output prediction (1120). For example, the output prediction can be displayed on an interface (e.g., display interface). The display interface can include, for example, any of various display interfaces 130 described with reference to FIG. 1. The output prediction can trigger a warning of the event. The output prediction can be a categorical prediction. For example, the output prediction can be positive or negative (e.g., yes or no, green light or red light). The output prediction can be a numerical (e.g., ordinal) scale related to the likelihood of the event. For example, the output prediction can be a number (e.g., a number between 1 and 11) representing the likelihood of the event. The likelihood of the event can include the likelihood the event will occur in a specific time interval (e.g., 10 minutes). Based on the output prediction, the medical team can decide how to respond to the potential adverse event. The method 1100 can include outputting a number corresponding to the likelihood of the event.

[0135] The method 1100 can include periodically applying the input to the plurality of machine learning models. For example, periodically applying the input to the plurality of machine learning models can be responsive to sampling the one or more features from one or more sensors. The periodic application of input can improve the accuracy and / orpredictive capability of the plurality of machine learning models. Data from the one or more sensors regarding the one or more features can be collected by the data processing system. The data processing system can collect data regarding the one or more features by sampling the one or more sensors at regular or irregular intervals. The data can be stored in or retrieved from the subject parameter database. Periodically applying the input to the plurality of machine learning models can include using data from a sliding window.

[0136] The method 1100 can include training each of the plurality of machine learning models with a subset of the training data. The training data can include a presence of the event (e.g., positive event) during the second period (e.g., as described above with reference to FIGS. 2A and 2B). The training data can include an absence of the event (e.g., negative event) during the second period (e.g., as described above with reference to FIG. 3B). The training data can have equal or roughly equal proportions (e.g., class balance) of positive events and negative events. The training data can have more positive events than negative events. The training data can have more negative events than positive events. The method 1100 can include detecting the event as a second event. The input can include an indication of a prior event and a dosage of a treatment administered subsequent to the prior event.

[0137] The method 1100 can include determining that a condition is present responsive to at least a subset of the plurality of machine learning models indicating that the condition is present. The method 1100 can include adjusting a threshold (e.g., voting threshold) for the subset of the plurality of machine learning models responsive to an input. The threshold can be changed so that the sensitivity of the plurality of machine learning models is changed. Increasing the sensitivity (e.g., decreasing the specificity) may be useful for cases in which the medical team would like extra vigilance, and they are willing to tolerate an increased number of false alarms. Conversely, the medical team can decrease the sensitivity (e.g., increase the specificity) by increasing the threshold. By changing the threshold, the sensitivity may be adjusted (e.g., dialed, changed) to increase or decrease the sensitivity of the method 1100. The method 1100 can have adjustable sensitivity. The output prediction can be determined by a subset of the plurality of machine learning models being greater than the threshold. The threshold can be in a range of 10% to 100% of the plurality of machine learning models. For example, the threshold can be in a range of 10% to 20%, 10% to 30%, 10% to 40%, 10% to 50%, 10% to 60%, 10% to 70%, 10% to 80%, 10% to 90%, 10% to 100%, 20% to 30%, 20% to 40%, 20% to 50%, 20% to 60%, 20% to70%, 20% to 80%, 20% to 90%, 20% to 100%, 30% to 40%, 30% to 50%, 30% to 60%, 30% to 70%, 30% to 80%, 30% to 90%, 30% to 100%, 40% to 50%, 40% to 60%, 40% to 70%, 40% to 80%, 40% to 90%, 40% to 100%, 50% to 60%, 50% to 70%, 50% to 80%, 50% to 90%, 50% to 100%, 60% to 70%, 60% to 80%, 60% to 90%, 60% to 100%, 70% to 80%, 70% to 90%, 70% to 100%, 80% to 90%, 80% to 100%, or 90% to 100%. A majority vote of the plurality of machine learning models can include a threshold of 50%. Adjustable sensitivity may be useful for cases in which the medical team only desires alerts when there is a strong indication and / or to reduce the percentage of false alarms.

[0138] FIG. 12 illustrates a method 1200 to train data for generating a prediction of a physiological event. The method 1200 can be implemented by any of various systems and devices described herein, including but not limited to the system 100 and / or the procedure monitor 120 described with reference to FIG. 1. The method 1200 can be implemented in any of various modalities, including but not limited to real-time, intraoperative techniques in which one or more aspects of the method 1200 are executed responsive to real-time sensor data detected regarding a subject on which a procedure is being performed, as well as any of various asynchronous implementations (e.g., for testing or validation, or updating of machine learning models of the procedure monitor 120). The method 1200 can train a plurality of machine learning models configured to perform MMV.

[0139] The method 1200 can use signals regarding the subject, such as at least one of record data regarding the subject or sensor data from one or more sensors that monitor the subject during the procedure. For example, the method 1200 can obtain (e.g., use, draw from) data from one or more subjects (patients, users) who have undergone a procedure such as a surgical procedure. The subject can be at risk of any of various intraoperative events, such as intraoperative hypotension, which can include an event associated with severe postoperative complications to renal, cardiovascular, and / or neurovascular systems.

[0140] The method 1200 can assist the medical team (e.g., surgeon, anesthesiologist, nurse, staff) by providing a model by which timely predictions (e.g., alarms, alerts, warnings) of an adverse event (e.g., imminent adverse event) such as intraoperative hypotension can be made. Based on the existence and / or timing of the warning, this can allow the medical team to decide how to respond to the potential adverse event. The method 1200 can be performed, for example, prior to, contemporaneous with, or after the method 1100. For example, the method 1100 can use the plurality of machine learning models trained in method 1200. The method 1200 can train the plurality ofmachine learning models while the method 1100 is implementing the plurality of machine learning models.

[0141] In brief summary, the method 1200 can include identifying training data including one or more features of a subject (1205). The method 1200 can include updating a plurality of machine learning models by applying the training data as input to the plurality of machine learning models (1210).

[0142] Referring to FIG. 12 in further detail, the method 1200 can include identifying training data including one or more features (e.g., first features) of a subject (1205). The training data can include one or more features of a plurality of example subjects. The one or more features of the subject can include at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate. The one or more features can include signs (e.g., vital signs, multivariate vital signs). The one or more features can include physiological signals. The one or more features can include at least one of body temperature, pulse rate, respiration rate, or blood pressure.

[0143] Data related to the one or more features can be received and / or monitored by a data processing system. The data processing system can include, for example, any of various data processing systems 105 described with reference to FIG. 1. The data can be obtained from or stored in, for example, any of various subject parameter databases 140 described with reference to FIG. 1. The data can be obtained from the one or more sensors. For example, a heart rate monitor coupled to the subject can sample the subject’s heart rate at a specified time interval. For an intraoperative early warning system to predict changes in physiological status during surgery, the one or more features can be indicative of an event, such as an intraoperative hypotension event. By identifying training data that includes the plurality of features, the method 1200 can update the plurality of machine learning models that can give an indication of an imminent adverse event and permit intervention before the adverse event occurs. The event can include a hypotensive event. For example, the event can include an intraoperative hypotension.

[0144] The training data can include one or more features of a plurality of example subjects during a first period. The first period can include a period of time during which the one or more features is monitored. The first period can have a duration that represents a datainterval. The first period can include a period of time during which data relating to the one or more features is collected from the subject. The first period can be in a range of 1 minute to 20 minutes. For example, the first period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The data interval can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. Similar to the sampling rate of retrieving data from the sensors, the data interval for the first period can be long enough to sample sufficient data for predictive capabilities; the data interval can be a configurable setting and / or selected by the machine learning models (e.g., based on testing or optimization of the machine learning models). The first period can be a fixed amount of time or a variable amount of time before the event. The first period can include the data interval 205. The length of the first period can be selected randomly during training of the plurality of machine learning models.

[0145] The training data can include one or more features of the plurality of example subjects during a second period. The second period can be a period that is during the procedure performed on the subject. The second period can be subsequent to the first period. The first period can be before the second period. The second period can be a fixed amount of time or a variable amount of time before the event. The second period can be a fixed amount of time or a variable amount of time after the first period. The event can occur during or after the second period.

[0146] The second period can have a duration that represents a forecast window, such as to represent a period of time in which the event is predicted to occur. The second period can be in a range of 1 minute to 20 minutes. For example, the second period can bein a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The forecast window (e.g., forecast interval) can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. The second period of time can include an amount of time that the medical team has after receiving a warning of a potential adverse event until the adverse event occurs. A longer second period can give the medical team more time to respond to a potential adverse event. For example, the anesthesiologist can intervene during the second period of time, given an early warning based on the data gathered during the first period. The second period can include the forecast window 210. The length of the second period can be selected randomly during training of the plurality of machine learning models.

[0147] The training data can include one or more second features of the plurality of example subjects. The one or more second features can include data from the subject’s electronic medical record (EMR). For example, in addition to the sensor data of the one or more first features, the one or more second features can include record data, such as personal information, historical information, and / or demographic information. The one or more second features can include available medical information. The one or more second features can include available medical information. The one or more second features can include at least one of age, sex, body mass index, height, or weight. The training data can include a presence of the event (e.g., positive event) during the second period (e.g., as described above with reference to FIGS. 2A and 2B). The training data can include an absence of the event (e.g., negative event) during the second period (e.g., as described above with reference to FIG. 3B). The training data can have equal or roughly equalproportions (e.g., class balance) of positive events and negative events. The training data can have more positive events than negative events. The training data can have more negative events than positive events.

[0148] Age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject. For example, age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure (e.g., as described above with reference to FIG. 7C). The one or more first features and the one or more second features can include one or more features with greater predictive capability (compared to other features) for a given event. For example, the one or more features can be a subset of the one or more first features and the one or more second features. These one or more features can be obtained from an ablation study (e.g., as described above with reference to FIG. 7C). The one or more features can include cardiac data regarding the subject and pulmonary data regarding the subject.

[0149] The method 1200 can include updating a plurality of machine learning models (1210). The plurality of machine learning models can include, for example, any of various machine learning models 125 described with reference to FIG. 1. For example, updating the plurality of machine learning models can include updating the plurality of machine learning models by applying the training data as input to the plurality of machine learning models.

[0150] The method 1200 can include performing a voting operation using the plurality of machine learning models. The plurality of machine learning models can be derived from the same underlying architecture. The predictions from the plurality of machine learning models can be pooled. For example, the predictions from the plurality of machine learning models can be pooled for voting.

[0151] The method 1200 can include sampling the one or more features from one or more sensors. For example, data from the one or more sensors regarding the one or more features can be collected by the data processing system. The data processing system can collect data regarding the one or more features by sampling the one or more sensors atregular or irregular intervals. The data can be stored in or retrieved from the subject parameter database.

[0152] The method 1200 can include determining that a condition is present responsive to at least a subset of the plurality of machine learning models indicating that the condition is present. The method 1200 can include adjusting a threshold for the subset of the plurality of machine learning models responsive to an input. The threshold can be changed so that the sensitivity of the plurality of machine learning models is changed. Increasing the sensitivity (e.g., decreasing the specificity) may be useful for cases in which the medical team would like extra vigilance, and they are willing to tolerate an increased number of false alarms. Conversely, the medical team can decrease the sensitivity (e.g., increase the specificity) by increasing the threshold. By changing the threshold, the sensitivity may be adjusted (e.g., dialed, changed) to increase or decrease the sensitivity of the method 1200. The method 1200 can have adjustable sensitivity. The output prediction can be determined by a subset of the plurality of machine learning models being greater than the threshold. The threshold can be in a range of 10% to 100% of the plurality of machine learning models. For example, the threshold can be in a range of 10% to 20%, 10% to 30%, 10% to 40%, 10% to 50%, 10% to 60%, 10% to 70%, 10% to 80%, 10% to 90%, 10% to 100%, 20% to 30%, 20% to 40%, 20% to 50%, 20% to 60%, 20% to 70%, 20% to 80%, 20% to 90%, 20% to 100%, 30% to 40%, 30% to 50%, 30% to 60%, 30% to 70%, 30% to 80%, 30% to 90%, 30% to 100%, 40% to 50%, 40% to 60%, 40% to 70%, 40% to 80%, 40% to 90%, 40% to 100%, 50% to 60%, 50% to 70%, 50% to 80%, 50% to 90%, 50% to 100%, 60% to 70%, 60% to 80%, 60% to 90%, 60% to 100%, 70% to 80%, 70% to 90%, 70% to 100%, 80% to 90%, 80% to 100%, or 90% to 100%. A majority vote of the plurality of machine learning models can include a threshold of 50%. Adjustable sensitivity may be useful for cases in which the medical team only desires alerts when there is a strong indication and / or to reduce the percentage of false alarms.SYSTEMS AND METHODS OF MULTIVARIATE PREDICTION OF PHYSIOLOGICAL STATES OF A SUBJECT

[0153] Systems and methods in accordance with the present disclosure can incorporate multiple variables and / or features (e.g., multivariate) to allow for more accurate prediction of physiological states of a subject, including real-time prediction during a procedure performed on the subject. In the multivariate approach, a plurality of features can be used in one or more models. At various instances during the procedure, such as periodicand / or user-trigger instances, the one or more models can compute a prediction of a state and / or event for the subject. The one or more models can incorporate features that are monitored in real-time and / or features that are obtained from the subject’s medical record, history, and / or personal information. Certain features can be stronger in predicting the state and / or event of the subject. The multivariate approach can allow for improved predictive capabilities of the one or more models and identification of the features that are more strongly correlated with prediction of a given event, allowing for improved accuracy of predictions at runtime.

[0154] FIG. 13 illustrates a method 1300 to generate a prediction of a physiological event. The method 1300 can be implemented by any of various systems and devices described herein, including but not limited to the system 100 and / or the procedure monitor 120 described with reference to FIG. 1. The method 1300 can be implemented in any of various modalities, including but not limited to real-time, intraoperative techniques in which one or more aspects of the method 1300 are executed responsive to real-time sensor data detected regarding a subject on which a procedure is being performed, as well as any of various asynchronous implementations (e.g., for testing or validation, or updating of machine learning models of the procedure monitor 120). The method 1300 can be implemented using one or more machine learning models configured to perform multivariate prediction.

[0155] In brief summary, the method 1300 can include monitoring a plurality of features of a subject (1305). The method 1300 can include applying, as input to one or more machine learning models, the plurality of features to cause the one or more machine learning models to generate a prediction of an event (1310). The method 1300 can include outputting an indication of the prediction (1315).

[0156] The method 1300 can be performed, for example, in conjunction with signals regarding the subject, such as at least one of record data regarding the subject or sensor data from one or more sensors that monitor the subject during the procedure. For example, the subject (e.g., patient, user) can undergo a procedure such as a surgical procedure. The subject can be at risk of any of various intraoperative events, such as intraoperative hypotension, which can include an event associated with severe postoperative complications to renal, cardiovascular, and / or neurovascular systems. The method 1300 can assist the medical team (e.g., surgeon, anesthesiologist, nurse, staff) by providing timely predictions (e.g., alarms, alerts, warnings) of an adverse event (e.g., imminent adverse event) such asintraoperative hypotension. Based on the existence and / or timing of the warning, this can allow the medical team to decide how to respond to the potential adverse event.

[0157] Referring to FIG. 13 in further detail, the method 1300 can include monitoring a plurality of features (e.g., first features) of a subject (1305). The plurality of features can be monitored by, for example, any of various procedure monitors 120 described with reference to FIG. 1. The plurality of features can include features monitored in real-time. Data related to the one or more features can be received by the procedure monitors as a stream of data (e.g., data stream). The plurality of features can include physiological features. The plurality of features can include at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate. The plurality of features can include signs (e.g., vital signs, multivariate vital signs). The plurality of features can include physiological signals. The plurality of features can include at least one of body temperature, pulse rate, respiration rate, or blood pressure.

[0158] The plurality of features can be monitored using one or more sensors. The one or more sensors can include, for example, any of various one or more sensors 115 described with reference to FIG. 1. Data from the plurality of features can be received and / or monitored by a data processing system. The data processing system can include, for example, any of various data processing systems 105 described with reference to FIG. 1. The data can be obtained from or stored in, for example, any of various subject parameter databases 140 described with reference to FIG. 1. The data from plurality of features can be sampled at various intervals. For example, the plurality of features can be sample at one second intervals, 30 second intervals, 1 -minute intervals, 2-minute intervals, or 5 -minute intervals. The intervals can be frequent enough to sample enough data for predictive capabilities. The data can be obtained from the one or more sensors. For example, a heart rate monitor coupled to the subject can sample the subject’s heart rate at a specified time interval. For an intraoperative early warning system to predict changes in physiological status during surgery, the plurality of features can be indicative of an event, such as an intraoperative hypotension event. By monitoring the plurality of features, the method 1300 can give an indication of an imminent adverse event and permit intervention before the adverse event occurs.

[0159] The plurality of features of the subject can be monitored during one or more periods relating to the procedure, including during a first period (e.g., a monitoring period). The first period can be a period during a procedure performed on the subject. The first period can occur prior to the event to be predicted (e.g., the event may not have yet occurred during the first period in which the features are monitored, such as to occur or be predicted to occur during a second period subsequent to the first period as described further below). The procedure can include a surgery (e.g., bariatric surgery, breast surgery, colon and rectal surgery, endocrine surgery, general surgery, gynecological surgery, hand surgery, head and neck surgery, hernia surgery, minimally invasive surgery, neurosurgery, orthopedic surgery, ophthalmological surgery, outpatient surgery, pediatric surgery, plastic and reconstructive surgery, robotic surgery, thoracic surgery, trauma surgery, urologic surgery, or vascular surgery). The event can include a hypotensive event. For example, the event can include an intraoperative hypotension.

[0160] The first period can include a period of time during which the plurality of features is monitored. The first period can have a duration that represents a data interval. The first period can include a period of time during which data relating to the plurality of features is collected from the subject. The first period can be in a range of 1 minute to 20 minutes. For example, the first period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The data interval can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. Similar to the sampling rate of retrieving data from the sensors, the data interval for the first period can be long enough to sample sufficient data for predictive capabilities; the data interval can be aconfigurable setting and / or selected by the one or more machine learning models (e.g., based on testing or optimization of the one or more machine learning models). The first period can be a fixed amount of time or a variable amount of time before the event. The plurality of features of the subject can include real-time signals retrieved from the one or more sensors during the first period. The first period can include the data interval 205. The length of the first period can be selected randomly during training of the one or more machine learning models.

[0161] The method 1300 can include applying, as input to one or more machine learning models, the plurality of features (1310). The one or more machine learning models can include, for example, any of various machine learning models 125 described with reference to FIG. 1. Applying the plurality of features as input to the one or more machine learning models can cause the one or more machine learning models to generate a prediction of the event of the subject. The prediction can include at least one of an indication that the event is expected to occur or a confidence score associated with the indication (e.g., a binary indication of whether the event is expected to occur; a percentage indication of a likelihood of the event being expected to occur; the binary indication can be determined responsive to comparing the confidence score with a threshold).

[0162] The event can be predicted to occur during a second period. The second period can be a future period relative to when the one or more machine learning models are caused to generate the prediction of the event (e.g., the one or more machine learning models can generate the prediction during the first period, where the prediction are for whether the event is likely to occur during the second period). The second period can be a period that is during the procedure performed on the subject. The second period can be subsequent to the first period. The first period can be before the second period. The second period can be a fixed amount of time or a variable amount of time before the event. The second period can be a fixed amount of time or a variable amount of time after the first period. The event can occur during or after the second period.

[0163] The second period can have a duration that represents a forecast window, such as to represent a period of time in which the event is predicted to occur. The second period can be in a range of 1 minute to 20 minutes. For example, the second period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The forecast window (e.g., forecast interval) can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. The second period of time can include an amount of time that the medical team has after receiving a warning of a potential adverse event until the adverse event occurs. A longer second period can give the medical team more time to respond to a potential adverse event. For example, the anesthesiologist can intervene during the second period of time, given an early warning based on the data gathered during the first period. The second period can include the forecast window 210. The length of the second period can be selected randomly during training of the one or more machine learning models.

[0164] The method 1300 can include applying, as input to the one or more machine learning models, one or more second features of the subject. The one or more second features can include data from the subject’s electronic medical record (EMR). For example, in addition to the sensor data of the one or more first features, the one or more second features can include record data, such as personal information, historical information, and / or demographic information. The one or more second features can include available medical information. The one or more second features can include at least one of age, sex, body mass index, height, or weight. Age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject. For example, age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure (e.g., as described above with reference to FIG. 7C). The one or more first features and the one or more second features can includeone or more features with greater predictive capability (compared to other features) for a given event. For example, the one or more features can be a subset of the one or more first features and the one or more second features. These one or more features can be obtained from an ablation study (e.g., as described above with reference to FIG. 7C). The one or more features can include cardiac data regarding the subject and pulmonary data regarding the subject.

[0165] The one or more machine learning models can be configured using training data. The training data can include example data of the plurality of features of a plurality of example subjects during a first example period. The training data can include example data of a presence of the event for the plurality of example subjects during a second example period. The second example period can be subsequent to the first example period. The first example period can be before the second example period. The one or more machine learning models can include a composite model configured using training data comprising data labeled with the procedure. The one or more machine learning models can be selected as a procedure-specific model using training data that includes data corresponding to the procedure.

[0166] The method 1300 can include determining an output prediction. The output prediction of the event can be based on a candidate prediction generated by each machine learning model of the one or more machine learning models. The candidate prediction can include at least one of an indication that the event is expected to occur or a confidence score associated with the indication (e.g., a binary indication of whether the event is expected to occur; a percentage indication of a likelihood of the event being expected to occur; the binary indication can be determined responsive to comparing the confidence score with a threshold). Determining the output prediction can include performing a voting operation using each candidate prediction. The output prediction can have an accuracy of greater than 80%. For example, the accuracy can be greater than 80%, greater than 85%, greater than 90%, greater than 91%, greater than 92%, greater than 93%, greater than 94%, or greater than 95% (e.g., as described above with reference to FIGS. 6A and 6B).

[0167] The method 1300 can include outputting an indication of the prediction (1315). For example, the output prediction can be displayed on an interface (e.g., display interface). The display interface can include, for example, any of various display interfaces 130 described with reference to FIG. 1. The output prediction can trigger a warning of the event. The output prediction can be a categorical prediction. For example, the outputprediction can be positive or negative (e.g., yes or no, green light or red light). The output prediction can be a numerical (e.g., ordinal) scale related to the likelihood of the event. For example, the output prediction can be a number (e.g., a number between 1 and 11) representing the likelihood of the event. The likelihood of the event can include the likelihood the event will occur in a specific time interval (e.g., 10 minutes). Based on the output prediction, the medical team can decide how to respond to the potential adverse event. The method 1300 can include outputting a number corresponding to the likelihood of the event.

[0168] The method 1300 can include periodically applying the input to the one or more machine learning models. For example, periodically applying the input to the one or more machine learning models can be responsive to sampling the plurality of features from one or more sensors. The periodic application of input can improve the accuracy and / or predictive capability of the one or more machine learning models. Data from the one or more sensors regarding the plurality of features can be collected by the data processing system. The data processing system can collect data regarding the plurality of features by sampling the one or more sensors at regular or irregular intervals. The data can be stored in or retrieved from the subject parameter database. Periodically applying the input to the one or more machine learning models can include using data from a sliding window.

[0169] The method 1300 can include training each of the one or more machine learning models with a subset of the training data. The training data can include a presence of the event (e.g., positive event) during the second period (e.g., as described above with reference to FIGS. 2A and 2B). The training data can include an absence of the event (e.g., negative event) during the second period (e.g., as described above with reference to FIG. 3B). The training data can have equal or roughly equal proportions (e.g., class balance) of positive events and negative events. The training data can have more positive events than negative events. The training data can have more negative events than positive events.

[0170] The method 1300 can include determining that a condition is present responsive to at least a subset of the one or more machine learning models indicating that the condition is present. The method 1300 can include adjusting a threshold for the subset of the one or more machine learning models responsive to an input. The threshold can be changed so that the sensitivity of the one or more machine learning models is changed.Increasing the sensitivity (e.g., decreasing the specificity) may be useful for cases in which the medical team would like extra vigilance, and they are willing to tolerate an increasednumber of false alarms. Conversely, the medical team can decrease the sensitivity (e.g., increase the specificity) by increasing the threshold. By changing the threshold, the sensitivity may be adjusted (e.g., dialed, changed) to increase or decrease the sensitivity of the method 1300. The method 1300 can have adjustable sensitivity. The output prediction can be determined by a subset of the plurality of machine learning models being greater than the threshold. The threshold can be in a range of 10% to 100% of the plurality of machine learning models. For example, the threshold can be in a range of 10% to 20%, 10% to 30%, 10% to 40%, 10% to 50%, 10% to 60%, 10% to 70%, 10% to 80%, 10% to 90%, 10% to 100%, 20% to 30%, 20% to 40%, 20% to 50%, 20% to 60%, 20% to 70%, 20% to 80%, 20% to 90%, 20% to 100%, 30% to 40%, 30% to 50%, 30% to 60%, 30% to 70%, 30% to 80%, 30% to 90%, 30% to 100%, 40% to 50%, 40% to 60%, 40% to 70%, 40% to 80%, 40% to 90%, 40% to 100%, 50% to 60%, 50% to 70%, 50% to 80%, 50% to 90%, 50% to 100%, 60% to 70%, 60% to 80%, 60% to 90%, 60% to 100%, 70% to 80%, 70% to 90%, 70% to 100%, 80% to 90%, 80% to 100%, or 90% to 100%. A majority vote of the plurality of machine learning models can include a threshold of 50%. Adjustable sensitivity may be useful for cases in which the medical team only desires alerts when there is a strong indication and / or to reduce the percentage of false alarms.

[0171] FIG. 14 illustrates a method 1400 to train data for generating a prediction of a physiological event. The method 1400 can be implemented by any of various systems and devices described herein, including but not limited to the system 100 and / or the procedure monitor 120 described with reference to FIG. 1. The method 1400 can be implemented in any of various modalities, including but not limited to real-time, intraoperative techniques in which one or more aspects of the method 1400 are executed responsive to real-time sensor data detected regarding a subject on which a procedure is being performed, as well as any of various asynchronous implementations (e.g., for testing or validation, or updating of machine learning models of the procedure monitor 120). The method 1400 can train one or more machine learning models configured to multivariate prediction.

[0172] The method 1400 can use signals regarding the subject, such as at least one of record data regarding the subject or sensor data from one or more sensors that monitor the subject during the procedure. For example, the method 1400 can obtain (e.g., use, draw from) data from one or more subjects (patients, users) who have undergone a procedure such as a surgical procedure. The subject can be at risk of any of various intraoperativeevents, such as intraoperative hypotension, which can include an event associated with severe postoperative complications to renal, cardiovascular, and / or neurovascular systems.

[0173] The method 1400 can assist the medical team (e.g., surgeon, anesthesiologist, nurse, staff) by providing a model by which timely predictions (e.g., alarms, alerts, warnings) of an adverse event (e.g., imminent adverse event) such as intraoperative hypotension can be made. Based on the existence and / or timing of the warning, this can allow the medical team to decide how to respond to the potential adverse event. The method 1400 can be performed, for example, prior to, contemporaneous with, or after the method 1300. For example, the method 1300 can use the one or more machine learning models trained in method 1400. The method 1400 can train the one or more machine learning models while the method 1300 is implementing the one or more machine learning models.

[0174] In brief summary, the method 1400 can include identifying training data including a plurality of features of a subject (1405). The method 1400 can include updating one or more machine learning models by applying the training data as input to the one or more machine learning models (1410).

[0175] Referring to FIG. 14 in further detail, the method 1400 can include identifying training data including a plurality of features (e.g., first features) of a subject (1405). The training data can include plurality of features of a plurality of example subjects. The plurality of features of the subject can include at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate. The plurality of features can include signs (e.g., vital signs, multivariate vital signs). The plurality of features can include physiological signals. The plurality of features can include at least one of body temperature, pulse rate, respiration rate, or blood pressure.

[0176] Data related to the plurality of features can be received and / or monitored by a data processing system. The data processing system can include, for example, any of various data processing systems 105 described with reference to FIG. 1. The data can be obtained from the one or more sensors. For example, a heart rate monitor coupled to the subject can sample the subject’s heart rate at a specified time interval. For an intraoperative early warning system to predict changes in physiological status during surgery, the pluralityof features can be indicative of an event, such as an intraoperative hypotension event. By identifying training data that includes the plurality of features, the method 1400 can update the one or more machine learning models that can give an indication of an imminent adverse event and permit intervention before the adverse event occurs. The event can include a hypotensive event. For example, the event can include an intraoperative hypotension.

[0177] The training data can include plurality of features of a plurality of example subjects during a first period. The first period can include a period of time during which the plurality of features is monitored. The first period can have a duration that represents a data interval. The first period can include a period of time during which data relating to the plurality of features is collected from the subject. The first period can be in a range of 1 minute to 20 minutes. For example, the first period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The data interval can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes,14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. Similar to the sampling rate of retrieving data from the sensors, the data interval for the first period can be long enough to sample sufficient data for predictive capabilities; the data interval can be a configurable setting and / or selected by the machine learning models (e.g., based on testing or optimization of the machine learning models). The first period can be a fixed amount of time or a variable amount of time before the event. The first period can include the data interval 205. The length of the first period can be selected randomly during training of the one or more machine learning models.

[0178] The training data can include plurality of features of a presence of the event for the plurality of example subjects during a second period. The second period can be a period that is during the procedure performed on the subject. The second period can be subsequent to the first period. The first period can be before the second period. The second period can be a fixed amount of time or a variable amount of time before the event. The second period can be a fixed amount of time or a variable amount of time after the first period. The event can occur during or after the second period.

[0179] The second period can have a duration that represents a forecast window, such as to represent a period of time in which the event is predicted to occur. The second period can be in a range of 1 minute to 20 minutes. For example, the second period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The forecast window (e.g., forecast interval) can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. The second period of time can include an amount of time that the medical team has after receiving a warning of a potential adverse event until the adverse event occurs. A longer second period can give the medical team more time to respond to a potential adverse event. For example, the anesthesiologist can intervene during the second period of time, given an early warning based on the data gathered during the first period. The second period can include the forecast window 210. The length of the second period can be selected randomly during training of the one or more machine learning models.

[0180] The training data can include one or more second features of the plurality of example subjects. The one or more second features can include data from the subject’selectronic medical record (EMR). For example, in addition to the sensor data of the one or more first features, the one or more second features can include record data, such as personal information or historical information. The one or more second features can include available medical information. The one or more second features can include available medical information. The one or more second features can include at least one of age, sex, body mass index, height, or weight. The training data can include a presence of the event (e.g., positive event) during the second period (e.g., as described above with reference to FIGS. 2A and 2B). The training data can include an absence of the event (e.g., negative event) during the second period (e.g., as described above with reference to FIG. 3B). The training data can have equal or roughly equal proportions (e.g., class balance) of positive events and negative events. The training data can have more positive events than negative events. The training data can have more negative events than positive events.

[0181] Age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject. For example, age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure (e.g., as described above with reference to FIG. 7C). The one or more first features and the one or more second features can include plurality of features with greater predictive capability (compared to other features) for a given event. For example, the plurality of features can be a subset of the one or more first features and the one or more second features. These plurality of features can be obtained from an ablation study (e.g., as described above with reference to FIG. 7C). The one or more features can include cardiac data regarding the subject and pulmonary data regarding the subject.

[0182] The method 1400 can include updating one or more machine learning models (1410). The one or more machine learning models can include, for example, any of various machine learning models 125 described with reference to FIG. 1. For example, updating the one or more machine learning models can include updating the one or more machine learning models by applying the training data as input to the one or more machine learning models.

[0183] The method 1400 can include performing a voting operation using the one or more machine learning models. The one or more machine learning models can be derived from the same underlying architecture. The predictions from the one or more machine learning models can be pooled. For example, the predictions from the one or more machine learning models can be pooled for voting.

[0184] The method 1400 can include sampling the plurality of features from one or more sensors. For example, data from the one or more sensors regarding the plurality of features can be collected by the data processing system. The data processing system can collect data regarding the plurality of features by sampling the one or more sensors at regular or irregular intervals. The data can be stored in or retrieved from the subject parameter database.

[0185] The method 1400 can include determining that a condition is present responsive to at least a subset of the one or more machine learning models indicating that the condition is present. The method 1400 can include adjusting a threshold for the subset of the one or more machine learning models responsive to an input. The threshold can be changed so that the sensitivity of the one or more machine learning models is changed. Increasing the sensitivity (e.g., decreasing the specificity) may be useful for cases in which the medical team would like extra vigilance, and they are willing to tolerate an increased number of false alarms. Conversely, the medical team can decrease the sensitivity (e.g., increase the specificity) by increasing the threshold. By changing the threshold, the sensitivity may be adjusted (e.g., dialed, changed) to increase or decrease the sensitivity of the method 1400. The method 1400 can have adjustable sensitivity. The output prediction can be determined by a subset of the plurality of machine learning models being greater than the threshold. The threshold can be in a range of 10% to 100% of the plurality of machine learning models. For example, the threshold can be in a range of 10% to 20%, 10% to 30%, 10% to 40%, 10% to 50%, 10% to 60%, 10% to 70%, 10% to 80%, 10% to 90%, 10% to 100%, 20% to 30%, 20% to 40%, 20% to 50%, 20% to 60%, 20% to 70%, 20% to 80%, 20% to 90%, 20% to 100%, 30% to 40%, 30% to 50%, 30% to 60%, 30% to 70%, 30% to 80%, 30% to 90%, 30% to 100%, 40% to 50%, 40% to 60%, 40% to 70%, 40% to 80%, 40% to 90%, 40% to 100%, 50% to 60%, 50% to 70%, 50% to 80%, 50% to 90%, 50% to 100%, 60% to 70%, 60% to 80%, 60% to 90%, 60% to 100%, 70% to 80%, 70% to 90%, 70% to 100%, 80% to 90%, 80% to 100%, or 90% to 100%. A majority vote of the plurality of machine learning models can include a threshold of 50%. Adjustable sensitivitymay be useful for cases in which the medical team only desires alerts when there is a strong indication and / or to reduce the percentage of false alarms.SYSTEMS AND METHODS OF MULTIPLE FORECASTERS OF PHYSIOLOGICAL STATES OF A SUBJECT

[0186] Systems and methods in accordance with the present disclosure can incorporate multiple forecasters to allow for more accurate prediction of physiological states of a subject, including real-time prediction during a procedure performed on the subject. In the multiple forecasters approach, one or more features can be used in one or more models. At various instances during the procedure, such as periodic and / or usertrigger instances, the one or more models can compute a prediction of a state and / or event for the subject. The one or more models can incorporate features that are monitored in realtime and / or features that are obtained from the subject’s medical record, history, and / or personal information. The one or more models can make multiple predictions during a plurality of forecast periods. The multiple forecasters approach can allow for predictions at multiple times, thus providing for a variable forecast range (e.g., receptive field). For example, the multiple forecasters (e.g., predictors) can include a short-range forecaster, medium-range forecaster, and / or a long-range forecaster. The multiple forecasters can give varying warnings at various times (e.g., multiple forecast ranges), with the short-range forecaster giving a warning closer to the predicted event compared to the medium-range forecaster and long-range forecaster. The medium-range forecaster can give a warning closer to the predicted event compared to the long-range forecaster.

[0187] FIG. 15 illustrates a method 1500 to generate a prediction of a physiological event. The method 1500 can be implemented by any of various systems and devices described herein, including but not limited to the system 100 and / or the procedure monitor 120 described with reference to FIG. 1. The method 1500 can be implemented in any of various modalities, including but not limited to real-time, intraoperative techniques in which one or more aspects of the method 1500 are executed responsive to real-time sensor data detected regarding a subject on which a procedure is being performed, as well as any of various asynchronous implementations (e.g., for testing or validation, or updating of machine learning models of the procedure monitor 120). The method 1500 can be implemented using one or more machine learning models configured to use multiple forecasters.

[0188] In brief summary, the method 1500 can include monitoring one or more features of a subject during one or more data periods (1505). The method 1500 can include applying, as an input to one or more machine learning models, the one or more features to cause the one or more machine learning models to generate a plurality of predictions of the event during a plurality of forecast periods each corresponding to the one or more data periods (1510). The method 1500 can include outputting a plurality of indications of the plurality of predictions (1515).

[0189] The method 1500 can be performed, for example, in conjunction with signals regarding the subject, such as at least one of record data regarding the subject or sensor data from one or more sensors that monitor the subject during the procedure. For example, the subject (e.g., patient, user) can undergo a procedure such as a surgical procedure. The subject can be at risk of any of various intraoperative events, such as intraoperative hypotension, which can include an event associated with severe postoperative complications to renal, cardiovascular, and / or neurovascular systems. The method 1500 can assist the medical team (e.g., surgeon, anesthesiologist, nurse, staff) by providing timely predictions (e.g., alarms, alerts, warnings) of an adverse event (e.g., imminent adverse event) such as intraoperative hypotension. Based on the existence and / or timing of the multiple warnings, this can allow the medical team to decide how to respond to the potential adverse event.

[0190] Referring to FIG. 15 in further detail, the method 1500 can include monitoring one or more features (e.g., first features) of a subject (1505). The one or more features can be monitored by, for example, any of various procedure monitors 120 described with reference to FIG. 1. The one or more features can include features monitored in real-time. Data related to the one or more features can be received by the procedure monitors as a stream of data (e.g., data stream). The one or more features can include physiological features. The one or more features can include at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate. The one or more features can include signs (e.g., vital signs, multivariate vital signs). The one or more features can include physiological signals. The one or more features can include at least one of body temperature, pulse rate, respiration rate, or blood pressure.

[0191] The one or more features can be monitored using one or more sensors. The one or more sensors can include, for example, any of various one or more sensors 115described with reference to FIG. 1. Data from the one or more features can be received and / or monitored by a data processing system. The data processing system can include, for example, any of various data processing systems 105 described with reference to FIG. 1. The data can be obtained from or stored in, for example, any of various subject parameter databases 140 described with reference to FIG. 1. The data from one or more features can be sampled at various intervals. For example, the one or more features can be sample at one second intervals, 30 second intervals, 1 -minute intervals, 2-minute intervals, or 5 -minute intervals. The intervals can be frequent enough to sample enough data for predictive capabilities. The data can be obtained from the one or more sensors. For example, a heart rate monitor coupled to the subject can sample the subject’s heart rate at a specified time interval. For an intraoperative early warning system to predict changes in physiological status during surgery, the one or more features can be indicative of an event, such as an intraoperative hypotension event. By monitoring the one or more features, the method 1500 can give multiple indications of an imminent adverse event and permit intervention before the adverse event occurs.

[0192] The one or more features of the subject can be monitored during one or more periods relating to the procedure, including during a data period (e.g., a monitoring period). The data period can be a period during a procedure performed on the subject. The data period can occur prior to the event to be predicted (e.g., the event may not have yet occurred during the data period in which the features are monitored, such as to occur or be predicted to occur during a forecast period subsequent to the data period as described further below). The procedure can include a surgery (e.g., bariatric surgery, breast surgery, colon and rectal surgery, endocrine surgery, general surgery, gynecological surgery, hand surgery, head and neck surgery, hernia surgery, minimally invasive surgery, neurosurgery, orthopedic surgery, ophthalmological surgery, outpatient surgery, pediatric surgery, plastic and reconstructive surgery, robotic surgery, thoracic surgery, trauma surgery, urologic surgery, or vascular surgery). The event can include a hypotensive event. For example, the event can include an intraoperative hypotension.

[0193] The data period can include a period of time during which the one or more features is monitored. The data period can have a duration that represents a data interval. The data period can include a period of time during which data relating to the one or more features is collected from the subject. The data period can be in a range of 1 minute to 20 minutes. For example, the data period can be in a range of 1 minute to 3 minutes, 1 minuteto 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The data interval can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. Similar to the sampling rate of retrieving data from the sensors, the data interval for the data period can be long enough to sample sufficient data for predictive capabilities; the data interval can be a configurable setting and / or selected by the one or more machine learning models (e.g., based on testing or optimization of the one or more machine learning models). The data period can be a fixed amount of time or a variable amount of time before the event. The one or more features of the subject can include real-time signals retrieved from the one or more sensors during the one or more data periods. The data period can include the data interval 205. The length of the data period can be selected randomly during training of the plurality of machine learning models.

[0194] The method 1500 can include applying, as an input to one or more machine learning models (e.g., first machine learning model, second machine learning model), the one or more features (1510). The one or more machine learning models can include, for example, any of various machine learning models 125 described with reference to FIG. 1. Applying the one or more features as input to the one or more machine learning models can cause the one or more machine learning models to generate a plurality of indications of the plurality of predictions of the event of the subject. The plurality of predictions can include at least one of an indication that the event is expected to occur or a confidence score associated with the indication (e.g., a binary indication of whether the event is expected to occur; a percentage indication of a likelihood of the event being expected to occur; thebinary indication can be determined responsive to comparing the confidence score with a threshold).

[0195] The first machine learning model can generate a first prediction of the plurality of predictions of the event. The first machine learning model can generate a second prediction of the plurality of predictions of the event. The one or more machine learning models can include a single machine learning model with a variable forecast range. The second machine learning model can generate a second prediction of the plurality of predictions of the event. The one or more machine learning models can include multiple machine learning models (e.g., first machine learning model, second machine learning model) trained separately on different sets of data. Each set of data can have a fixed forecast range.

[0196] The event can be predicted to occur during a plurality of forecast periods (e.g., first forecast period, second forecast period, third forecast period). The plurality of forecast periods can each correspond to the one or more data periods. The plurality of forecast periods can be a future period relative to when the one or more machine learning models are caused to generate the plurality of predictions of the event (e.g., the one or more machine learning models can generate the plurality of predictions during the data period, where the plurality of predictions are for whether the event is likely to occur during the plurality of forecast periods). The plurality of forecast periods can be a period that is during the procedure performed on the subject. The plurality of forecast periods can be subsequent to the data period. The data period can be before the plurality of forecast periods. The plurality of forecast periods can be a fixed amount of time or a variable amount of time before the event. The plurality of forecast periods can be a fixed amount of time or a variable amount of time after the data period. The event can occur during or after the plurality of forecast periods. The second forecast period can be longer than the first forecast period. The third forecast period can be longer than the first forecast period and the second forecast period. The first forecast period can be shorter than the second forecast period and the third forecast period.

[0197] The plurality of forecast periods can have a duration that represents a forecast window, such as to represent a period of time in which the event is predicted to occur. The plurality of forecast periods can be in a range of 1 minute to 20 minutes. For example, the plurality of forecast periods can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes,1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The forecast window (e.g., forecast interval) can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. The period of time of the plurality of forecast periods can include an amount of time that the medical team has after receiving a warning of a potential adverse event until the adverse event occurs. A longer forecast period can give the medical team more time to respond to a potential adverse event. For example, the anesthesiologist can intervene during one of the plurality of forecast periods of time, given an early warning based on the data gathered during the data period. The plurality of forecast periods can include a short forecast period, medium forecast period, and / or long forecast period. The plurality of forecast periods can include the forecast window 210. The length of the plurality of forecast periods can be selected randomly during training of the one or more machine learning models.

[0198] The method 1500 can include applying, as input to the one or more machine learning models, one or more second features of the subject. The one or more second features can include data from the subject’s electronic medical record (EMR). For example, in addition to the sensor data of the one or more first features, the one or more second features can include record data, such as personal information or historical information. The one or more second features can include available medical information. The one or more second features can include at least one of age, sex, body mass index, height, or weight. The data can include a presence of the event (e.g., positive event) during the second period (e.g., as described above with reference to FIGS. 2A and 2B). The data can include an absence of the event (e.g., negative event) during the second period (e.g., as described above with reference to FIG. 3B). The data can have equal or roughly equal proportions (e.g., classbalance) of positive events and negative events. The data can have more positive events than negative events. The data can have more negative events than positive events.

[0199] Age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject. For example, age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure (e.g., as described above with reference to FIG. 7C). The one or more first features and the one or more second features can include one or more features with greater predictive capability (compared to other features) for a given event. For example, the one or more features can be a subset of the one or more first features and the one or more second features. These one or more features can be obtained from an ablation study (e.g., as described above with reference to FIG. 7C). The one or more features can include cardiac data regarding the subject and pulmonary data regarding the subject.

[0200] The one or more machine learning models can be configured using training data. The training data can include example data of the one or more features of a plurality of example subjects during an example data period. The training data can include example data of a presence of the event for the plurality of example subjects during at least one of the plurality of forecast periods. The at least one of the plurality of forecast periods can be subsequent to the example data period. The example data period can be before the at least one of the plurality of forecast periods. The one or more machine learning models can include a composite model configured using training data comprising data labeled with the procedure. The one or more machine learning models can be selected as a procedurespecific model using training data that includes data corresponding to the procedure.

[0201] The method 1500 can include determining an output prediction. The output prediction of the event can be based on a candidate prediction generated by each machine learning model of the one or more machine learning models. The candidate prediction can include at least one of an indication that the event is expected to occur or a confidence score associated with the indication (e.g., a binary indication of whether the event is expected to occur; a percentage indication of a likelihood of the event being expected to occur; the binary indication can be determined responsive to comparing the confidence score with athreshold). Determining the output prediction can include performing a voting operation using each candidate prediction. The output prediction can have an accuracy of greater than 80%. For example, the accuracy can be greater than 80%, greater than 85%, greater than 90%, greater than 91%, greater than 92%, greater than 93%, greater than 94%, or greater than 95% (e.g., as described above with reference to FIGS. 6A and 6B).

[0202] The method 1500 can include outputting a plurality of indications of the plurality of predictions (1515). For example, the output plurality of predictions can be displayed on an interface (e.g., display interface). The display interface can include, for example, any of various display interfaces 130 described with reference to FIG. 1. The output plurality of predictions can trigger multiple warnings of the event. Alternatively, the output plurality of predictions can be numerical scales related to the likelihood of the event. The output plurality of predictions can be a plurality of categorical predictions. For example, the output plurality of predictions can be positive or negative (e.g., yes or no, green light or red light). The output plurality of predictions can be numerical (e.g., ordinal) scales related to the likelihood of the event. For example, the output plurality of predictions can be a plurality of numbers (e.g., a plurality of numbers between 1 and 11) representing the likelihood of the event. The likelihood of the event can include the likelihood the event will occur in a specific time interval (e.g., 10 minutes). Based on the output plurality of predictions, the medical team can decide how to respond to the potential adverse event. The method 1500 can include outputting a plurality of numbers corresponding to the likelihood of the event.

[0203] The method 1500 can include periodically applying the input to the one or more machine learning models. For example, periodically applying the input to the one or more machine learning models can be responsive to sampling the one or more features from one or more sensors. The periodic application of input can improve the accuracy and / or predictive capability of the one or more machine learning models. Data from the one or more sensors regarding the one or more features can be collected by the data processing system. The data processing system can collect data regarding the one or more features by sampling the one or more sensors at regular or irregular intervals. The data can be stored in or retrieved from the subject parameter database. Periodically applying the input to the plurality of machine learning models can include using data from a sliding window.

[0204] The method 1500 can include training each of the one or more machine learning models with a subset of the training data. The training data can include a presenceof the event (e.g., positive event) during the plurality of forecast periods (e.g., as described above with reference to FIGS. 2A and 2B). The training data can include an absence of the event (e.g., negative event) during the plurality of forecast periods (e.g., as described above with reference to FIG. 3B). The training data can have equal or roughly equal proportions (e.g., class balance) of positive events and negative events. The training data can have more positive events than negative events. The training data can have more negative events than positive events.

[0205] The method 1500 can include determining that a condition is present responsive to at least a subset of the one or more machine learning models indicating that the condition is present. The method 1500 can include adjusting a threshold for the subset of the one or more machine learning models responsive to an input. The threshold can be changed so that the sensitivity of the one or more machine learning models is changed. Increasing the sensitivity (e.g., decreasing the specificity) may be useful for cases in which the medical team would like extra vigilance, and they are willing to tolerate an increased number of false alarms. Conversely, the medical team can decrease the sensitivity (e.g., increase the specificity) by increasing the threshold. By changing the threshold, the sensitivity may be adjusted (e.g., dialed, changed) to increase or decrease the sensitivity of the method 1500. The method 1500 can have adjustable sensitivity. The output prediction can be determined by a subset of the plurality of machine learning models being greater than the threshold. The threshold can be in a range of 10% to 100% of the plurality of machine learning models. For example, the threshold can be in a range of 10% to 20%, 10% to 30%, 10% to 40%, 10% to 50%, 10% to 60%, 10% to 70%, 10% to 80%, 10% to 90%, 10% to 100%, 20% to 30%, 20% to 40%, 20% to 50%, 20% to 60%, 20% to 70%, 20% to 80%, 20% to 90%, 20% to 100%, 30% to 40%, 30% to 50%, 30% to 60%, 30% to 70%, 30% to 80%, 30% to 90%, 30% to 100%, 40% to 50%, 40% to 60%, 40% to 70%, 40% to 80%, 40% to 90%, 40% to 100%, 50% to 60%, 50% to 70%, 50% to 80%, 50% to 90%, 50% to 100%, 60% to 70%, 60% to 80%, 60% to 90%, 60% to 100%, 70% to 80%, 70% to 90%, 70% to 100%, 80% to 90%, 80% to 100%, or 90% to 100%. A majority vote of the plurality of machine learning models can include a threshold of 50%. Adjustable sensitivity may be useful for cases in which the medical team only desires alerts when there is a strong indication and / or to reduce the percentage of false alarms.

[0206] FIG. 16 illustrates a method 1600 to train data for generating a prediction of a physiological event. The method 1600 can be implemented by any of various systems anddevices described herein, including but not limited to the system 100 and / or the procedure monitor 120 described with reference to FIG. 1. The method 1600 can be implemented in any of various modalities, including but not limited to real-time, intraoperative techniques in which one or more aspects of the method 1600 are executed responsive to real-time sensor data detected regarding a subject on which a procedure is being performed, as well as any of various asynchronous implementations (e.g., for testing or validation, or updating of machine learning models of the procedure monitor 120). The method 1600 can train one or more machine learning models configured to use multiple forecasters.

[0207] The method 1600 can use signals regarding the subject, such as at least one of record data regarding the subject or sensor data from one or more sensors that monitor the subject during the procedure. For example, the method 1600 can obtain (e.g., use, draw from) data from one or more subjects (patients, users) who have undergone a procedure such as a surgical procedure. The subject can be at risk of any of various intraoperative events, such as intraoperative hypotension, which can include an event associated with severe postoperative complications to renal, cardiovascular, and / or neurovascular systems.

[0208] The method 1600 can assist the medical team (e.g., surgeon, anesthesiologist, nurse, staff) by providing a model by which timely predictions (e.g., alarms, alerts, warnings) of an adverse event (e.g., imminent adverse event) such as intraoperative hypotension can be made. Based on the existence and / or timing of the multiple warnings, this can allow the medical team to decide how to respond to the potential adverse event. The method 1600 can be performed, for example, prior to, contemporaneous with, or after the method 1500. For example, the method 1500 can use the one or more machine learning models trained in method 1600. The method 1600 can train the one or more machine learning models while the method 1500 is implementing the one or more machine learning models.

[0209] In brief summary, the method 1600 can include identifying training data including one or more features of a subject during one or more data periods (1605). The method 1600 can include updating one or more machine learning models by applying the training data as input to the one or more machine learning models (1610).

[0210] Referring to FIG. 16 in further detail, the method 1600 can include identifying training data including one or more features (e.g., first features) of a subject (1605). The training data can include one or more features of a plurality of examplesubjects. The one or more features of the subject can include at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate. The one or more features can include signs (e.g., vital signs, multivariate vital signs). The one or more features can include physiological signals. The one or more features can include at least one of body temperature, pulse rate, respiration rate, or blood pressure.

[0211] Data related to the one or more features can be received and / or monitored by a data processing system. The data processing system can include, for example, any of various data processing systems 105 described with reference to FIG. 1. The data can be obtained from the one or more sensors. For example, a heart rate monitor coupled to the subject can sample the subject’s heart rate at a specified time interval. For an intraoperative early warning system to predict changes in physiological status during surgery, the one or more features can be indicative of an event, such as an intraoperative hypotension event. By identifying training data that includes the one or more features, the method 1600 can update the one or more machine learning models that can give multiple indications of an imminent adverse event and permit intervention before the adverse event occurs. The event can include a hypotensive event. For example, the event can include an intraoperative hypotension.

[0212] The training data can include one or more features of a plurality of example subjects during a data period. The data period can include a period of time during which the one or more features is monitored. The data period can have a duration that represents a data interval. The data period can include a period of time during which data relating to the one or more features is collected from the subject. The data period can be in a range of 1 minute to 20 minutes. For example, the data period can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10 minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The data interval can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. Similar to the sampling rate of retrieving data from the sensors, the data interval for the data period can be long enough to sample sufficient data for predictive capabilities; the data interval can be a configurable setting and / or selected by the machine learning models (e.g., based on testing or optimization of the machine learning models). The data period can be a fixed amount of time or a variable amount of time before the event. The one or more features of the subject can include real-time signals retrieved from the one or more sensors during the one or more data periods. The data period can include the data interval 205. The length of the data period can be selected randomly during training of the plurality of machine learning models.

[0213] The training data can include one or more features of the plurality of example subjects during a plurality of forecast periods. The plurality of forecast periods can be a period that is during the procedure performed on the subject. The plurality of forecast periods can be subsequent to the data period. The data period can be before the plurality of forecast periods. The plurality of forecast periods can be a fixed amount of time or a variable amount of time before the event. The plurality of forecast periods can be a fixed amount of time or a variable amount of time after the data period. The event can occur during or after the plurality of forecast periods.

[0214] The plurality of forecast periods can have a duration that represents a forecast window, such as to represent a period of time in which the event is predicted to occur. The plurality of forecast periods can be in a range of 1 minute to 20 minutes. For example, the plurality of forecast periods can be in a range of 1 minute to 3 minutes, 1 minute to 5 minutes, 1 minute to 8 minutes, 1 minute to 10 minutes, 1 minute to 13 minutes, 1 minute to 15 minutes, 1 minute to 18 minutes, 1 minute to 20 minutes, 3 minutes to 5 minutes, 3 minutes to 8 minutes, 3 minutes to 10 minutes, 3 minutes to 13 minutes, 3 minutes to 15 minutes, 3 minutes to 18 minutes, 3 minutes to 20 minutes, 5 minutes to 8 minutes, 5 minutes to 10 minutes, 5 minutes to 13 minutes, 5 minutes to 15 minutes, 5 minutes to 18 minutes, 5 minutes to 20 minutes, 8 minutes to 10 minutes, 8 minutes to 13 minutes, 8 minutes to 15 minutes, 8 minutes to 18 minutes, 8 minutes to 20 minutes, 10minutes to 13 minutes, 10 minutes to 15 minutes, 10 minutes to 18 minutes, 10 minutes to 20 minutes, 13 minutes to 15 minutes, 13 minutes to 18 minutes, 13 minutes to 20 minutes, 15 minutes to 18 minutes, 15 minutes to 20 minutes, or 18 minutes to 20 minutes. The forecast window (e.g., forecast interval) can be 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 11 minutes, 12 minutes, 13 minutes, 14 minutes, 15 minutes, 16 minutes, 17 minutes, 18 minutes, 19 minutes, or 20 minutes. The period of time of the plurality of forecast periods can include an amount of time that the medical team has after receiving a warning of a potential adverse event until the adverse event occurs. A longer forecast period can give the medical team more time to respond to a potential adverse event. For example, the anesthesiologist can intervene during one of the plurality of forecast periods of time, given an early warning based on the data gathered during the data period. The plurality of forecast periods can include a short forecast period, medium forecast period, and / or long forecast period. The plurality of forecast periods can include the forecast window 210. The length of the plurality of forecast periods can be selected randomly during training of the one or more machine learning models.

[0215] The training data can include one or more second features of the plurality of example subjects. The one or more second features can include data from the subject’s electronic medical record (EMR). For example, in addition to the sensor data of the one or more first features, the one or more second features can include record data, such as personal information or historical information. The one or more second features can include available medical information. The one or more second features can include available medical information. The one or more second features can include at least one of age, sex, body mass index, height, or weight. The training data can include a presence of the event (e.g., positive event) during the second period (e.g., as described above with reference to FIGS. 2A and 2B). The training data can include an absence of the event (e.g., negative event) during the second period (e.g., as described above with reference to FIG. 3B). The training data can have equal or roughly equal proportions (e.g., class balance) of positive events and negative events. The training data can have more positive events than negative events. The training data can have more negative events than positive events.

[0216] Age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject. For example, age, body mass index, arterial linediastolic, tidal volume, and mean arterial pressure can be selected from the one or more first features of the subject and the one or more second features of the subject according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure (e.g., as described above with reference to FIG. 7C). The one or more first features and the one or more second features can include one or more features with greater predictive capability (compared to other features) for a given event. For example, the one or more features can be a subset of the one or more first features and the one or more second features. These one or more features can be obtained from an ablation study (e.g., as described above with reference to FIG. 7C). The one or more features can include cardiac data regarding the subject and pulmonary data regarding the subject.

[0217] The method 1600 can include updating one or more machine learning models (1610). The one or more machine learning models can include, for example, any of various machine learning models 125 described with reference to FIG. 1. For example, updating the one or more machine learning models can include updating the one or more machine learning models by applying the training data as input to the one or more machine learning models.

[0218] The method 1600 can include performing a voting operation using the one or more machine learning models. The one or more machine learning models can be derived from the same underlying architecture. The predictions from the one or more machine learning models can be pooled. For example, the predictions from the one or more machine learning models can be pooled for voting.

[0219] The method 1600 can include sampling the one or more features from one or more sensors. For example, data from the one or more sensors regarding the one or more features can be collected by the data processing system. The data processing system can collect data regarding the one or more features by sampling the one or more sensors at regular or irregular intervals. The data can be stored in or retrieved from the subject parameter database.

[0220] The method 1600 can include determining that a condition is present responsive to at least a subset of the one or more machine learning models indicating that the condition is present. The method 1600 can include adjusting a threshold for the subset of the one or more machine learning models responsive to an input. The threshold can bechanged so that the sensitivity of the one or more machine learning models is changed. Increasing the sensitivity (e.g., decreasing the specificity) may be useful for cases in which the medical team would like extra vigilance, and they are willing to tolerate an increased number of false alarms. Conversely, the medical team can decrease the sensitivity (e.g., increase the specificity) by increasing the threshold. By changing the threshold, the sensitivity may be adjusted (e.g., dialed, changed) to increase or decrease the sensitivity of the method 1600. The method 1600 can have adjustable sensitivity. The output prediction can be determined by a subset of the plurality of machine learning models being greater than the threshold. The threshold can be in a range of 10% to 100% of the plurality of machine learning models. For example, the threshold can be in a range of 10% to 20%, 10% to 30%, 10% to 40%, 10% to 50%, 10% to 60%, 10% to 70%, 10% to 80%, 10% to 90%, 10% to 100%, 20% to 30%, 20% to 40%, 20% to 50%, 20% to 60%, 20% to 70%, 20% to 80%, 20% to 90%, 20% to 100%, 30% to 40%, 30% to 50%, 30% to 60%, 30% to 70%, 30% to 80%, 30% to 90%, 30% to 100%, 40% to 50%, 40% to 60%, 40% to 70%, 40% to 80%, 40% to 90%, 40% to 100%, 50% to 60%, 50% to 70%, 50% to 80%, 50% to 90%, 50% to 100%, 60% to 70%, 60% to 80%, 60% to 90%, 60% to 100%, 70% to 80%, 70% to 90%, 70% to 100%, 80% to 90%, 80% to 100%, or 90% to 100%. A majority vote of the plurality of machine learning models can include a threshold of 50%. Adjustable sensitivity may be useful for cases in which the medical team only desires alerts when there is a strong indication and / or to reduce the percentage of false alarms.

[0221] FIG. 17 is a block diagram of an example computing system suitable for use in the various arrangements described herein. In a non-limiting example, the computing system 1700 may implement a data processing system 105 of FIG. 1, or various other example systems and devices described in the present disclosure.

[0222] The computing system 1700 includes a bus 1702 or other communication component for communicating information and a processor 1704 coupled to the bus 1702 for processing information. The computing system 1700 also includes main memory 1706, such as a RAM or other dynamic storage device, coupled to the bus 1702 for storing information, and instructions to be executed by the processor 1704. Main memory 1706 may also be used for storing position information, temporary variables, or other intermediate information during execution of instructions by the processor 1704. The computing system 1700 may further include a ROM 1708 or other static storage device coupled to the bus 1702 for storing static information and instructions for the processor1704. A storage device 1710, such as a solid-state device, magnetic disk, or optical disk, is coupled to the bus 1702 for persistently storing information and instructions.

[0223] The computing system 1700 may be coupled via the bus 1702 to a display 1714, such as a liquid crystal display, or active-matrix display, for displaying information to a user. An input device 1712, such as a keyboard including alphanumeric and other keys, may be coupled to the bus 1702 for communicating information, and command selections to the processor 1704. In another implementation, the input device 1712 has a touch screen display. The input device 1712 may include any type of biometric sensor, or a cursor control, such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor 1704 and for controlling cursor movement on the display 1714.

[0224] In some implementations, the computing system 1700 may include a communications adapter 1716, such as a networking adapter. Communications adapter 1716 may be coupled to bus 1702 and may be configured to enable communications with a computing or communications network or other computing systems. In various illustrative implementations, any type of networking configuration may be achieved using communications adapter 1716, such as wired (e.g., via Ethernet), wireless (e.g., via Wi-Fi, Bluetooth), satellite (e.g., via GPS) pre-configured, ad-hoc, LAN, WAN, and the like.

[0225] According to various implementations, the processes of the illustrative implementations that are described herein may be achieved by the computing system 1700 in response to the processor 1704 executing an implementation of instructions contained in main memory 1706. Such instructions may be read into main memory 1706 from another computer-readable medium, such as the storage device 1710. Execution of the implementation of instructions contained in main memory 1706 causes the computing system 1700 to perform the illustrative processes described herein. One or more processors in a multi-processing implementation may also be employed to execute the instructions contained in main memory 1706. In alternative implementations, hard-wired circuitry may be used in place of or in combination with software instructions to implement illustrative implementations. Thus, implementations are not limited to any specific combination of hardware circuitry and software.

[0226] The systems and methods of the present disclosure can have a high level of performance in predicting IOH, compared to other methods that may only attempt genericrisk prediction (e.g., IOH occurring at any time in the procedure at all) or prediction within the first 10 minutes of the procedure (early stage only, without continuously streaming physiological data to continuously monitor trajectory and predict an imminent IOH event). Initial ablation studies demonstrated that age, BMI, arterial line diastolic pressure, tidal volume, and MAP can be among the most important features in predicting IOH. The systems and methods of the present disclosure can be applicable to other surgeries and / or predict other forms of adverse events and / or physiological trajectories (e.g., hypothermia or hypoxia). The systems and methods of the present disclosure can also be applicable to predict other forms of post-operative status, including post-operative pain, cognitive deficit, and length-of-stay.

[0227] Appendix A provides additional implementations of the systems and methods of the present disclosure, including, for example, static and / or dynamic sliding windows to test and / or improve the models with “truly negative” data before possible IOH events, solutions to class imbalance in training, and configurations of the models based on specificity and / or sensitivity data. Appendix A is incorporated herein by reference in its entirety.

[0228] Having now described some illustrative implementations, it is apparent that the foregoing is illustrative and not limiting, having been presented by way of example. In particular, although many of the examples presented herein involve specific combinations of method acts or system elements, those acts and those elements may be combined in other ways to accomplish the same objectives. Acts, elements and features discussed in connection with one implementation are not intended to be excluded from a similar role in other implementations or implementations.

[0229] The phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including” “comprising” “having” “containing” “involving” “characterized by” “characterized in that” and variations thereof herein, is meant to encompass the items listed thereafter, equivalents thereof, and additional items, as well as alternate implementations consisting of the items listed thereafter exclusively. In one implementation, the systems and methods described herein consist of one, each combination of more than one, or all of the described elements, acts, or components.

[0230] Embodiments of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. The subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more circuits of computer program instructions, encoded on one or more computer storage media for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate components or media (e.g., multiple CDs, disks, or other storage devices).

[0231] The operations described in this specification can be performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources. The term “data processing apparatus” or “computing device” encompasses various apparatuses, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures.

[0232] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a circuit, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more circuits, subprograms, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0233] Processors suitable for the execution of a computer program include, by way of example, microprocessors, and any one or more processors of a digital computer. A processor can receive instructions and data from a read only memory or a random-access memory or both. The elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. A computer can include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. A computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a personal digital assistant (PDA), a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0234] To provide for interaction with a user, implementations of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used toprovide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0235] The implementations described herein can be implemented in any of numerous ways including, for example, using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

[0236] Also, a computer may have one or more input and output devices. These devices can be used, among other things, to present a user interface. Examples of output devices that can be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound generating devices for audible presentation of output. Examples of input devices that can be used for a user interface include keyboards, and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computer may receive input information through speech recognition or in other audible format.

[0237] Such computers may be interconnected by one or more networks in any suitable form, including a local area network or a wide area network, such as an enterprise network, and intelligent network (IN) or the Internet. Such networks may be based on any suitable technology and may operate according to any suitable protocol and may include wireless networks, wired networks or fiber optic networks.

[0238] A computer employed to implement at least a portion of the functionality described herein may comprise a memory, one or more processing units (also referred to herein simply as “processors”), one or more communication interfaces, one or more display units, and one or more user input devices. The memory may comprise any computer- readable media, and may store computer instructions (also referred to herein as “processorexecutable instructions”) for implementing the various functionalities described herein. The processing unit(s) may be used to execute the instructions. The communication interface(s) may be coupled to a wired or wireless network, bus, or other communication means and may therefore allow the computer to transmit communications to or receive communications from other devices. The display unit(s) may be provided, for example, toallow a user to view various information in connection with execution of the instructions. The user input device(s) may be provided, for example, to allow the user to make manual adjustments, make selections, enter data or various other information, or interact in any of a variety of manners with the processor during execution of the instructions.

[0239] The various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine.

[0240] In this respect, various inventive concepts may be embodied as a computer readable storage medium (or multiple computer readable storage media) (e.g., a computer memory, one or more floppy discs, compact discs, optical discs, magnetic tapes, flash memories, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other non-transitory medium or tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the solution discussed above. The computer readable medium or media can be transportable, such that the program or programs stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the present solution as discussed above.

[0241] The terms “program” or “software” are used herein to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor to implement various aspects of embodiments as discussed above. One or more computer programs that when executed perform methods of the present solution need not reside on a single computer or processor, but may be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the present solution.

[0242] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Program modules can include routines, programs, objects, components, data structures, or other components thatperform particular tasks or implement particular abstract data types. The functionality of the program modules can be combined or distributed as desired in various embodiments.

[0243] Also, data structures may be stored in computer-readable media in any suitable form. For simplicity of illustration, data structures may be shown to have fields that are related through location in the data structure. Such relationships may likewise be achieved by assigning storage for the fields with locations in a computer-readable medium that convey relationship between the fields. However, any suitable mechanism may be used to establish a relationship between information in fields of a data structure, including through the use of pointers, tags or other mechanisms that establish relationship between data elements.

[0244] Any references to implementations or elements or acts of the systems and methods herein referred to in the singular can include implementations including a plurality of these elements, and any references in plural to any implementation or element or act herein can include implementations including only a single element. References in the singular or plural form are not intended to limit the presently disclosed systems or methods, their components, acts, or elements to single or plural configurations. References to any act or element being based on any information, act or element may include implementations where the act or element is based at least in part on any information, act, or element.

[0245] Any implementation disclosed herein may be combined with any other implementation, and references to “an implementation,” “some implementations,” “an alternate implementation,” “various implementations,” “one implementation” or the like are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the implementation may be included in at least one implementation. Such terms as used herein are not necessarily all referring to the same implementation. Any implementation may be combined with any other implementation, inclusively or exclusively, in any manner consistent with the aspects and implementations disclosed herein.

[0246] References to “or” may be construed as inclusive so that any terms described using “or” may indicate any of a single, more than one, and all of the described terms. References to at least one of a conjunctive list of terms may be construed as an inclusive OR to indicate any of a single, more than one, and all of the described terms. For example, areference to “at least one of ‘A’ and ‘B’” can include only ‘A’, only ‘B’, as well as both ‘A’ and ‘B’. Elements other than ‘A’ and ‘B’ can also be included.

[0247] The systems and methods described herein may be embodied in other specific forms without departing from the characteristics thereof. The foregoing implementations are illustrative rather than limiting of the described systems and methods.

[0248] Where technical features in the drawings, detailed description or any claim are followed by reference signs, the reference signs have been included to increase the intelligibility of the drawings, detailed description, and claims. Accordingly, neither the reference signs nor their absence have any limiting effect on the scope of any claim elements.

[0249] The systems and methods described herein may be embodied in other specific forms without departing from the characteristics thereof. The foregoing implementations are illustrative rather than limiting of the described systems and methods. Scope of the systems and methods described herein is thus indicated by the appended claims, rather than the foregoing description, and changes that come within the meaning and range of equivalency of the claims are embraced therein.

Claims

WHAT IS CLAIMED IS:

1. A method, comprising: monitoring, by one or more processors, one or more features of a subject during a first period during a procedure performed on the subject; applying, by the one or more processors, as input to a plurality of machine learning models, the one or more features to cause the plurality of machine learning models to each generate a candidate prediction of an event of the subject during a second period subsequent to the first period, the plurality of machine learning models configured using training data comprising example data of the one or more features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period; determining, by the one or more processors, an output prediction of the event based on each candidate prediction generated by each machine learning model of the plurality of machine learning models; and outputting, by the one or more processors, an indication of the output prediction of the event.

2. The method of claim 1, wherein the one or more features of the subject comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

3. The method of claim 1, wherein the event comprises a hypotensive event.

4. The method of claim 1, wherein: the first period is in a range of 1 minute to 20 minutes; and the second period is in a range of 1 minute to 20 minutes.

5. The method of claim 1, wherein the one or more features comprise one or more first features of the subject, the method further comprising applying, as input to the plurality of machine learning models, one or more second features of the subject, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

6. The method of claim 1, wherein determining the output prediction comprises performing a voting operation using each candidate prediction.

7. The method of claim 1, further comprising periodically applying the input to the plurality of machine learning models responsive to sampling the one or more features from one or more sensors.

8. The method of claim 1, further comprising training each of the plurality of machine learning models with a subset of the training data.

9. The method of claim 1, wherein the training data comprises an absence of the event during the second period.

10. The method of claim 1, wherein the one or more features comprise one or more first features of the subject, the method further comprising: applying, as input to the plurality of machine learning models, one or more second features of the subject; wherein the one or more first features and the one or more second features are selected from and according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure.

11. The method of claim 1, further comprising: determining that a condition is present responsive to at least a subset of the plurality of machine learning models indicating that the condition is present.

12. The method of claim 11, further comprising: adjusting a threshold for the subset of the plurality of machine learning models responsive to an input.

13. The method of claim 1, further comprising: outputting, by the one or more processors, a number corresponding to a likelihood of the event.

14. The method of claim 1, comprising: detecting, by the one or more processors, the event as a second event; wherein the input comprises an indication of a prior event and a dosage of a treatment administered subsequent to the prior event.

15. The method of claim 1, wherein the one or more features comprise (1) cardiac data regarding the subject and (2) pulmonary data regarding the subject.

16. A system, comprising: one or more processors configured to: monitor one or more features of a subject during a first period during a procedure performed on the subject; apply, as input to a plurality of machine learning models, the one or more features to cause the plurality of machine learning models to each generate a candidate prediction of an event of the subject during a second period subsequent to the first period, the plurality of machine learning models configured using training data comprising example data of the one or more features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period; determine an output prediction of the event based on each candidate prediction generated by each machine learning model of the plurality of machine learning models; and output an indication of the output prediction of the event.

17. The system of claim 16, wherein the one or more features of the subject comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

18. The system of claim 16, wherein: the event comprises a hypotensive event; the first period is in a range of 1 minute to 20 minutes; and the second period is in a range of 1 minute to 20 minutes.

19. The system of claim 16, wherein: the one or more features comprise one or more first features of the subject; and the one or more processors is configured to apply, as input to the plurality of machine learning models, one or more second features of the subject, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

20. The system of claim 16, wherein the one or more processors is configured to perform a voting operation using each candidate prediction.

21. A method, comprising: identifying, by one or more processors, training data comprising one or more features of a plurality of example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period; and updating, by the one or more processors, a plurality of machine learning models by applying a respective subset of the training data as input to each respective machine learning model of the plurality of machine learning models.

22. The method of claim 21, wherein the one or more features of the plurality of example subject comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

23. The method of claim 21, wherein the event comprises a hypotensive event.

24. The method of claim 21, wherein the first period is in a range of 1 minute to 20 minutes.

25. The method of claim 21, wherein the second period is in a range of 1 minute to 20 minutes.

26. The method of claim 21, wherein: the one or more features comprise one or more first features of the plurality of example subjects; andthe training data comprises one or more second features of the plurality of example subjects, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

27. The method of claim 21, further comprising performing a voting operation using the plurality of machine learning models.

28. The method of claim 21, further comprising sampling the one or more features from one or more sensors.

29. The method of claim 21, wherein the training data comprises an absence of the event during the second period.

30. The method of claim 21, wherein: the one or more features comprise one or more first features of the plurality of example subjects; the training data comprises one or more second features of the plurality of example subjects; and age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure are selected from the one or more first features of the plurality of example subjects and the one or more second features of the plurality of example subjects according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure.

31. The method of claim 21, further comprising: determining that a condition is present responsive to at least a subset of the plurality of machine learning models indicating that the condition is present.

32. The method of claim 31, further comprising: adjusting a threshold for the subset of the plurality of machine learning models responsive to an input.

33. A system, comprising: one or more processors configured to:identify training data comprising one or more features of a plurality of example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period; and update a plurality of machine learning models by applying a respective subset of the training data as input to each machine learning model of the plurality of machine learning models.

34. The system of claim 33, wherein the one or more features of the plurality of example subjects comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

35. The system of claim 33, wherein: the event comprises a hypotensive event; the first period is in a range of 1 minute to 20 minutes; and the second period is in a range of 1 minute to 20 minutes.

36. The system of claim 33, wherein: the one or more features comprise one or more first features of the plurality of example subjects; and the training data comprises one or more second features of the plurality of example subjects, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

37. The system of claim 33, wherein the one or more processors is configured to perform a voting operation using the plurality of machine learning models.

38. The system of claim 33, wherein the one or more processors is configured to sample the one or more features from one or more sensors.

39. The system of claim 33, wherein the training data comprises an absence of the event during the second period.

40. The system of claim 33, wherein: the one or more features comprise one or more first features of the plurality of example subjects; the training data comprises one or more second features of the plurality of example subjects; and age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure are selected from the one or more first features of the plurality of example subjects and the one or more second features of the plurality of example subjects according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure.

41. A method, comprising: monitoring, by one or more processors, a plurality of features of a subject during a first period during a procedure performed on the subject, at least one feature of the plurality of features selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject; applying, by the one or more processors, as input to one or more machine learning models, the plurality of features to cause the one or more machine learning models to generate a prediction of the event during a second period subsequent to the first period, the one or more machine learning models configured using training data comprising example data of the plurality of features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period; and outputting, by the one or more processors, an indication of the prediction of the event.

42. The method of claim 41, wherein the plurality of features of the subject comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

43. The method of claim 41, wherein the event comprises a hypotensive event.

44. The method of claim 41, wherein the first period is in a range of 1 minute to 20 minutes.

45. The method of claim 41, wherein the second period is in a range of 1 minute to 20 minutes.

46. The method of claim 41, wherein the plurality of features comprise one or more first features of the subject, the method further comprising applying, as input to the one or more machine learning models, one or more second features of the subject, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

47. The method of claim 41, further comprising performing a voting operation using a candidate prediction.

48. The method of claim 41, further comprising periodically applying the input to the one or more machine learning models responsive to sampling the plurality of features from one or more sensors.

49. The method of claim 41, further comprising training each of the one or more machine learning models with a subset of the training data.

50. The method of claim 41, wherein the training data comprises an absence of the event during the second period.

51. The method of claim 41, wherein the plurality of features comprise one or more first features of the subject, the method further comprising: applying, as input to the one or more machine learning models, one or more second features of the subject; wherein age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure are selected from the one or more first features of the subject and the one or more second features of the subject according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure.

52. The method of claim 41, wherein the plurality of features comprise (1) cardiac data regarding the subject and (2) pulmonary data regarding the subject.

53. The method of claim 41, wherein the one or more machine learning models comprises a composite model configured using training data comprising data labeled with the procedure.

54. The method of claim 41, wherein the one or more machine learning models is selected as a procedure-specific model using training data comprising data corresponding to the procedure.

55. A system, comprising: one or more processors configured to: monitor a plurality of features of a subject during a first period during a procedure performed on the subject, at least one feature of the plurality of features selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject; apply, as input to one or more machine learning models, the plurality of features to cause the one or more machine learning models to generate a prediction of the event during a second period subsequent to the first period, the one or more machine learning models configured using training data comprising example data of the plurality of features of a plurality of example subjects during a first example period and of a presence of the event for the plurality of example subjects during a second example period subsequent to the first example period; and output an indication of the prediction of the event.

56. The system of claim 55, wherein the plurality of features of the subject comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

57. The system of claim 55, wherein: the event comprises a hypotensive event; the first period is in a range of 1 minute to 20 minutes; and the second period is in a range of 1 minute to 20 minutes.

58. The system of claim 55, wherein: the one or more features comprise one or more first features of the subject; andthe one or more processors is configured to apply, as input to the one or more machine learning models, one or more second features of the subject, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

59. The system of claim 55, further comprising performing a voting operation using a candidate prediction.

60. The system of claim 55, wherein: the one or more processors is configured to periodically apply the input to the one or more machine learning models responsive to sampling the plurality of features from one or more sensors; and the one or more processors is configured to train each of one or more machine learning models with a subset of the training data.

61. A method, comprising: identifying, by one or more processors, training data comprising a plurality of features of a plurality of example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period; and updating, by the one or more processors, one or more machine learning models by applying the training data as input to the one or more machine learning models.

62. The method of claim 61, wherein the plurality of features of the plurality of example subjects comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

63. The method of claim 61, wherein the event comprises a hypotensive event.

64. The method of claim 61, wherein the first period is in a range of 1 minute to 20 minutes.

65. The method of claim 61, wherein the second period is in a range of 1 minute to 20 minutes.

66. The method of claim 61, wherein: the plurality of features comprise one or more first features of the plurality of example subjects; and the training data comprises a plurality of second features of the plurality of example subjects, the plurality of second features comprising at least one of age, sex, body mass index, height, or weight.

67. The method of claim 61, further comprising performing a voting operation using the one or more machine learning models.

68. The method of claim 61, further comprising sampling the one or more features from one or more sensors.

69. The method of claim 61, wherein the training data comprises an absence of the event during the second period.

70. The method of claim 61, wherein: the one or more features comprise one or more first features of the plurality of example subjects; the training data comprises one or more second features of the plurality of example subjects; and age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure are selected from the one or more first features of the plurality of example subjects and the one or more second features of the plurality of example subjects according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure.

71. A system, comprising: one or more processors configured to: identify training data comprising a plurality of features of a plurality of example subjects during a first period and of a presence of an event for the plurality of example subjects during a second period subsequent to the first period; and update one or more machine learning models by applying the training data as input to the one or more machine learning models.

72. The system of claim 71, wherein the plurality of features of the plurality of example subjects comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

73. The system of claim 71, wherein the event comprises a hypotensive event.

74. The system of claim 71, wherein the first period is in a range of 1 minute to 20 minutes.

75. The system of claim 71, wherein the second period is in a range of 1 minute to 20 minutes.

76. The system of claim 71, wherein: the plurality of features comprise a plurality of first features of the plurality of example subjects; and the training data comprises a plurality of second features of the plurality of example subjects, the plurality of second features comprising at least one of age, sex, body mass index, height, or weight.

77. The system of claim 71, wherein the one or more processors is configured to perform a voting operation using the one or more machine learning models.

78. The system of claim 71, wherein the one or more processors is configured to sample the plurality of features from one or more sensors.

79. The system of claim 71, wherein the training data comprises an absence of the event during the second period.

80. The system of claim 71, wherein: the plurality of features comprise a plurality of first features of the plurality of example subjects;the training data comprises a plurality of second features of the plurality of example subjects; and age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure are selected from the plurality of first features of the plurality of example subjects and the plurality of second features of the plurality of example subjects according to an evaluation of predictive capability of age, body mass index, arterial line diastolic, tidal volume, and mean arterial pressure.

81. A method, comprising: monitoring, by one or more processors, one or more features of a subject during one or more data periods during a procedure performed on the subject, at least one feature of the one or more features selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject; applying, by the one or more processors, as input to one or more machine learning models, the one or more features to cause the one or more machine learning models to generate a plurality of predictions of the event during a plurality of forecast periods each corresponding to the one or more data periods, each of the plurality of forecast periods subsequent to the corresponding one or more data periods; and outputting, by the one or more processors, a plurality of indications of the plurality of predictions of the event.

82. The method of claim 81, wherein: the one or more machine learning models comprises a first machine learning model; the first machine learning model to generate a first prediction of the plurality of predictions of the event; and the first machine learning model to generate a second prediction of the plurality of predictions of the event.

83. The method of claim 81, wherein: the one or more machine learning models comprises a first machine learning model and a second machine learning model; the first machine learning model to generate a first prediction of the plurality of predictions of the event; andthe second machine learning model to generate a second prediction of the plurality of predictions of the event.

84. The method of claim 81, wherein the plurality of forecast periods comprise: a first forecast period; and a second forecast period longer than the first forecast period.

85. The method of claim 81, wherein the plurality of forecast periods comprise: a first forecast period; a second forecast period longer than the first forecast period; and a third forecast period longer than the first forecast period and the second forecast period.

86. The method of claim 81, further comprising periodically applying the input to the one or more machine learning models responsive to sampling the one or more features from one or more sensors.

87. The method of claim 81, wherein monitoring the one or more features of the subject comprises retrieving real-time signals from one or more sensors during the one or more data periods.

88. The method of claim 81, wherein the one or more features of the subject comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

89. The method of claim 81, wherein: the event comprises a hypotensive event; at least one of the one or more data periods is in a range of 1 minute to 20 minutes; and at least one of the plurality of forecast periods is in a range of 1 minute to 20 minutes.

90. The method of claim 81, wherein the one or more features comprise one or more first features of the subject, the method further comprising applying, as input to the one or more machine learning models, one or more second features of the subject, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

91. The method of claim 81, further comprising performing a voting operation using a candidate prediction.

92. The method of claim 81, further comprising training each of the one or more machine learning models with training data.

93. The method of claim 92, wherein the training data comprises an absence of the event during the plurality of forecast periods.

94. The method of claim 81, wherein the one or more machine learning models comprises a composite model configured using training data comprising data labeled with the procedure.

95. The method of claim 81, wherein the one or more machine learning models is selected as a procedure-specific model using training data comprising data corresponding to the procedure.

96. A system, comprising: one or more processors configured to: monitor one or more features of a subject during one or more data periods during a procedure performed on the subject, at least one feature of the one or more features selected according to a predictive capability of the at least one feature with respect to predicting an event of the subject; apply, as input to one or more machine learning models, the one or more features to cause the one or more machine learning models to generate a plurality of predictions of the event during a plurality of forecast periods each corresponding to the one or more data periods, each of the plurality of forecast periods subsequent to the corresponding one or more data periods; and output a plurality of indications of the plurality of predictions of the event.

97. The system of claim 96, wherein the one or more features of the subject comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

98. The system of claim 96, wherein the event comprises a hypotensive event.

99. The system of claim 96, wherein: the one or more features comprise one or more first features of the subject; the one or more processors is configured to apply, as input to the one or more machine learning models, one or more second features of the subject, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

100. The system of claim 96, further comprising performing a voting operation using a candidate prediction.

101. A method, comprising: identifying, by one or more processors, training data comprising one or more features of a plurality of example subjects during one or more data periods and of a presence of an event for the plurality of example subjects during a plurality of forecast periods each corresponding to the one or more data periods, each of the plurality of forecast periods subsequent to the corresponding one or more data periods; and updating, by the one or more processors, one or more machine learning models by applying the training data as input to the one or more machine learning models.

102. The method of claim 101, wherein: the one or more machine learning models comprises a first machine learning model; the first machine learning model to generate a first prediction of the event; and the first machine learning model to generate a second prediction of the event.

103. The method of claim 101, wherein: the one or more machine learning models comprises a first machine learning model and a second machine learning model;the first machine learning model to generate a first prediction of the event; and the second machine learning model to generate a second prediction of the event.

104. The method of claim 101, wherein the plurality of forecast periods comprise: a first forecast period; and a second forecast period longer than the first forecast period.

105. The method of claim 101, wherein the plurality of forecast periods comprise: a first forecast period; a second forecast period longer than the first forecast period; and a third forecast period longer than the first forecast period and the second forecast period.

106. The method of claim 101, further comprising sampling the one or more features from one or more sensors.

107. The method of claim 101, further comprising retrieving real-time signals from one or more sensors during the one or more data periods.

108. The method of claim 101, wherein the one or more features of the plurality of example subjects comprise at least one of arterial line diastolic, arterial line systolic, mean arterial pressure, peak inspiratory pressure, minute volume, fraction of inspired O2, O2 flow, bispectral index monitor, inspired desflurane, expired desflurane, tidal volume, respiratory rate, minute volume, or heart rate.

109. The method of claim 101, wherein the event comprises a hypotensive event.

110. The method of claim 101, wherein at least one of the one or more data periods is in a range of 1 minute to 20 minutes.

111. The method of claim 101, wherein at least one of the plurality of forecast periods is in a range of 1 minute to 20 minutes.

112. The method of claim 101, wherein:the one or more features comprise one or more first features of the plurality of example subjects; and the training data comprises one or more second features of the plurality of example subjects, the one or more second features comprising at least one of age, sex, body mass index, height, or weight.

113. The method of claim 101, further comprising performing a voting operation using the one or more machine learning models.

114. The method of claim 101, wherein the training data comprises an absence of the event during the plurality of forecast periods.

115. A system, comprising: one or more processors configured to: identify training data comprising one or more features of a plurality of example subjects during one or more data periods and of a presence of an event for the plurality of example subjects during a plurality of forecast periods each corresponding to the one or more data periods, each of the plurality of forecast periods subsequent to the corresponding one or more data periods; and update one or more machine learning models by applying the training data as input to the one or more machine learning models.

116. The system of claim 115, wherein: the one or more machine learning models comprises a first machine learning model; the first machine learning model to generate a first prediction of the event; and the first machine learning model to generate a second prediction of the event.

117. The system of claim 115, wherein: the one or more machine learning models comprises a first machine learning model and a second machine learning model; the first machine learning model to generate a first prediction of the event; and the second machine learning model to generate a second prediction of the event.

118. The system of claim 115, wherein the plurality of forecast periods comprise:a first forecast period; and a second forecast period longer than the first forecast period.

119. The system of claim 115, wherein the plurality of forecast periods comprise: a first forecast period; a second forecast period longer than the first forecast period; and a third forecast period longer than the first forecast period and the second forecast period.

120. The system of claim 115, wherein the one or more processors is configured to retrieve real-time signals from one or more sensors during the one or more data periods.

Images