COMPUTERIZED SYSTEMS AND METHODS FOR ENSEMBLE MODEL-BASED DRUG DISCOVERY - Patent application

Abstract

A system and method are disclosed that provides a novel framework for decision intelligence (DI)-based drug adjudication. The disclosed framework may leverage dynamically and recursively trained artificial intelligence / machine learning (AI / ML) ensemble configurations to analyze genomic data and features arriving from the same. Ensemble adjudication and application can increase the accuracy of training, validation, and external test sets associated with drug discovery and personalization. The ensemble-based computerized framework can be configured for analysis of samples using ensemble algorithms trained with binary mutation data and hierarchical clustering data, which can enable adjudication of drug efficacy and patent stratification.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Provisional Application No. 63 / 269,148, filed March 10, 2022, which is incorporated herein by reference in its entirety.

[0002] FIELD OF THEINVENTION The present disclosure relates generally to drug efficacy prediction and patient stratification, and more specifically to a computerized framework for leveraging the application of decision-intelligence (DI)-based models for drug discovery and applicability monitoring. [Background technology]

[0003] Over the past few years, data digitization in the pharmaceutical sector has increased dramatically. However, such digitization comes with challenges to capture, scrutinize, and apply that knowledge to solve complex clinical problems, among other drawbacks. Summary of the Invention

[0004] According to some embodiments, systems and methods for a novel computerized framework for such digitalized DI-based drug determination are disclosed. As discussed herein, the computerized framework may leverage dynamically and recursively trained artificial intelligence / machine learning (AI / ML) ensemble configurations to analyze genomic data and features arriving from the same. According to some embodiments, ensemble determination and application can increase the accuracy of training, validation, and external test sets associated with drug discovery and personalization. Furthermore, in some embodiments, the ensemble-based computerized framework can be configured for analysis of samples using ensemble algorithms trained with binary mutation data and hierarchical clustering data, which can enable determination of drug efficacy and patent stratification.

[0005] Thus, as is evident from the present disclosure, the use of an ensemble-based algorithmic approach provides improved accuracy compared to the use of a single algorithm (e.g., on the same data). For example, in predicting a patient's drug response, the ensemble approach can demonstrate a 7.7% increase in accuracy of drug selection and dosage discrimination for a particular patient. Thus, as discussed herein, more robust and accurate analysis and decision making is advanced through ensemble configuration.

[0006] As discussed herein, an ensemble model refers to a multi-layered combination of AI / ML (and / or deep learning) algorithms that trains a final algorithm and combines the predictions of other algorithms as features used to generate a final ensemble. The ensemble of predictors can be trained individually and have their hyperparameters tuned for optimal performance. Conceptually, the ensemble allows the higher layer algorithms to learn from the performance of the lower layers. Furthermore, the nature of ensembles provides a solution to some common problems in machine learning. Thus, when tuning a single algorithm, there may be contexts where one set of hyperparameters is optimal, but there may be frequent exceptions that lead to inaccurate results. However, as discussed herein, in an ensemble, an algorithm (and / or set of algorithms) may be repeated many times with different tuning, thereby expanding the functionality and applicability of drug selection and application to patient conditions.

[0007] According to some embodiments, a method for a DI-based computerized framework for deterministically monitoring and tracking identified drug applicability for a patient's condition is disclosed. According to some embodiments, the present disclosure provides a non-transitory computer-readable storage medium for performing the above-mentioned technical steps of the functionality of the framework. The non-transitory computer-readable storage medium has computer-readable instructions tangibly stored thereon or tangibly encoded thereon, the instructions, when executed by a device, causing at least one processor to perform the method for a DI-based computerized framework for deterministically monitoring and tracking identified drug applicability for a patient's condition.

[0008] According to one or more embodiments, a system is provided that includes one or more processors and / or computing devices configured to provide functionality according to such embodiments. According to one or more embodiments, the functionality is embodied in method steps performed by at least one computing device. According to one or more embodiments, program code (or program logic) executed by a processor of a computing device to perform functionality according to one or more such embodiments is embodied in, by, and / or on a non-transitory computer-readable medium. [Brief description of the drawings]

[0009] Features and advantages of the present disclosure will become apparent from the following description of embodiments as illustrated in the accompanying drawings, in which reference characters refer to the same parts throughout the various views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure. [Figure 1] FIG. 1 is a block diagram of an example configuration in which the systems and methods disclosed herein may be implemented according to some embodiments of the present disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating components of an example system according to some embodiments of the present disclosure. [Diagram 3] 1 illustrates an example workflow according to some embodiments of the present disclosure. [Figure 4] 1 depicts a non-limiting ensemble model configuration according to some embodiments of the present disclosure. [Diagram 5] 1 depicts an example implementation of an architecture according to some embodiments of the present disclosure. [Figure 6] 1 depicts an example implementation of an architecture according to some embodiments of the present disclosure. [Figure 7] FIG. 1 is a block diagram illustrating a computing device that represents an example of a client or server device for use in various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The present disclosure will now be described more fully below with reference to the accompanying drawings, which form a part of this specification and show, by way of non-limiting illustration, certain exemplary embodiments. However, the subject matter may be embodied in a variety of different forms, and therefore, it is intended that the subject matter encompassed or claimed be construed as not being limited to any exemplary embodiment described herein, which exemplary embodiment is provided merely for illustration. Likewise, a reasonably broad scope of the subject matter claimed or encompassed is intended. Among other things, for example, the subject matter may be embodied as a method, device, component, or system. Thus, the embodiments may take the form of, for example, hardware, software, firmware, or any combination thereof (other than software itself). Thus, the following detailed description is not intended to be construed in a limiting sense.

[0011] Throughout this specification and claims, terms may have subtle meanings that are suggested or implied in the context beyond the explicitly stated meaning. Similarly, the phrase "in one embodiment" as used herein does not necessarily refer to the same embodiment, and the phrase "in another embodiment" as used herein does not necessarily refer to different embodiments. For example, the claimed subject matter is intended to include in whole or in part a combination of the exemplary embodiments.

[0012] Generally, terminology may be understood at least in part from use in context. For example, terms such as "and", "or", or "and / or" as used herein may include various meanings that may depend at least in part on the context in which such terms are used. Typically, when "or" is used to relate a list such as A, B, or C, it is intended to mean A, B, and C, which are used herein in an inclusive sense, as well as A, B, or C, which are used herein in an exclusive sense. In addition, the term "one or more" as used herein may be used to describe any feature, structure, or characteristic in any of the singular forms, or may be used to describe a combination of features, structures, or characteristics in the plural, at least in part depending on the context. Similarly, terms such as "a", "an", or "the" may be understood to convey the use of the singular form or to convey the use of the plural, again at least in part depending on the context. In addition, the term "based on" may be understood not necessarily intended to convey an exclusive set of factors, but instead may allow for the presence of additional factors not necessarily explicitly listed, again at least in part depending on the context.

[0013] The present disclosure is described below with reference to block diagrams and operational illustrations of methods and devices. It should be understood that each block of the block diagrams or operational illustrations, and combinations of blocks in the block diagrams or operational illustrations, may be implemented by analog or digital hardware and computer program instructions. These computer program instructions may be provided to a general purpose computer, a special purpose computer, an ASIC, or a processor of another programmable data processing device to execute the functions / operations specified in the block diagrams or one or more operational blocks, and to modify its functions as detailed herein. In some alternative implementations, the functions / operations noted in the blocks may be performed in a different order than that noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functions / operations involved.

[0014] For purposes of this disclosure, a non-transitory computer-readable medium (or computer-readable storage medium) stores computer data, which data may include computer program code (or computer-executable instructions) in machine-readable form that can be executed by a computer. By way of example and not limitation, a computer-readable medium may include a computer-readable storage medium for tangible or fixed storage of data, or a communication medium for the transitory interpretation of a signal containing the code. A computer-readable storage medium, as used herein, refers to a physical or tangible storage (as opposed to a signal), including, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for the tangible storage of information, such as computer-readable instructions, data structures, program modules, or other data. A computer-readable storage medium includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, optical storage, cloud storage, magnetic storage devices, or any other physical or material medium that can be used to tangibly store desired information or data or instructions and that can be accessed by a computer or processor.

[0015] For purposes of this disclosure, the term "server" should be understood to refer to a service point that provides processing, database, and communication facilities. By way of example and not limitation, the term "server" can refer to a single physical processor with associated communication and data storage and database facilities, or it can refer to a networked or clustered complex of processors and associated networks and storage devices, as well as operating software and one or more database systems and application software that support the services provided by the server. A cloud server is an example.

[0016] For purposes of this disclosure, a "network" should be understood to refer to a network that may couple devices such that communications may be exchanged, such as between a server and a client device or other type of device, including between wireless devices coupled via a wireless network. A network may also include mass storage, such as a network attached storage (NAS), a storage area network (SAN), a content delivery network (CDN), or other forms of computer or machine-readable media. A network may include the Internet, one or more local area networks (LANs), one or more wide area networks (WANs), wired type connections, wireless type connections, or any combination thereof. Similarly, sub-networks that may employ different architectures or conform to or be compatible with different protocols may interoperate within a larger network.

[0017] For purposes of this disclosure, a "wireless network" should be understood to combine client devices with a network. A wireless network may employ a standalone ad-hoc network, a mesh network, a Wireless LAN (WLAN) network, a cellular network, etc. A wireless network may further employ multiple network access technologies including Wi-Fi, Long Term Evolution (LTE), WLAN, wireless router mesh, or second, third, fourth, or fifth generation (2G, 3G, 4G, or 5G) cellular technologies, mobile edge computing (MEC), Bluetooth, 802.11 b / g / n, etc. A network access technology may enable wide area coverage for devices, such as client devices, with various degrees of mobility.

[0018] In short, a wireless network may include virtually any type of wireless communication mechanism by which signals may be communicated, such as between devices, such as client devices or computing devices, between networks, or within a network.

[0019] A computing device may be capable of transmitting or receiving signals, such as over a wired or wireless network, or may be capable of processing or storing signals, such as in memory, as physical memory states, and may therefore operate as a server. Thus, devices capable of operating as a server may include, by way of example, dedicated rack-mounted servers, desktop computers, laptop computers, set-top boxes, integrated devices that combine various features, such as features of two or more of the foregoing devices, and the like.

[0020] For purposes of this disclosure, a client (or user, entity, subscriber, or customer) device may include a computing device capable of transmitting or receiving signals, such as over a wired or wireless network. A client device may include, for example, a desktop computer or a portable device, such as a cellular telephone, a smartphone, a display pager, a radio frequency (RF) device, an infrared (IR) device, a Near Field Communication (NFC) device, a Personal Digital Assistant (PDA), a handheld computer, a tablet computer, a phablet, a laptop computer, a set-top box, a wearable computer, a smart watch, an integrated or distributed device that combines various features, such as features of the aforementioned devices.

[0021] Client devices may vary in terms of capabilities or features. The claimed subject matter is intended to encompass a wide range of potential variations, such as a web-enabled client device or such devices may include a high resolution screen (e.g., HD or 4K), one or more physical or virtual keyboards, mass storage, one or more accelerometers, one or more gyroscopes, global positioning system (GPS) or other location identification type capabilities, or displays with advanced functionality, such as, for example, touch-sensitive color 2D or 3D displays.

[0022] Certain embodiments and principles are discussed in more detail with reference to the drawings. According to some embodiments, as discussed herein, the disclosed system and method provide a computerized mechanism for the analysis of genomic data and functions arriving from genomic data. As discussed herein, an ensemble-based computerized framework can be configured for the analysis of samples using an ensemble algorithm trained with binary mutation data and hierarchical clustering data, which can enable the determination of drug efficacy and patient stratification. In some embodiments, as discussed below, such drug efficacy and patient stratification can be based on the determined scores corresponding to drug responsiveness for the patient's condition.

[0023] 1, a system 100 is depicted that, according to some embodiments, may include user equipment (UE) 102 (e.g., a user device as described above and discussed below in connection with FIG. 7), a network 104, a cloud system 106, a database 108, and a rating engine 200. It should be understood that while system 100 is depicted as including such components, this should not be construed as limiting as one of ordinary skill in the art would readily appreciate that a varying number of UEs, cloud systems, databases, and / or networks may be utilized without departing from the scope of the present disclosure; however, for purposes of illustration, system 100 will be discussed in connection with the exemplary depiction of FIG. 1.

[0024] According to some embodiments, the UE 102 can be any type of device, such as, but not limited to, a mobile phone, a tablet, a laptop, a sensor, a wearable device, a wearable camera, a wearable garment, a patch, an Internet of Things (IoT) device, an autonomous machine, and any other type of modem device. In some embodiments, the UE 102 can be a device associated with an individual (or set of individuals) for whom drug responsiveness monitoring is being provided.

[0025] In some embodiments, the UE 102 may be associated with peripheral devices (not shown) that may be connected to the UE 102 and may be any type of peripheral device, including but not limited to a wearable device (e.g., a smart watch), a printer, a speaker, a sensor, etc. In some embodiments, the peripheral device may be any type of device that can be connected to the UE 102 via any type of known or to become known pairing mechanism, including but not limited to WiFi, Bluetooth™, Bluetooth Low Energy (BLE), NFC, etc.

[0026] In some embodiments, the network 104 can be any type of network, such as, but not limited to, a wireless network (as discussed above), a cellular network, the Internet, etc. The network 104 facilitates connectivity of the components of the system 100, as illustrated in FIG.

[0027] According to some embodiments, cloud system 106 may be any type of cloud operating platform and / or network-based system upon which applications, operations, and / or other forms of network resources may be located. For example, system 106 may be a service and / or health provider and / or a service and / or network provider from which applications may be accessed, provided, or executed. For example, system 106 may represent a cloud-based architecture associated with a healthcare provider having associated network resources hosted on the Internet or a private network (e.g., network 104) that enables the patient monitoring and management (via engine 200) discussed herein.

[0028] In some embodiments, the cloud system 106 may be a private cloud where access is restricted by isolating the network, such as preventing external access, or by using encryption to limit access to only authorized users. Alternatively, the cloud system 106 may be a public cloud where access is broadly available via the Internet. A public cloud may not be secured or may include limited healthcare functionality.

[0029] In some embodiments, cloud system 106 may include servers and / or databases of information accessible via network 104. In some embodiments, database 108 of cloud system 106 may store data sets of data and metadata associated with users of UEs 102 and local and / or network information related to UEs 102, as well as services and applications provided by cloud system 106 and / or rating engine 200.

[0030] In some embodiments, for example, the cloud system 106 may provide a private / privately managed platform whereby the engine 200 discussed below accommodates the novel functionality that the system 106 enables, hosts, and provides for the network 104 and other devices / platforms operating thereon.

[0031] 5 and 6, in some embodiments, the exemplary computer-based system / platform, exemplary computer-based device, and / or exemplary computer-based component of the present disclosure may be configured to operate in a cloud computing / architecture 106, such as, but not limited to, infrastructure as a service (IaaS) 610, platform as a service (PaaS) 608, and / or software as a service (SaaS) 606, particularly using, but not limited to, a web browser, mobile app, thin client, terminal emulator, or other endpoint 604. Figures 5-6 illustrate schematic diagrams of non-limiting implementations of cloud computing / architectures in which the exemplary computer-based system for managed customization and control of network-hosted APIs of the present disclosure may be specifically configured to operate.

[0032] Returning to FIG. 1 , according to some embodiments, database 108 may correspond to data storage for a platform (e.g., a network-hosted platform such as cloud system 106, as discussed above) or multiple platforms. Database 108 may receive storage instructions / requests from engine 200 (and associated microservices), for example, which may be in any type of known or future format, such as, for example, standard query language (SQL). According to some embodiments, database 108 may correspond to any type of known or future type of storage, such as, but not limited to, a look-up table (LUT), a distributed ledger of a distributed network, etc.

[0033] Rating engine 200 may include components for the disclosed functions as discussed above and in further detail below. According to some embodiments, rating engine 200 may be a dedicated machine or processor and may be hosted by a device on network 104, in cloud system 106, and / or on UE 102. In some embodiments, engine 200 may be hosted by a server and / or set of servers associated with cloud system 106.

[0034] According to some embodiments, as discussed in more detail below, evaluation engine 200 may be configured to perform and / or control a number of services and / or microservices, each of which is configured to execute a number of workflows associated with performing the disclosed drug discovery and patient monitoring and management. Non-limiting examples of such workflows are provided below in conjunction with at least FIG.

[0035] According to some embodiments, as discussed above, the rating engine 200 may function as an application provided by the cloud system 106. In some embodiments, the engine 200 may function as an application installed on a server, network location, and / or other type of network resource associated with the system 106. In some embodiments, the rating engine 200 may function as an application operating via a traditional edge device (not shown) at a location associated with the system 100. In some embodiments, the engine 200 may function as an application installed and / or executed on the UE 102. In some embodiments, such an application may be a web-based application accessed by the UE 102 over the network 104 from the cloud system 106. In some embodiments, the engine 200 may be configured and / or installed as an extension script, program, or application (e.g., a plug-in or extension) to another application or program provided by the cloud system 106 and / or executing on the UE 102.

[0036] 2, according to some embodiments, the evaluation engine 200 includes an identification module 202, an analysis module 204, a determination module 206, and an output module 208. It should be understood that the engines and modules discussed herein are non-exhaustive, as additional or fewer engines and / or modules (or sub-modules) may be applicable to the system and method embodiments discussed. Further details of the operation, configuration, and functionality of engine 200 and each of its modules, and their roles within embodiments of the present disclosure, are discussed below.

[0037] Referring to Figure 3, process 300 provides a non-limiting exemplary embodiment for the disclosed framework. According to some embodiments, process 300 provides a computerized mechanism for analysis of genomic data and functions arriving from genomic data. As discussed herein, an ensemble-based computerized framework can be configured for analysis of samples using an ensemble algorithm (see Figure 4, discussed below) trained with binary mutation data and hierarchical clustering data, which can enable determination of drug efficacy and patent stratification.

[0038] According to some embodiments, steps 302-304 and 314 of process 300 may be performed by the identification module 202 of the rating engine 200, steps 306 and 310 may be performed by the analysis module 204, steps 308, 312 and 316 may be performed by the determination module 206, and step 318 may be performed by the output module 208.

[0039] According to some embodiments, the process 300 begins at step 302, where the engine 200 can identify a type of data and a corresponding data store (e.g., a database that houses such a particular type of data). For example, step 302 can include the engine 200 receiving a request from a user for identification of genomic data from public and / or private databases. In some embodiments, the datasets stored in such databases can include, but are not limited to, genomic data, mutation data, clinical outcome data (e.g., data representative of treatment outcomes), treatment data, drug response data (e.g., half-maximal inhibitory concentration (IC50) data), molecular data, patient identifiers, demographics, biometrics, etc., or some combination thereof. Thus, such a database can store multiple separate datasets each derived from genomic data of multiple different diseased cells, each dataset including multiple pathway element data.

[0040] In step 304, engine 200 may query an identification database to capture, retrieve, extract, or otherwise identify the specifically requested type (and / or amount) of data. Thus, in step 304, the requested data may be retrieved or captured from the database.

[0041] At step 306, engine 200 can analyze the captured data, and at step 308, a set of binary mutations can be determined based on the analysis of step 306. That is, in some embodiments, engine 200 can receive a separate data set queried from a database and identify determinant pathway elements (or features, used interchangeably) within the separate data set that are associated with a state (e.g., sensitivity or resistance) of a treatment parameter (e.g., treatment with a drug) of the diseased cell.

[0042] According to some embodiments, as discussed herein, pathway elements / features may be derived from binary values ​​representing mutation status as 0 or 1 to continuous values ​​based on the proportion of mutations that affect the same biological pathway. As an example, if four mutations functionally contribute to the same biological function and two of these are mutated in one tumor sample, a pathway feature may be generated with a score such as 2 / 4 or 0.5 for the pathway feature for that sample.

[0043] As a result, instead of many sparse data features that are typically mostly 0s and a few 1s, engine 200 can create information-rich features on a continuous gradient of values. Of note, with a realistic sample size (e.g., a sample size of 100), the occurrence of 5 mutations for a gene is too few for any currently known AI / ML algorithm to train on in cross-validation and split into a test set, so mutations occurring at an average frequency may not be possible for use in an AI / ML environment (note that this may change upon discovery of more independent and less training-dependent AI / ML). As discussed herein, pathway feature engineering expands the number of features with enough variation to be used by 10-50 fold. The result is many more features that are usable and of higher information content. Thus, reliance on mutations (rather than mutations alone) plus feature engineering (as provided herein) in an AI / ML environment enables the performance of predictions of drug response, a process that is not feasible without pathway information.

[0044] In some embodiments, pathway-based features can be created from pathways that are organized to contain consistent effectors when derived from a database, rather than clustering. For example, it may be common for some pathways commonly used in biological analysis to contain single active activators and repressors in the list of pathway genes, which is not suitable for AI / ML feature calculation, e.g., the derived pathways are inconsistent. For example, a score of 0.5 from such a pathway may mean two repressors, or two activators, or a mutation in one of each. The inconsistencies result in ambiguous features that are not necessarily associated with a single functional effect. Thus, pathways with consistent feature lists, such as the human REACTOME database (reactome.org), or pathways from experimental protein-protein interactions, among others, can be used as a source of lists for pathway feature calculation.

[0045] According to some embodiments, a score can be calculated from the original binary mutations to represent the degree of binary mutations in the pathway, which can be normalized by the length of the pathway. This divides the sum of binary mutations by the length of the available pathway to give a feature score in one sample. The available length is the length of the number of genes in the pathway after removing genes that were not available in the original data.

[0046] As a non-limiting example, in data from an assay where only 350 genes are measured, many pathways have genes that are not tested, and therefore are removed from the calculation due to undue indirect influence of the measurement technique on the feature weights. Thus, the disclosed methods can provide a mechanism for standardizing across different techniques (e.g., any technique or technical field where feature extraction and reduction can be utilized, e.g., to focus on actionable elements for AI / ML processing). In some embodiments, for example, pathway features are robust regardless of whether the genomic data is composed of more than 300, 500, or 20,000 genes.

[0047] Thus, the query in step 304 may include, for example, conditions corresponding to elements (e.g., pathway elements / features), conditions, and / or treatment parameters of / for pathway engineering.

[0048] In some embodiments, in addition to the feature engineering discussed above, steps 306-308 may additionally (in some embodiments, or alternatively) include analysis of a separate dataset associated with the diseased cells and determining pathway elements within the dataset that are adjusted to generate a modified dataset. Thus, in some embodiments, engine 200 may utilize the modified dataset to identify changes in the state of treatment parameters for the diseased cells. In some embodiments, if desired and / or required, engine 200 may pre-process the dataset (e.g., feature selection, data transformation metadata transformation, and / or splitting into training and validation datasets) prior to the analysis of step 306.

[0049] In some embodiments, as described above, such analysis and determination (e.g., steps 306-308) may be performed by engine 200 utilizing any type of known or to-be-known AI / ML algorithm or technique, including, but not limited to, computer vision, classifiers, feature vector analysis, decision trees, boosting, support-vector machines (SVMs), neural networks (e.g., convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc.), nearest neighbor algorithms, naive Bayes, bagging, random forests, logistic regression, etc.

[0050] Thus, as depicted in FIG. 4, an ensemble 400 of such AI / ML algorithms may be utilized. As provided herein, the combination and / or iteration of the ensemble 400 AI / ML may enable more robust and accurate analysis of genomic data, among other types of computational analysis discussed herein. In some embodiments, the ensemble model may include, but is not limited to, an area under the curve equation, an R matrix (e.g., a matrix of applied algorithms as depicted in the example of FIG. 4), and a p-value equation (e.g., a statistical and / or predictive measurement equation). Thus, in some embodiments, the ensemble may enable enhanced accuracy (or greater accuracy) than NIR (no information rate), as one of skill in the art will appreciate from the disclosure herein.

[0051] In some embodiments, and optionally in combination with any of the embodiments described above or below, the neural network technique may be one of, but not limited to, a feedforward neural network, a radial basis function network, a recurrent neural network, a convolutional network (e.g., U-net), or other suitable networks. In some embodiments, and optionally in combination with any of the embodiments described above or below, the neural network implementation may be performed as follows: a. Define neural network architectures / models. b. Transfer the input data to the neural network model. c. Incrementally train the model. d. Determine the accuracy for a particular number of time steps. e. Applying the trained model to process newly received input data. f. Optionally, and in parallel, continue training the trained model at a predetermined periodicity.

[0052] In some embodiments, and optionally in combination with any of the embodiments described above or below, the trained neural network model may specify the neural network by at least a neural network topology, a set of activation functions, and connection weights. For example, the topology of the neural network may include the configuration of nodes of the neural network and the connections between such nodes. In some embodiments, and optionally in combination with any of the embodiments described above or below, the trained neural network model may also be specified to include other parameters, including but not limited to bias values / functions and / or aggregation functions. For example, the activation functions of the nodes may be step functions, sine functions, continuous or piecewise linear functions, sigmoid functions, hyperbolic tangent functions, or other types of mathematical functions that represent thresholds at which the nodes are activated. In some embodiments, and optionally in combination with any of the embodiments described above or below, the aggregation functions may be mathematical functions that combine (e.g., sums, products, etc.) the input signals to the nodes. In some embodiments, and optionally in combination with any of the embodiments described above or below, the output of the aggregation function may be used as an input to the activation function. In some embodiments, and optionally in combination with any of the embodiments above or below, the bias may be a constant value or a function that may be used by the aggregation function and / or activation function to make a node more or less likely to be activated.

[0053] In some embodiments, the analysis and determination of steps 306-308 may include any non-linear ensemble 400 of AI / ML models, as depicted in Figure 4. For example, such non-linear combinations may include, but are not limited to, ensemble trees with bagging models, extreme gradient boosting, random forest models, SVM with Gaussian kernel models, elastic net logistic regression, power in at least the alpha band and power in the beta band, and / or any combination thereof. In some embodiments, the features of the analyzed data may include, but are not limited to, at least BSR, standard deviation of FM, SVDE, and FD.

[0054] According to some embodiments, based on the type of data (from step 302) retrieved (in step 304), a particular algorithm may be selected, which may be tuned and / or configured for a particular type, format, and / or amount of data and computational analysis based thereon, as discussed above. In some embodiments, an AI / ML model may be applied to determine the type, identity, and / or quantity of AI / ML algorithms to leverage as part of the ensemble 400. For example, for a particular BSR, deviation, and SVDE, algorithms X and Y may be selected to operate iteratively in a non-linear manner on the target data. Thus, different AI / ML algorithms may be specialized to recognize parts of a solution, and thus a combination approach may enable identification of the complete solution.

[0055] According to some embodiments, as a non-limiting example, a custom ensemble of algorithms (an example of which is visually depicted in FIG. 4) may be used with various types of data and model training targets. For example, the customized AI / ML-based ensemble 400 may be applied to, but is not limited to, i) clinical patient mutation data, which may include information corresponding to cluster derivation modules and pathway scores, as well as data corresponding to categorical training targets of patient response based on RECIST (Response evaluation criteria in solid tumors); ii) clinical patient mutation data from The Cancer Genome Atlas (TCGA) glioblastoma cohort, which may include information corresponding to cluster derivation modules and pathway scores, as well as data corresponding to training targets of patient response based on increased survival above or below median survival; iii) preclinical cellular mutation data using cells from the National Cancer Institute (NCI) NCI60 panel, without cluster derivation modules and pathway scores, and using regression training targets of IC50 of cells treated with a drug (e.g., in-log10 IC50(M) format); and iv) mutation data prepared from tumor samples, which may be scored in a binary manner (e.g., if a mutation occurs, the sample is given a 1, otherwise a 0). In some embodiments, mutation data with synonymous mutations may not be duplicated and may be performed accordingly (e.g., synonymous mutations for a patient, genome, and / or drug may be excluded and / or filtered from the analyzed data, for example).

[0056] Thus, in some embodiments, step 308 may include determining a score (or metric) for the data, thereby providing an indication as to the value and / or suitability of the binary mutation. As discussed above, such scoring may be based on and / or may include the feature engineering processes discussed above. As provided below, this scoring may be utilized to group or cluster the data for subsequent drug response analysis.

[0057] At step 310, engine 200 can analyze the determined data from step 308 (e.g., output from ensemble 400 analysis, e.g., determined binary mutations) and can determine a clustering of the data at step 312. According to some embodiments, such clustering can target (or group) one or more genes to identify features (e.g., pathway features) that correlate with responsiveness to a drug or multiple drugs and obtain hierarchical clustering data, as discussed below.

[0058] According to some embodiments, such analysis and clustering determination may be based on and / or may include the ensemble determination and application discussed above with respect to steps 306-308. For example, based on the type of binary mutation, feature engineering, and / or clustering criteria (e.g., the amount and / or quality of the clustered dataset), a specific AI / ML algorithm may be identified and executed in a non-linear and / or sequential manner.

[0059] As a non-limiting example, engine 200 may cluster data by grouping (or "cutting" or filtering) the data according to certain criteria (e.g., cluster height or desired number). For example, if the criteria corresponds to k clusters, then k=40, and of the 400 binary mutations analyzed, 40 clusters may be generated.

[0060] According to some embodiments, engine 200 may perform clustering in step 312 based on the set of features and / or biological pathways. In some embodiments, the data may be clustered using hierarchical clustering to create clusters or groups that allow for a reduction in sparsity of the data. In some embodiments, the data may be clustered based on their determined scores from step 308, as discussed above. For example, mutations within a range of scores may be clustered together. In another non-limiting example, mutations may be grouped when their corresponding scores are equal to a sum that is associated with and / or indicative of a particular biological pathway and / or mutations in such biological pathway.

[0061] According to some embodiments, in yet another non-limiting example, the initial data may be thinned or reduced using iterative feature reduction via an algorithm trained with hyper-parameters tuned to optimize performance according to a set of constraints, initially using 100-1100 distinct features in the sequence. In some embodiments, upon evaluation of this analysis, feature reduction may be implemented when the evaluation does not meet a performance threshold. In some embodiments, such feature reduction may be performed stepwise until the total features (e.g., clusters) are equal to or less than the feature threshold (e.g., feature reduction continues with stepwise reduction by 90% or 50% of the features until the number of usable features is a predefined percentage (e.g., 10%) of the total number of features (e.g., prior to reduction). Thus, such feature reduction may prevent overfitting, thereby allowing the ensemble mechanisms and methods discussed herein to be applied efficiently while ensuring that improved accuracy is demonstrated upon implementation.

[0062] Upon clustering the data in step 312, engine 200 can identify a target drug and target sample, as in step 314, thereby feeding the clustered / reduced feature set (from step 312) to an ensemble algorithm model in step 316. In some embodiments, the target sample corresponds to and / or is represented by the clustered data. In some embodiments, the target sample and / or drug may be identified from a database, as discussed above.

[0063] According to some embodiments, an ensemble algorithm model may be constructed and / or determined based on the type of clustering and / or the amount of clusters (e.g., which algorithms are part of the ensemble), which may be determined in a manner similar to that discussed above. In some embodiments, an ensemble compiled from a previous step in process 300 may be identified and selected for use.

[0064] Thus, in step 316, engine 200 can identify and / or determine an ensemble, whereby, upon execution, a determination regarding the responsiveness of a target drug can be made. Thus, in step 316, engine 200 can determine how the drug acts (e.g., how effective the drug is) on a condition (e.g., represented via a target sample). Such data can then be stored in a database, as in step 318. In some embodiments, such responsiveness data can be recursively fed back into the ensemble (or at least a portion of the algorithm used in processing process 300) to effect further training of the model implemented therein.

[0065] According to some embodiments, the input of a list of algorithms "R", e.g. "algo_namel", "algo_name2", with possible iterations (e.g., ensembles can have the same algorithm tuned differently and implemented differently, both possibly contributing to the ensemble), can be utilized as information when generating the ensembles discussed above, and / or can be associated with stored information to provide instructions regarding the analysis that led to the responsiveness determination. In some embodiments, with regard to training and / or retraining (e.g., via step 318), some customized steps can be performed to perform different data splits on the training data and 25-50% (optional) of the training data, so that the ensemble does not overfit (as discussed above). In some embodiments, different algorithms can be trained and saved in a "list" object in R, and then predictions using the models can be saved, which can be added to the patient samples and the data with mutations and clusters. Thus, upon storage and training of the ensemble, the “final” algorithm selected for step 316 can be used to perform AI / ML training on a new training set that includes the stored predictions in step 318.

[0066] 7 is a schematic diagram illustrating a client device showing an example embodiment of a client device that may be used within the present disclosure. The client device 700 may include more or fewer components than those shown in FIG. 7. However, the components shown are sufficient to disclose an example embodiment for implementing the present disclosure. The client device 700 may represent, for example, at least the UE 102 discussed above with respect to FIG.

[0067] As shown in the figure, in some embodiments, the client device 700 includes a processing unit (CPU) 722 in communication with a mass memory 730 via a bus 724. The client device 700 also includes a power supply 726, one or more network interfaces 750, an audio interface 752, a display 754, a keypad 756, an illuminator 758, an input / output interface 760, a haptic interface 762, an optional Global Positioning System (GPS) receiver 764, and a camera or other optical, thermal, or electromagnetic sensor 766. The device 700 may include one camera / sensor 766, or multiple cameras / sensors 766, as would be understood by one of ordinary skill in the art. The power supply 726 provides power to the client device 700.

[0068] The client device 700 may optionally communicate with a base station (not shown) or directly with another computing device. In some embodiments, the network interface 750 is sometimes known as a transceiver, a transceiver device, or a network interface card (NIC).

[0069] The audio interface 752, in some embodiments, is arranged to generate and receive audio signals, such as the sound of a human voice. The display 754 may be a liquid crystal display (LCD), gas plasma, light emitting diode (LED), or any other type of display used with a computing device. The display 754 may also include a touch-sensitive screen arranged to accept input from an object, such as a stylus or a finger from a human hand.

[0070] Keypad 756 may include any input device arranged to accept input from a user. Illuminator 758 may provide a status indication and / or provide light.

[0071] The client device 700 also includes an input / output interface 760 for communicating with the outside world, which in some embodiments may utilize one or more communication technologies such as USB, infrared, Bluetooth™, etc. A haptic interface 762 is arranged to provide haptic feedback to a user of the client device.

[0072] The optional GPS transceiver 764 can determine the physical coordinates of the client device 700 on the surface of the Earth, which typically outputs a location as latitude and longitude values. The GPS transceiver 764 can also employ other geographic positioning mechanisms, including but not limited to triangulation, assisted GPS (AGPS), E-OTD, CI, SAI, ETA, BSS, etc., to further determine the physical location of the client device 700 on the surface of the Earth. However, in one embodiment, the client device may provide other information through other components that can be employed to determine the device's physical location, including, for example, a MAC address, an Internet Protocol (IP) address, etc.

[0073] Mass memory 730 includes RAM 732, ROM 734, and other storage means. Mass memory 730 illustrates another embodiment of a computer storage medium for storing information such as computer-readable instructions, data structures, program modules, or other data. Mass memory 730 stores a basic input / output system ("basic input / output system, BIOS") 740 for controlling low-level operation of client device 700. Mass memory also stores operating system 741 for controlling operation of client device 700.

[0074] Memory 730 further includes one or more data stores that may be utilized by client device 700 to store, among other things, applications 742 and / or other information or data. For example, a data store may be used to store information describing various capabilities of client device 700. The information may then be provided to another device based on any of a variety of events, including being sent as part of a header during a communication (e.g., an index file for an HLS stream), sent on request, etc. At least a portion of the capability information may also be stored on a disk drive or other storage medium (not shown) within client device 700.

[0075] Applications 742 may include computer-executable instructions that, when executed by client device 700, transmit, receive, and / or otherwise process audio, video, images, and enable electronic communication with a server and / or other users of other client devices. Applications 742 may further include clients configured to transmit, receive, and / or otherwise process games, goods / services, and / or other forms of data, messages, and content hosted and offered by a platform associated with engine 200 and its affiliates.

[0076] As used herein, the terms "computer engine" and "engine" identify at least one software component and / or a combination of at least one software component and at least one hardware component designed / programmed / configured to manage / control other software and / or hardware components (libraries, software development kits (SDKs), objects, etc.).

[0077] Examples of hardware elements may include a processor, a microprocessor, a circuit, a circuit element (e.g., a transistor, a resistor, a capacitor, an inductor, etc.), an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a digital signal processor (DSP), a field programmable gate array (FPGA), a logic gate, a register, a semiconductor device, a chip, a microchip, a chipset, etc. In some embodiments, the one or more processors may be implemented as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processor, an x86 instruction set compatible processor, a multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, the one or more processors may be a dual-core processor, a dual-core mobile processor, etc.

[0078] As used herein, computer-related systems, computer systems, and systems include any combination of hardware and software. Examples of software may include software components, programs, applications, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computer code, computer code segments, words, values, symbols, or any combination thereof. Determining whether an embodiment is implemented using hardware and / or software elements may vary according to any number of factors, such as desired computation rates, power levels, thermal tolerances, processing cycle budgets, input data rates, output data rates, memory resources, data bus speeds, and other design or performance constraints.

[0079] For purposes of this disclosure, a module is a software, hardware, or firmware (or combination thereof) system, process, or functionality, or components thereof, that performs or facilitates a process, feature, and / or function described herein (with or without human interaction or augmentation). A module may include sub-modules. Software components of a module may be stored on a computer-readable medium for execution by a processor. A module may be integral to one or more servers, or may be loaded and executed by one or more servers. One or more modules may be grouped into an engine or application.

[0080] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium that represent various logic within a processor, which, when read by a machine, causes the machine to fabricate logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on tangible machine-readable media and provided to various customers or manufacturing facilities for loading into fabrication machines that create the logic or processors. Of note, the various embodiments described herein may, of course, be implemented using any suitable hardware and / or computing software language (e.g., C++, Objective-C, Swift, Java, JavaScript, Python, Perl, QT, etc.).

[0081] For example, exemplary software specifically programmed according to one or more principles of the present disclosure may be downloadable from a network, such as a website, as a stand-alone product or as an add-in package for installation into an existing software application. For example, exemplary software specifically programmed according to one or more principles of the present disclosure may also be available as a client-server software application or as a web-enabled software application. For example, exemplary software specifically programmed according to one or more principles of the present disclosure may also be embodied as a software package installed on a hardware device.

[0082] For the purposes of this disclosure, the term "user" or "patient" should be understood to refer to a user of an application as described herein and / or a consumer of data provided by a data provider. By way of example and not limitation, the term "user" or "patient" can refer to a person receiving data over the Internet in a browser session or data provided by a service provider, or can refer to an automated software application that receives data and stores or processes the data. Those skilled in the art will recognize that the methods and systems of the present disclosure can be implemented in many ways and therefore should not be limited by the exemplary embodiments and examples described above. In other words, in various combinations of hardware and software or firmware, functional elements implemented by single or multiple components, as well as individual functions, can be distributed among software applications at either the client level or the server level, or both. In this regard, any number of features of the different embodiments described herein can be combined in a single or multiple embodiments, and alternative embodiments having less than or more than all of the features described herein are possible.

[0083] Functions may also be distributed, in whole or in part, among multiple components in ways now known or to become known. Thus, a myriad of software / hardware / firmware combinations are possible in achieving the functions, features, interfaces, and preferences described herein. Moreover, the scope of the present disclosure encompasses conventionally known methods for implementing the described features and functions and interfaces, as well as variations and modifications thereof that may be made to the hardware or software or firmware components described herein, as now and hereafter understood by those skilled in the art.

[0084] Furthermore, the method embodiments presented and described as flow charts in this disclosure are provided as examples to provide a more complete understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are performed independently.

[0085] Although various embodiments have been described for purposes of this disclosure, such embodiments should not be construed as limiting the teachings of this disclosure to those embodiments. Various changes and modifications may be made to the elements and operations described above to obtain results that remain within the scope of the systems and processes described in this disclosure.

Claims

1. 1. A method comprising: identifying, by the device, genomic mutation data; analyzing, by the device, the genomic mutation data through an ensemble model; determining, by the device, a set of binary mutations based on the analysis; and clustering, by the device, the set of binary mutations; analyzing, by the device, the clustered binary mutations based on information related to the identified drug; determining, by the device, a responsiveness of each cluster to the identified drug based on an analysis of the clustered binary mutations; storing, by the device, information indicative of the determined responsiveness in a database.

2. determining a set of algorithms based on said analysis of said genomic mutation data; The method of claim 1 , further comprising: compiling the ensemble model based on the determined set of algorithms.

3. The method of claim 2 , wherein the ensemble model comprises a non-linear implementation of the determined set of algorithms.

4. The method of claim 2 , wherein the information stored in the database further indicates the set of algorithms.

5. The method of claim 1 , wherein the ensemble is trained based on the determined responsiveness.

6. The method of claim 1 , wherein the ensemble comprises a set of algorithms based on information including an area under the curve formula, an R matrix, and a p-value formula.

7. 2. The method of claim 1, wherein the determined responsiveness comprises a score value, the value indicating how the drug affected binary mutations of a particular cluster.

8. identifying a type of each of said binary mutations; The method of claim 1 , further comprising: performing the clustering of the binary mutations based on the identified types, each cluster corresponding to a particular type of binary mutation.

9. 2. The method of claim 1, further comprising performing feature reduction on a set of binary mutation data iteratively, as a stepwise function, until a set of features for the clustering meets a threshold, and the clustering is based on the feature reduction.

10. The method of claim 9 , wherein the feature reduction is based on a set of constraints.

11. Identifying a type of data and a corresponding database; Querying the database; and The method of claim 1 , further comprising: searching the genomic mutation data based on the query, wherein the identification of the genomic mutation data is based on the searching.

12. A device, comprising: a processor, the processor comprising: Identifying genomic mutation data; Analyzing the genomic mutation data through an ensemble model; determining a set of binary mutations based on said analysis; clustering the set of binary mutations; analyzing the clustered binary mutations based on the identified drug-related information; determining the responsiveness of each cluster to the identified drug based on an analysis of the clustered binary mutations; The device is configured to store information indicative of the determined responsiveness in a database.

13. The processor, determining a set of algorithms based on said analysis of said genomic mutation data; 13. The device of claim 12, further configured to compile the ensemble model based on the determined set of algorithms, the ensemble model including a non-linear implementation of the determined set of algorithms.

14. The processor, identifying a type of each of said binary mutations; The device of claim 12 , further configured to perform the clustering of the binary mutations based on the identified types, each cluster corresponding to a particular type of binary mutation.

15. The processor, 13. The device of claim 12, further configured to perform feature reduction on a set of binary mutation data iteratively, as a stepwise function, until a set of features for the clustering satisfies a threshold, the clustering being based on the feature reduction, the feature reduction being based on a set of constraints.

16. The processor, Identifying the type of data and the corresponding database; Querying the database; The device of claim 12 , further configured to search the genomic mutation data based on the query, and wherein the identification of the genomic mutation data is based on the search.

17. A non-transitory computer-readable storage medium tangibly encoded thereon having computer-executable instructions that, when executed by a device, identifying, by said device, genomic mutation data; analyzing, by the device, the genomic mutation data through an ensemble model; determining, by the device, a set of binary mutations based on the analysis; and clustering, by the device, the set of binary mutations; analyzing, by the device, the clustered binary mutations based on information related to the identified drug; determining, by the device, a responsiveness of each cluster to the identified drug based on an analysis of the clustered binary mutations; storing, by the device, information indicative of the determined responsiveness in a database.

18. determining a set of algorithms based on said analysis of said genomic mutation data; 20. The non-transitory computer-readable storage medium of claim 17, further comprising: compiling the ensemble model based on the determined set of algorithms, the ensemble model comprising a non-linear implementation of the determined set of algorithms.

19. identifying a type of each of said binary mutations; 20. The non-transitory computer-readable storage medium of claim 17, further comprising: performing the clustering of the binary mutations based on the identified types, each cluster corresponding to a particular type of binary mutation.

20. 20. The non-transitory computer-readable storage medium of claim 17, further comprising: performing feature reduction on a set of binary mutation data iteratively, as a stepwise function, until a set of features for the clustering meets a threshold, the clustering being based on the feature reduction, the feature reduction being based on a set of constraints.

Images