Future biological state prediction using generative artificial intelligence

Abstract

Described herein are techniques for generating a prediction of a future biological state of a biological system. In some embodiments, a first biological state and a second biological state of a biological system may be determined. The first biological state may describe the biological system at a first time and the second biological state may describe the biological system at a second time. Using one or more machine learning models, a first embedding representing the first biological state at the first time and a second embedding representing the second biological state at the second time may be generated. Using the machine learning models, a future embedding representing a future biological state of the biological system at a future time may be predicted. The machine learning models may be used to generate a prediction of the future biological state at the future time based on the future embedding.

Description

FUTURE BIOLOGICAL STATE PREDICTION USING GENERATIVEARTIFICIAL INTELLIGENCECROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application Nos. 63 / 539,082, filed September 18, 2023; 63 / 539,083, filed September 18, 2023; 63 / 539,084, filed September 18, 2023; and 63 / 602,039, filed November 22, 2023, each of which is incorporated herein by reference in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates generally to methods and systems for predicting a full biological state of a biological system from a partial biological state of the biological system. The present disclosure also relates generally to methods and systems for generating tokenized representations of biological states for input to machine learning models. The present disclosure yet also relates to methods and system for determining a future biological state of a biological system.BACKGROUND

[0003] Retrospective data on biological systems often comprise incomplete datasets with partial information. For example, microarrays provide RNA expression data on a limited set of RNA species as compared to more modern methods based on high-throughput sequencing- by-synthesis (NGS). Artificial intelligence (Al) approaches generally benefit from larger training datasets and achieve higher accuracy with larger training sets. However, the limited / incomplete datasets hinder such Al approaches from to fully utilizing retrospective incomplete biological datasets. Additionally, even with modem characterization and techniques, there is often a cost-benefit tradeoff with diminish returns. For example, the same blood sample could be sequenced given a total of 10 million reads, 50 million reads (typical in 2023), or 500 million reads. Additional NGS reads linearly increase sequencing cost, but provide marginally higher accuracy on low-expressing genes by reducing Poisson noise from sampling. Thus, the ability to "outpaint" incomplete datasets can both allow use of large amounts of older data for training Al for biology, and enable cost effective collection of new data.

[0004] Biology Al applications can be broadly classified into (1) drug design and / or evaluation, (2) electronic medical record-based analysis of patients, and (3) disease state analysis including therapy selection and diagnostics. The first class primarily considers representations of static chemical molecules. The second class primarily consider natural language and tabular patient data. The third class considers data from diagnostic images and biomarkers.

[0005] Existing disease state prediction models based on patient biomarkers and radiological images over-simplify the disease diagnosis and characterization process by using current and / or prior biological state information to directly classify the disease state. These techniques, however, can result in inaccurate prediction of disease state. Thus, what is needed are techniques that can accurately predict future disease state using the available data.BRIEF SUMMARY

[0006] Described herein are techniques using current biological states to predict future biological states of the subjects using machine learning. In some embodiments, the future biological states can be used to predict a future disease or health state of the subject using disease classification Al. The techniques described herein separate the biological state evolution prediction problem from the disease state classification problems, and enables more accurate predictions about future disease states. Furthermore, this separation of future biological state prediction from future disease state classification better enables transfer learning across different species and different individuals. This is because the underlying biological networks for different biological systems can be more similar to each other than specific disease phenotypes across different biological species.

[0007] Also described are techniques that aim to solve the issues faced by current methods of converting biological states data to machine learning model inputs. In particular, current methods cannot tokenize the data into consecutive categorical sequences, which are particularly important for modern machine learning models. Therefore, the techniques described herein overcome these issues by generating tokenized representations of biological states, which can then be used as the machine learning model inputs.

[0008] Techniques for generating a prediction of a future biological state of a biological system are described. In some embodiments, a first biological state and a second biological state of a biological system may be determined. The first biological state may describe the biological system at a first time and the second biological state may describe the biological system at a second time. Using one or more machine learning models, a first embeddingrepresenting the first biological state at the first time and a second embedding representing the second biological state at the second time may be generated. Using the one or more machine learning models, a future embedding representing a future biological state of the biological system at a future time may be predicted. The one or more machine learning models may further be used to generate a prediction of the future biological state describing the biological system at the future time based on the future embedding.

[0009] Techniques for generating a tokenized representation of a biological state for a machine learning model are described. In some embodiments, a biological state describing a biological system may be obtained. The biological state may comprise a plurality of biological state features each having an expression value. For each of a subset of the plurality of biological state features: a logarithmic expression value of the biological state feature may be computed based on the expression value and the logarithmic expression value may be discretized into a discretized expression value. A tokenized representation of the biological state comprising the plurality of biological state features structured using an ordering function may be generated based on the discretized expression value of each of the subset of the plurality of biological state features.

[0010] Techniques for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state are described. In some embodiments, a plurality of biological state features of the biological state may be input into one or more machine learning models to generate an embedding representing the biological state. The plurality of biological state features may comprise the one or more known biological state features and the one or more unknown biological state features. Using the one or more machine learning models, at least one of the one or more unknown biological state features may be predicted based on the embedding.

[0011] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.

[0012] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer readable storage medium containing instructions which, when executed on the one or moredata processors, cause the one or more data processors to perform part or all of one or more methods and / or part or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer-program product tangibly embodied in a non- transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform part or all of one or more methods and / or part or all of one or more processes disclosed herein.

[0013] The terms and expressions which have been employed are used as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding any equivalents of the biological state features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention claimed. Thus, it should be understood that although the present invention as claimed has been specifically disclosed by embodiments and optional biological state features, modification and variation of the concepts herein disclosed can be resorted to by those skilled in the art, and that such modifications and variations are considered to be within the scope of this invention as defined by the appended claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Various aspects of the disclosed methods, devices, and systems are set forth with particularity in the appended claims. A better understanding of the biological state features and advantages of the disclosed methods, devices, and systems will be obtained by reference to the following detailed description of illustrative embodiments and the accompanying drawings, of which:

[0015] FIG. 1 illustrates an example system for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state, in accordance with various embodiments.

[0016] FIGS. 2A-2B illustrate example biological states of biological systems, in accordance with various embodiments.

[0017] FIG. 3 illustrates an example process for using a machine learning model to predict one or more unknown biological state features from one or more known biological state features, in accordance with various embodiments.

[0018] FIG. 4 illustrates an example of transforming measured biological states into training data, in accordance with various embodiments.

[0019] FIG. 5 illustrates an example training process for training a machine learning model to predict one or more unknown biological state features from one or more known biological state features, in accordance with various embodiments.

[0020] FIGS. 6A-6B illustrate examples of a process for generating a gene regulatory network, in accordance with various embodiments.

[0021] FIG. 7 illustrates an example of a masked gene expression matrix and a predicted gene expression matrix, in accordance with various embodiments.

[0022] FIG. 8 illustrates a flowchart of an example method for predicting one or more unknown biological state features of a biological state from one or more known biological state features, in accordance with various embodiments.

[0023] FIG. 9 illustrates a flowchart of an example method for training a machine learning model to predict one or more unknown biological state features from one or more known biological state features, in accordance with various embodiments.

[0024] FIG. 10 illustrates an example system for generating a tokenized representation of a biological state for a machine learning model, in accordance with various embodiments.

[0025] FIG. 11 illustrates an example process for computing an average biological state for a plurality of biological systems, in accordance with various embodiments.

[0026] FIG. 12 illustrates an example process for removing one or more biological state features from a plurality of biological state features representing the biological state, in accordance with various embodiments.

[0027] FIG. 13 illustrates an example process for generating a tokenized representation of a biological state, in accordance with various embodiments.

[0028] FIG. 14 illustrates a flowchart of an example method for generating a tokenized representation of a biological state, in accordance with various embodiments.

[0029] FIG. 15 illustrates an example system for generating a prediction of a future biological state of a biological system, in accordance with various embodiments.

[0030] FIG. 16A illustrates an example process for using a machine learning model to generate a prediction of a future biological state, in accordance with various embodiments.

[0031] FIG. 16B illustrates an example process for determining a disease state of the biological system based on the prediction of the future biological state, in accordance with various embodiments.

[0032] FIG. 17 illustrates an example of a training process for training a machine learning model to predict a future biological state of a biological system, in accordance with various embodiments.

[0033] FIG. 18 illustrates a flowchart of an example method for generating a tokenized representation of a biological state, in accordance with various embodiments.

[0034] FIGS. 19A-19D illustrate non-limiting exemplary data showing an overview of longitudinal rat RNAseq experiments.

[0035] FIGS. 20A-20H illustrate non-limiting exemplary data showing control experiments the impact of daily blood draws from rat subjects.

[0036] FIGS. 21A-21F illustrate non-limiting exemplary data showing rat responses to tetracyline and other drugs.

[0037] FIGS. 22A-22H illustrate non-limiting exemplary data showing recovery trajectories in RNAseq PCA space.

[0038] FIGS. 23A-23E illustrate non-limiting exemplary data showing an expansion study on individual outcome variability for rats dosed with valproate.

[0039] FIG. 24 illustrates an example computer system used to implement some or all of the techniques described herein.DETAILED DESCRIPTION

[0040] In some aspects, described herein are systems, methods, and programming for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state. Described herein are also systems, methods, and programming for generating a tokenized representation of a biological state for a machine learning model. Described herein further include systems, methods, and programming for generating a prediction of a future biological state of a biological system.

[0041] In one aspect, described herein are techniques for generating a prediction of a future biological state of a biological system. In some embodiments, a first biological state and a second biological state of a biological system may be determined. The first biological state may describe the biological system at a first time and the second biological state may describe the biological system at a second time. Using one or more machine learning models, a first embedding representing the first biological state at the first time and a second embedding representing the second biological state at the second time may be generated.Using the machine learning models, a future embedding representing a future biological state of the biological system at a future time may be predicted. The machine learning modelsmay be used to generate a prediction of the future biological state at the future time based on the future embedding.

[0042] In another aspect, described herein are techniques for generating a tokenized representation of a biological state for a machine learning model. In some embodiments, a biological state describing a biological system may be obtained. The biological state may comprise a plurality of biological state features each having an expression value. For each of a subset of the plurality of biological state features: a logarithmic expression value of the biological state feature may be computed based on the expression value and the logarithmic expression value may be discretized into a discretized expression value. A tokenized representation of the biological state comprising the plurality of biological state features structured using an ordering function may be generated based on the discretized expression value of each of the subset of the plurality of biological state features.

[0043] In yet another aspect, described herein are techniques for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state. In some embodiments, a plurality of biological state features of the biological state may be input into one or more machine learning models to generate an embedding representing the biological state. The plurality of biological state features may comprise the one or more known biological state features and the one or more unknown biological state features. Using the one or more machine learning models, at least one of the one or more unknown biological state features may be predicted based on the embedding.I. Example System for Full Biological State Prediction

[0044] FIG. 1 illustrates an example system for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state, in accordance with various embodiments. System 100 may include a computing system 102, user devices 130-1 to 130-N (also referred to collectively as “user devices 130” and individually as “user device 130”), databases 140 (e.g., biological state database 142, training data database 144, model database 146), or other components. In some embodiments, components of system 100 may communicate with one another using network 150, such as the Internet.

[0045] User devices 130 may communicate with one or more components of system 100 via network 150 and / or via a direct connection. User devices 130 may be a computing device configured to interface with various components of system 100 to control one or more tasks,cause one or more actions to be performed, or effectuate other operations. For example, user device 130 may be configured to receive and display an image of one or more cells. Example computing devices that user devices 130 may correspond to include, but are not limited to, which is not to imply that other listings are limiting, desktop computers, servers, mobile computers, smart devices, wearable devices, cloud computing platforms, or other client devices. In some embodiments, each user device 130 may include one or more processors, memory, communications components, display components, audio capture / output devices, image capture components, or other components, or combinations thereof. Each user device 130 may include any type of wearable device, mobile terminal, fixed terminal, or other device.

[0046] It should be noted that while one or more operations are described herein as being performed by particular components of computing system 102, those operations may, in some embodiments, be performed by other components of computing system 102 or other components of system 100. As an example, while one or more operations are described herein as being performed by components of computing system 102, those operations may, in some embodiments, be performed by aspects of user devices 130. It should also be noted that, although some embodiments are described herein with respect to machine learning models, other prediction models (e.g., statistical models or other analytics models) may be used in lieu of or in addition to machine learning models (e.g., a statistical model replacing a machine-learning model and a non-statistical model replacing a non-machine-leaming model in one or more embodiments). Still further, although a single instance of computing system 102 is depicted within system 100, additional instances of computing system 102 may be included (e.g., computing system 102 may comprise a distributed computing system).

[0047] Computing system 102 may include a biological state prediction subsystem 110, a training subsystem 112, or other components. Biological state prediction subsystem 110 and training subsystem 112 may be configured to communicate with one another, one or more other devices, systems, and / or servers, using network 150 (e.g., the Internet, an Intranet). System 100 may also include one or more databases 140 (e.g., biological state database 142, training data database 144, model database 146) used to store data for training machine learning models, storing machine learning models, or storing other data used by one or more components of system 100. This disclosure anticipates the use of one or more of each type of system and component thereof without necessarily deviating from the teachings of thisdisclosure. Although not illustrated, other intermediary devices (e.g., data stores of a server connected to computing system 102) can also be used.

[0048] The components of system 100 of FIG. 1 can be used in a variety of contexts where scanning and evaluating digital pathology images, such as whole slide images, or biological samples, are essential components of the work. As an example, system 100 can be associated with a clinical environment where a user is evaluating cells for drug discovery and disease monitoring. The user can review an image using user device 130 prior to providing the image to computing system 102. The user can provide additional information to computing system 102 that can be used to guide or direct the analysis of the image. Persons of ordinary skill in the art will recognize that, in some examples, no user review of the image may be needed.

[0049] In some embodiments, biological state prediction subsystem 110 may be configured to predict one or more unknown biological state features of a biological state from one or more known biological state features of the biological state. For example, biological state prediction subsystem 110 may be configured to obtain a biological state describing the biological system from biological state database 142. The biological state may be represented using a plurality of biological state features. In some embodiments, the biological state features may include one or more known biological state features, one or more unknown biological state features, or combinations thereof. In one or more examples, the known biological state features may comprise biological state features whose expression value is determined empirically (e.g., from gene expression data). In one or more examples, the known biological state features may comprise biological state features whose expression value is predicted (e.g., predicted biological state features) using one or more machine learning models.

[0050] As described herein, a biological system refers to a subject from which a biological sample is obtained (e.g., tissue sample, blood sample, DNA samples, etc.). The subject may be a human-subject or an animal- subject. In some examples, the biological state features representing the biological state of the biological system may be determined based on the biological sample. An example of a biological state is described in greater detail below with respect to FIGS. 2A-2B.

[0051] FIGS. 2A-2B illustrate example biological states of biological systems, in accordance with various embodiments. As seen in FIG. 2A, biological state 200 may be determined using a biological sample obtained from a biological system (e.g., a subject). Forexample, the biological sample may be a blood sample from which RNAseq data is extracted. In some embodiments, biological state 200 may be represented using a feature vector 201. For example, biological state 200 may be an n-dimensional feature vector. Biological state 200 may be represented using a plurality of biological state features. In some embodiments, the biological state features may comprise biological markers. For example, the biological markers may include genomic data, proteomic data, transcriptomic data, physiological data, phenotypical data, or other biological markers, or combinations thereof.

[0052] In some embodiments, a biological state, such as biological state 200 may also be referred to herein interchangeably as a “biostate.” In some examples, the biological state features, which may include known biological state features and / or unknown biological state features, may be grouped into one or more sets of biological state features. For example, biological state 200 may include a first set of biological state features 202 corresponding to RNA expression data, a second set of biological state features 204 corresponding to protein expression data, and / or a third set of biological state features 206 corresponding to methylation markers. In some examples, biological state 200 may further include a fourth set of biological state features 208 corresponding to phenotype data. Persons of ordinary skill in the art will recognize that a biological state may include additional data or may include less data, depending on the use case, available sample, or other considerations.

[0053] In some examples, each biological state feature of first set of biological state features 202 may comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature (e.g., Gene 1, Gene 2, Gen 3, ... Gene and an expression value indicating a level with which that gene is expressed within a biological sample of the biological system. In the example of FIG. 2A, the expression level is represented using a color bar. Genes that have a greater expression within the biological sample may be represented with the color “red” whereas genes that are not expressed as much within the biological sample may be represented with the color “green.” As an illustrative example, in first set of biological state features 202, Gene 3 has a greater expression level (as indicated by the redder coloring) than Gene 1 (as indicated by the greener coloring). Persons of ordinary skill in the art will recognize that the coloring scheme used is merely exemplary.

[0054] In one or more examples, each biological state feature of second set of biological state features 204 may comprise: a label indicating a corresponding protein from a set of proteins represented by the biological state feature (e.g., Protein 1, Protein 2, Protein 3, ...Protein M) and an expression value indicating a level with which that protein is expressed within the sample of the biological system. As with biological state features from first set of biological state features 202, proteins that have a greater expression within the biological sample may be represented with the color “red” whereas proteins that are not expressed as much within the biological sample may be represented with the color “green.” As an illustrative example, in second set of biological state features 204, Protein M has a greater expression level (as indicated by the redder coloring) than Protein 1 (as indicated by the greener coloring).

[0055] In one or more examples, each biological state feature of third set of biological state features 206 may comprise: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature (e.g., Marker 1, Marker 2, Marker 3, ... Marker K) and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system. As with biological state features from first set of biological state features 202 and second set of biological state features 204, methylation markers that have a greater expression within the biological sample may be represented with the color “red” whereas methylation markers that are not expressed as much within the biological sample may be represented with the color “green.” As an illustrative example, in third set of biological state features 206, methylation marker 1 has a greater expression level (as indicated by the redder coloring) than methylation markers K (as indicated by the greener coloring).

[0056] In one or more examples, each biological state feature of fourth set of biological state features 208 comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature (e.g., Phenotype ID 1, Phenotype ID 2, Phenotype ID 3, ... Phenotype ID Z). The biological state features corresponding to the phenotypes may include expression values and / or confidence values. As with biological state features from first set of biological state features 202, second set of biological state features 204, and third set of biological state features 204, Phenotype IDs that have a greater expression / confidence within the biological sample may be represented with the color “red” whereas Phenotype IDs that are not expressed as much within the biological sample / have less confidence may be represented with the color “green.” As an illustrative example, in fourth set of biological state features 208, Phenotype ID (Z-2) has a greater expression level (as indicated by the redder coloring) than Phenotype ID 2 (as indicated by the greener coloring).

[0057] In some examples, for instance biological state feature vector 201 may include the biological state features stored as a numerical value (e.g., a number between 0.0 and approximately 1.0, a number between 0.0 and 100, or using another numbering convention). For example, first set of biological state features 202 may be represented using variables X0- XN in biological state feature vector 201, second set of biological state features 204 may be represented using variables X(N+1)-X(N+M), third set of biological state features 206 may be represented using variables X(M+1)-X(M+K), and fourth set of biological state features 208 may be represented using variables X(K+1)-X(K+Z). A total number of biological state features represented by biological state feature vector 201, and thus a total number of biological markers included in biological state 200 may be 103or more, 105or more, 107or more, or other values.

[0058] In some embodiments, one or more of the biological markers of biological state 200 may not be expressed within the biological sample or one or more biological markers may not exceed a threshold level of expression within the sample such that those biological markers are detectable. For example, the expression level, as indicated by a color within biological state 200, may be represented via a numerical value. If the numerical value associated with a particular biological state feature, and thus biological marker, is unknown, then an accurate assessment of a disease state of a corresponding biological system may not be able to be predicted. This can severely limit the predictive abilities for identifying potential health risks and drug interactions. Thus, techniques are needed to “fill-in” the unknown biological state features by predicting their expression values.

[0059] In some embodiments, biological state prediction subsystem 110 may be configured to generate a tokenized representation of the partial biological state. The tokenized representation may input to the one or more machine learning models. The tokenized representation may enable faster and more accurate model training biological state prediction.

[0060] In some embodiments, the plurality of biological state features may each having a label (e.g., biological state features X0-X(K+Z). In one or more examples, one or more of the biological state features may have an unknown expression value. In one or more examples, one or more of the biological state features may have a known expression value. The known expression value may be determined empirically (e.g., by measuring how much a particular biological marker is expressed within the biological sample) or predicted. The tokenized representation may be structured such that the biological state features are orderedbased on the expression value of each of the plurality of biological state features. In contrast, the partial biological state and the full biological state may be structured in order of biological state feature (e.g., biological state feature 1, biological state feature 2, .... biological state features (K+Z).

[0061] In one or more examples, biological state prediction subsystem 110 may be configured to structure the biological state features based on each biological state feature’s corresponding expression value. For example, the biological state features may be structured from high expression value to low expression value (e.g., XO>X1....>X(K+Z) or from low expression value to high expression value (e.g., XO<X1... <X(K+Z)). In some embodiments, the order with which the biological features are structured corresponds to their positional order within the feature vector representing the biological state (e.g., feature vector 201). For example, biological state features having higher expression values can be positioned such that they are fed to the machine learning model as input prior to the biological state features having lower expression values. Additional details relating to the tokenization process are described herein.

[0062] In some embodiments, contextual information may be included with the biological state features. As an example, with reference to FIG. 2B, each biological state feature of the plurality of biological state features of biological state 200 may comprise: a label indicating that the biological state feature corresponds to a gene (e.g., one of first set of biological state features 202), a protein (e.g., one of second set of biological state features 204), a methylation marker (e.g., one of third set of biological state features 206), or a phenotype (e.g., one of fourth set of biological state features 208). In some embodiments, each biological state feature may be translated or represented using representation 210. For example, biological state features of first set of biological state features 202 may be represented by numerical tokens. For instance, each of the labels may be translated into a numerical token indicating a position of that biological state feature within the biological state feature vector representing the biological state. For example, first set of biological state features 202 including Gene 1, Gene 2, ..., Gene N may be represented as Token 1, Token 2, ... Token N, second set of biological state features 204 including Protein 1, Protein 2, ... Protein M may be represented as Token N+l, Token N+2, ... Token N+M, third set of biological state features 206 including Marker M+l, Marker M+2, ... Marker M+K, and fourth set of biological state features 208 including Phenotype ID K+l, Phenotype ID K+2, ... Phenotype ID K+Z may be represented as Token K+l, Token K+2, ... Token K+Z. The tokens may indicate eachbiological state feature’s position within feature vector 201. Each biological state feature may also include an expression value associated with the biological state feature. The expression value may be a numerical value indicating how much that particular biological marker was expressed within the biological sample.

[0063] In some embodiments, each biological state feature may also include at least one conditional attribute, as seen, for example, via biological state 220. The at least one conditional attribute may comprise at least one of: epigenetic markers or environmental variables. The conditional attributes may indicate pre-known or determined relationships between biological state features, or other pertinent information for analysis of the biological state. In some embodiments, the conditional attributes may be stored with the biological state (e.g., as metadata appended to the biological state feature vector representing the biological state). In some embodiments, a biological state feature vector of conditional attributes may also be stored and applied to the biological state feature vector representing the biological state. In still further embodiments, the conditional attributes may be stored in a multidimensional tensor.

[0064] FIG. 3 illustrates an example process 300 for using a machine learning model 310 to predict one or more unknown biological state features from one or more known biological state features, in accordance with various embodiments. In some embodiments, process 300 may include an input 302 being provided to machine learning model 310. Input 302 may include a biological state represented using a plurality of biological state features. The biological state features may include one or more known biological state features (e.g., predicted biological state features, empirically determined biological state features) and / or one or more unknown biological state features. For example, the biological state may include one or more unknown biological state features 304. In one or more examples, the biological state included in input 302 may be stored in biological state database 142 or may be retrieved from a client device 130 at run time.

[0065] The biological state may describe a biological system (e.g., a subject). A biological sample (e.g., tissue sample, blood sample) may be obtained from the biological system. The biological state, including (known) biological state features may be determined from the biological sample. In some embodiments, the unknown biological state features may correspond to certain biological markers having an expression value that is less than a threshold expression value. Expression values less than the threshold value may not be detectable (e.g., values less than 0.1, values less than 0.01, values less than 0.001, etc.). Insome cases, certain biological state features may not be expressed within the partial biological state for other reasons, such as due to a disease onset, therapeutic intervention, sampling error, or other reasons, or combinations thereof.

[0066] Machine learning model 310 may comprise a trained machine learning model, as described in greater detail below. In some embodiments, machine learning model 310 comprises a neural network, a long- short term memory (LSTM) model, a transformer model, or other machine learning models, or combinations thereof. For example, machine learning model 310 may comprise an encoder-decoder. In some embodiments, the biological state features representing the biological state may be input to machine learning model 310 to generate an embedding representing the biological state features. The embedding comprises a lower-dimensional representation of the biological state features in an embedding space. In one or more examples, the encoder may be used to generate the embedding.

[0067] In some embodiments, machine learning model 310 may generate output 320. Output 320 may include predicted biological state features. For example, the predicted biological state features may comprise predictions for some or all of the unknown biological state features. For example, machine learning model 310 may determine predicted biological state features 324. Predicted biological state features 324 may correspond to unknown biological state features 304. In some examples, the predicted biological state feature may also include predictions for some or all of the known biological state features. The differences between the predicted biological state features and, if applicable, their corresponding known biological state features, may be used as error correction and / or for fine-tuning machine learning model 310. In some embodiments, the decoder may be used to predict the biological state features. The decoder may map the embedding to a set of predicted biological state features. In one or more examples, the predicted biological state features may comprise a prediction of the one or more unknown biological state features. In one or more examples, the predicted biological state features may also comprise a prediction of at least one of the one or more unknown biological state features.

[0068] In some embodiments, the prediction of the unknown biological state features of the biological system (e.g., output 320) may comprise at least one of: an intra-omic prediction, an inter-omic prediction, or an inter-phenotype prediction. For the intra-omic prediction, the biological state of the biological system may comprise previously captured microarray data. In one or more examples, the biological state comprising partial RNA sequencing data may include a subset of a full set of genes, the subset includes genes havingmeasured expression values. In some examples, where the prediction comprises the inter- omic prediction, the partial biological state of the biological system may comprise RNA sequencing data and partial proteomics. For instance, the biological state may not include methylation markers. In some examples where the prediction comprises the inter-phenotype prediction, the partial biological state of the biological system may comprise RNA sequencing data and partial disease phenotype information. For instance, the partial biological state of the biological system may further comprise partial proteomics and partial methylation markers.

[0069] Returning to FIG. 1, training subsystem 112 may be configured to train a machine learning model to predict one or more unknown biological state features. Training subsystem 112 may train the machine learning model using the training data. In one or more examples, training subsystem 112 may further train the machine learning model to predict one or more known biological state features.

[0070] In some embodiments, training subsystem 112 may be configured to obtain a plurality of measured biological states each represented by a plurality of biological state features. For example, biological state database 142 may store a plurality of biological states. Different biological states may correspond to different biological systems. For example, a first collection of biological states may describe a first biological system, a second collection of biological states may describe a second biological system, and so on. In one or more examples, the plurality of biological state features of each biological system may comprise known biological state features. For example, an expression value for each biological state feature may be known. These known biological state feature’s expression values may be determined via empirical data or predictions. In some examples, the biological state features of each biological system may include one or more unknown biological state features.

[0071] For each measured biological state, training subsystem 112 may be configured to mask one or more biological state features from the plurality of biological state features representing that measured biological state. A masked biological state feature may appear (i.e., to the machine learning model receiving that biological state feature as an input) to be a null or unknown value. The model will learn to predict an expression value for a given biological state feature. In some examples, the model will learn to predict expression values for unknown biological state features. In some examples, the model will also learn to predict expression values for one or known biological state features. The predicted expression valuescan be used to determine whether there are errors in the modeling, measuring, or at another step in the data collection process, as well as for improving model accuracy.

[0072] As an example, with reference to FIG. 4, training subsystem 112 may obtain biological states 400 from biological state database 142. For example, biological states 400 may include N biological states, each corresponding to a different biological system. In some examples, two or more biological states may be obtained for a same biological system. Training subsystem 112 may be configured to mask one or more biological state features from each biological state to form training data 450, stored in training data database 144. In some embodiments, the one or more masked biological state features correspond to a same biological state feature across the measured biological states (e.g., biological state 400). For example, as illustrated in FIG. 4, for each of biological states 400, a 3rd, 7th, 9th, 13th, 17th, and 19thbiological state feature may be masked. In some embodiments, training subsystem 112 may obtain a selection of a set of biological state features to mask. The one or more masked biological state features of each measured biological state may correspond to a same feature across the plurality of measured biological states. In some embodiments, as detailed below, multiple models may be trained using models having different masked biological state features.

[0073] In some embodiments, training subsystem 112 may be configured to train aspects of the machine learning model using publicly available data and then fine-tune the model using training data stored in training data database 146. For example, the models may be trained using publicly available gene expression data, and then fine tuned on curated gene expression data. In some embodiments, only some components of the machine learning model may include a fine-tuning step in the training process.

[0074] In some embodiments, training subsystem 112 may be configured to, for each measured biological state, after the one or more biological state features having been masked, input the biological state features including the one or more masked biological state features to a machine learning model. The machine learning model may output a predicted biological state. The predicted biological state may comprise a plurality of predicted biological state features. In one or more examples, the machine learning model comprises an encoderdecoder architecture, however other machine learning model types may be used (e.g., neural networks, LSTMs, transformers, large language models (LLMs), etc.). For example, with reference to FIG. 5, machine learning model 510 (which may be the same or similar as machine learning model 310 of FIG. 3) may include an encoder 512 and a decoder 514.Machine learning model 310 may correspond to the trained instances of machine learning model 510, including a trained encoder (e.g., encoder 512) and a trained decoder (e.g., decoder 514).

[0075] Training subsystem 112 may be configured to perform process 500 to train machine learning model 510. Persons of ordinary skill in the art will recognize that other training processes may be used to train a machine learning model used by components of system 100. Thus, process 500 should not be construed as limiting the disclosed embodiments to particular training processes. In process 500, biological state 504, including biological state features (known, unknown, and / or masked), may be input to encoder 512 of machine learning model 510. In some embodiments, biological state 504 may be one of the biological states included within training data 450 (e.g., including one or more masked biological state features). In some embodiments, training data 450 may be split into training, testing, and validation data. For simplicity, training data 450 may correspond to the training data used for training, and testing and validation data will have similar formats and can be stored within training data database 146. .

[0076] In some embodiments, training subsystem 112 may select a to-be-trained machine learning model (e.g., machine learning model 510), which may be retrieved from model database 144. Machine learning model 510 may be selected based on a type of biological data being analyzed (e.g., microarray, single cell). Training subsystem 112 may also select the training data to be used to train machine learning model 510. The training data (e.g., training data 450) may be retrieved from training data database 146.

[0077] In some embodiments, the biological state features of each biological states may be input to machine learning model serially; however, parallelization may also be performed. Machine learning model 510 may be configured to generate a predicted biological state 506. Predicted biological state 506 may describe a disease state of the same biological system as biological state 504. Predicted biological state 506 may include a plurality of predicted biological state features. In one or more examples, the predicted biological state features may correspond to some or all of the masked biological state features. The predicted biological state may also correspond to some or all of the unknown and / or known biological state features from biological state 504.

[0078] In some embodiments, encoder 512 may generate an embedding 522 representing biological state 504 in an embedding space. The embedding space may be a reduceddimensional space. Thus, embedding 522 may be a compressed version of biological state504. The embedding may be input to decoder 514, which may map the embedding to predicted biological state 506. In particular, decoder 514 may map each biological state feature from biological state 504 to a predicted biological state feature.

[0079] Process 500 may repeat until an accuracy of machine learning model 510 reaches a predefined accuracy level (e.g., 95% accuracy or greater, 99% accuracy or greater, etc.), at which point machine learning model 510 may be stored in model database 144 as a trained machine learning model. The accuracy of machine learning model 510 may be determined based on a number of correct predictions (e.g., predicted biological state 506).

[0080] In some embodiments, machine learning model 510 may comprise a first machine learning model, the one or more masked biological state features may comprise one or more first masked biological state features, the predicted biological state may comprise a first predicted biological state, and the loss may comprise a first loss. In one or more examples, for each of the plurality of measured biological states, training subsystem 112 may be configured to mask one or more second biological state features from the plurality of biological state features. The second masked biological state features may correspond to a different biological state features than the first masked biological state features. For example, for the first machine learning model, the 1st, 5th, 9th, and 13thbiological state features may be masked for each of the measured biological states. For the second machine learning model, the 3rd, 8th, 12th, and 20thbiological state features may be masked for each of the measured biological states. The number of different sets of biological state features that are masked may vary depending on processing capabilities, the models being used, the size of each biological state, or based on other factors. In some examples, two or more sets of masked biological state features may include at least one similar biological state feature being masked.

[0081] In some embodiments, training subsystem 112 may be configured to train multiple machine learning models. For example, the machine learning model may comprise a first machine learning model, the one or more masked biological state features comprise one or more first masked biological state features, the predicted biological state comprises a first predicted biological state, the plurality of predicted biological state features comprises a first biological state features, the loss comprises a first loss, and the one or more parameters comprise one or more first parameters of the first machine learning model. In this example, for each of measured biological state, training subsystem 112 may be configured to mask one or more second biological state features from the plurality of biological state features. Thesebiological state features, including the second masked biological state features, may be input to a second machine learning model to obtain a second predicted biological state. The second predicted biological state may include a second plurality of predicted biological state features. Training subsystem 112 may be configured to compute a second loss based on the second plurality of biological state features and the second plurality of predicted biological state features adjust one or more parameters of the second machine learning model based on the second computed loss until the threshold training condition is satisfied. If more than two machine learning models are to be trained, then the same process may repeat. In some embodiments, the number of machine learning models to be trained may be based on the number of permutations of biological state features that can be masked.

[0082] In some embodiments, an output of the first machine learning model may comprise a first plurality of predicted biological states respectively corresponding to the plurality of measured biological states and an output of the second machine learning model may comprise a second plurality of predicted biological states respectively corresponding to the plurality of measured biological states. As an example, with reference to FIG. 6A, each of machine learning models 610-1 thru 610-N may receive different training data as inputs 602 in order to obtain predictions 604. For example, first machine learning model 610-1 may receive first training data 602- 1 including a first plurality of measured biological states each represented by a plurality of biological state features including one or more masked biological state features. First machine learning model 610-1 may generate predictions 604-1 based on first training data 602-1. Predictions 604-1 may include a predicted biological state for the corresponding predicted expression values associated with each of the masked biological state features included within first training data 602-1. A similar process applies for each of the other machine learning models. For example, second machine learning model 610-2 may use second training data 602-2 to obtain predictions 604-2, and A-th machine learning model 610-N may use training data 602-N to obtain predictions 604-N.

[0083] In some embodiments, training subsystem 112 may be configured to generate attention maps 606 for each set of predictions 604. For example, a first attention map 606-1 may be generated using the first plurality of predicted biological states (e.g., predictions 604- 1). Similarly, second attention map 606-2 may be generated using the second plurality of predicted biological states (e.g., predictions 604-2), and the A-th attention map 606-N may be generated using the A-th plurality of predicted biological states (e.g., predictions 604-N).

[0084] In some embodiments, training subsystem 112 may be configured to determine relationship maps between the plurality of biological state features represented by the biological states and the masked biological state features. The relationship maps indicate correlations between the plurality of biological state features of the plurality of measured biological states and the one or more masked biological state features. As an example, with reference to FIG. 6B, relationship maps 608 may be generated for each machine learning model. Each relationship map may describe a relationship between the unmasked biological state features and the masked biological state features based on the corresponding attention map. For example, first relationship map 608-1 may be generated based on first attention map 606-1, second relationship map 608-2 may be generated based on second attention map 606-2, and AMh relationship map 608-N may be generated based on A-th attention map 606- N.

[0085] In some embodiments, training subsystem 112 may be configured to combine relationship maps 608 to generate a gene regulatory network 620. For example, first relationship map 608-1, second relationship map 608-2, thru A-th relationship map 608-N may be combined to generate gene regulatory network 620. Gene regulatory network 620 may graphically indicate the relationships between different biological state features of a biological state by learning which biological state features are correlated with one another and / or which biological state features are uncorrelated.

[0086] In some embodiments, the predicted biological state comprising the plurality of biological state features includes predicted expression values for each of the one or more masked biological state features. For example, if, for a given biological sample, the 1st, 5th, 13th, and 19thbiological state features are masked, the predicted biological state may include predicted expression values for each of the 1st, 5th, 13th, and 19thbiological state features.

[0087] In some embodiments, training subsystem 112 may be configured compute a loss using the measured biological state and the predicted biological state. The measured biological state may comprise the plurality of biological state features without the one or more masked biological state features being masked. The loss may be computed based on a difference between an actual expression value, as included by the measured biological state, and a predicted expression value.

[0088] In some embodiments, training subsystem 112 may be configured to adjust one or more parameters of the machine learning model based on the computed loss until a threshold training condition has been satisfied. For example, one or more weights and biases ofmachine learning model 510 (e.g., including encoder 512 and decoder 514) may be varied based on the computed loss.

[0089] FIG. 7 illustrates an example of a masked gene expression matrix 710 and a predicted gene expression matrix 720, in accordance with various embodiments. As illustrated by FIG. 7, masked gene expression matrix 710 includes multiple biological state features whose expression value is below a threshold expression value. On the other hand, using the techniques described herein for predicting a full biological state based on a partial biological state (i.e., including one or more masked biological state features). The masked biological state features correspond to the “white” squares within masked gene expression matrix 710. For example, the expression values are indicated by the legend on the right hand side, which details that an expression value of X is represented using color Y (e.g., expression level < threshold expression value = “white”; expression level > threshold expression level but less than second threshold expression value = “brown”; expression level greater than a third threshold expression level (e.g., 90) = “yellow”). Predicted gene expression matrix 720, thus, includes very few, if any, biological state features that have an unknown expression level (e.g., less than the threshold expression level).

[0090] In some embodiments, prior to masking biological state features from the plurality of biological state features of the measured biological state, the biological state features may be re-structured. For example, a tokenized representation of the measured biological state may be generated, as detailed below, and the tokenized representation may serve as the basis for masking. In some embodiments, a tokenized representation may be generated for each of the plurality of measured biological states. In one or more examples, the tokenized representation of the biological state (full biological state or partial biological state) may be input to the machine learning model (e.g., machine learning model 510).

[0091] In one or more examples, each of the plurality of biological state features may include a label and an expression value. For example, first set of biological state features 202 may include a label indicating a corresponding gene of a set of genes represented by the biological state feature (e.g., Gene 1, Gene 2, Gene 3, ... Gene N) and an expression level of the phenotype within the sample of the biological system. In one or more examples, the tokenized representation of a biological state may comprise the corresponding plurality of biological state features being ordered based on the expression value of each of the plurality of biological state features. In some embodiments, this may include structuring the plurality of biological state features such that higher expression values are included at a beginning ofdata representing the tokenized representation and lower expression values are included at an end of the data.

[0092] In some embodiments, each biological state feature of the plurality of biological state features associated with each biological state may comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature; and at least one conditional attribute associated with the biological state feature. In one or more examples, the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

[0093] In some embodiments, the plurality of biological state features representing each of the plurality of measured biological states may comprise an n-di mcnsional biological state feature vector. In one or more examples, the first set of biological state features may correspond to RNA expression data, the second set of biological state features may correspond to protein expression data, the third set of biological state features may correspond to methylation markers, and the fourth set of biological state features may correspond to phenotype data. The biological state may be determined using a biological sample obtained from a biological system (e.g., a subject). For example, the biological sample may be a blood sample from which RNAseq data is extracted. Each biological state feature may represent a biological marker. In one or more examples, the biological markers may include genomic data, proteomic data, transcriptomic data, physiological data, phenotypical data, or other biological markers, or combinations thereof.

[0094] In some examples, each biological state feature of the first set of biological state features may comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature (e.g., Gene 1, Gene 2, Gen 3, ... Gene A) and an expression level of the gene within a sample of the biological system. The expression level is represented using a color bar (e.g., red for biological markers that are expressed more, green for biological markers expressed less) and / or using a numerical value (e.g., a value between 0.0 and 1.0). In one or more examples, each biological state feature of the second set of biological state features may comprise: a label indicating a corresponding protein from a set of proteins represented by the biological state feature (e.g., Protein 1, Protein 2, Protein 3, ... Protein M) and an expression level of the protein within the sample of the biological system. In one or more examples, each biological state feature of the third set of biological state features may comprise: a label indicating a corresponding methylation marker of a set ofmethylation markers represented by the biological state feature (e.g., Marker 1, Marker 2, Marker 3, ... Marker K) and an expression level of the methylation marker within the sample of the biological system. In one or more examples, each biological state feature of the fourth set of biological state features may comprise: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature (e.g., Phenotype ID 1, Phenotype ID 2, Phenotype ID 3, ... Phenotype ID Z) and an expression level of the phenotype within the sample of the biological system. Similar to the first set of biological state features, the expression value of the second set of biological state features, the third set of biological state features, and fourth set of biological state features may be represented using a color bar and / or numerically. In some embodiments, the masked biological state features may be selected from at least one of the first set of biological state features, the second set of biological state features, the third set of biological state features, or the fourth set of biological state features.

[0095] FIG. 8 illustrates a flowchart of an example method 800 for generating a prediction of a full biological state of a biological system, in accordance with various embodiments. In some embodiments, method 800 may be performed by a subsystem that is the same or similar as biological state prediction subsystem 110. In some embodiments, method 800 may begin at step 802. At step 802, a plurality of biological state features of the biological state into one or more machine learning models to generate an embedding representing the biological state. The plurality of biological state features may comprise one or more known biological state features and one or more unknown biological state features. At step 804, at least one of the one or more unknown biological state features may be predicted. The at least one unknown biological state feature may be predicted using the one or more machine learning models. In one or more examples, the at least one unknown biological state feature may be predicted based on the embedding.

[0096] FIG. 9 illustrates a flowchart of an example method 900 for training a machine learning model to predict one or more unknown biological state features from one or more known biological state features, in accordance with various embodiments. In some embodiments, method 900 may be performed by a subsystem that is the same or similar as training subsystem 112. In some embodiments, method 900 may begin at step 902.

[0097] At step 902, a plurality of measured biological states may be obtained. Each of the measured biological states may be represented by a plurality of biological state features. At step 904, a measured biological state may be selected from the plurality of measuredbiological states. At step 906, one or more biological state features from the plurality of biological state features may be masked. At step 908, the biological state features, including the one or more masked biological state features, may be input to a machine learning model. The machine learning model may output a predicted biological state. The predicted biological state may be represented using a plurality of predicted biological state features. At step 910, a loss may be computed based on the biological state features of the selected measured biological state and the predicted biological state features. At step 912, one or more parameters of the machine learning model may be adjusted. In one or more examples, the adjustments may be based on the computed loss. At step 914, a determination may be made as to whether there are any more measured biological states to analyze. If so, method 900 may return to step 904, where another measured biological state may be selected. In some embodiments, steps 904-914 may repeat until a threshold number of biological states have been analyzed. In some embodiments, steps 904-914 may repeat until all of the measured biological state have been analyzed. In some embodiments, steps 904-914 may repeat until the model’s accuracy exceeds a threshold accuracy (e.g., 90% or greater, 95% or greater, etc.). If, however, at step 914, it is determined that no more measured biological states are to be analyzed, then method 900 may end. In some embodiments, method 900 may repeat until a threshold training condition has been satisfied.II. Example System for Tokenization

[0098] FIG. 10 illustrates an example system 1000 for generating a tokenized representation of a biological state for a machine learning model, in accordance with various embodiments. Some components of system 1000 may be the same or similar to those of system 100 of FIG. 1, and the previous descriptions of those components apply here. However, system 1000 may include one or more other components not previously described by system 1000. That is to say, some or all of the components of system 1000 may be used in conjunction with the components of system 100.

[0099] System 1000 may include a computing system 1002, user devices 130, databases 140 (e.g., biological state database 142, training data database 144, model database 146), or other components. In some embodiments, components of system 1000 may communicate with one another using network 150, such as the Internet.

[0100] It should be noted that while one or more operations are described herein as being performed by particular components of computing system 1002, those operations may, in some embodiments, be performed by other components of computing system 1002 or othercomponents of system 1000. As an example, while one or more operations are described herein as being performed by components of computing system 1002, those operations may, in some embodiments, be performed by aspects of user devices 130. It should also be noted that, although some embodiments are described herein with respect to machine learning models, other prediction models (e.g., statistical models or other analytics models) may be used in lieu of or in addition to machine learning models (e.g., a statistical model replacing a machine-learning model and a non-statistical model replacing a non-machine-leaming model in one or more embodiments). Still further, although a single instance of computing system 1002 is depicted within system 1000, additional instances of computing system 1002 may be included (e.g., computing system 1002 may comprise a distributed computing system).

[0101] Computing system 1002 may include a reduction subsystem 1010, a transformation subsystem 1012, a discretization subsystem 1014, a structuring subsystem 1018, a model training subsystem 1020, or other components. Each of subsystems 1010-1020 may be configured to communicate with one another, one or more other devices, systems, and / or servers, using network 150 (e.g., the Internet, an Intranet). This disclosure anticipates the use of one or more of each type of system and component thereof without necessarily deviating from the teachings of this disclosure. Furthermore, although not illustrated, other intermediary devices (e.g., data stores of a server connected to computing system 1002) can also be used.

[0102] The components of system 1000 of FIG. 10 can be used in a variety of contexts where scanning and evaluating digital pathology images, such as whole slide images, or biological samples, are essential components of the work. As an example, system 1000 can be associated with a clinical environment where a user is evaluating cells for drug discovery and disease monitoring. The user can review an image using user device 130 prior to providing the image to computing system 1002. The user can provide additional information to computing system 1002 that can be used to guide or direct the analysis of the image. Persons of ordinary skill in the art will recognize that, in some examples, no user review of the image may be needed.

[0103] In some embodiments, reduction subsystem 1010 may be configured to obtain a biological state describing a biological system. The biological state may comprise a plurality of biological state features each having an expression value. In one or more examples, obtaining the biological state may comprise receiving expression data comprising a plurality of samples obtain for the biological system. The biological system, for example, maycorrespond to a subject (e.g., a human subject, an animal subject, etc.). The samples may therefore include a plurality of tissue samples and / or blood samples obtained from the subject. The biological state may represent a health state of the subject at a given time (when the sample was collected). In one or more examples, each of the samples may be represented using a sample- specific biological state. For example, a plurality of samples may be obtained for a given subject and the plurality of samples may be used to determine the biological state of the subject.

[0104] In one or more examples, each biological state may be derived from RNA sequencing data and / or single cell RNA sequencing data obtained from the biological system. For example, RNA sequencing data may be obtained from a biological sample (e.g., blood sample) of the biological system. In one or more examples, the plurality of biological state features may comprise a set of biological state features each corresponding to a gene of a set of genes. In one or more examples, each gene of the set of genes may comprise an identifier for identifying the gene in one or more biological databases.

[0105] In some embodiments, the plurality of biological state features representing each of the plurality of measured biological states may comprise an n-di mcnsional biological state feature vector. In one or more examples, the first set of biological state features may correspond to RNA expression data, the second set of biological state features may correspond to protein expression data, the third set of biological state features may correspond to methylation markers, and the fourth set of biological state features may correspond to phenotype data. The biological state may be determined using a biological sample obtained from a biological system (e.g., a subject). For example, the biological sample may be a blood sample from which RNAseq data is extracted. In some embodiments, the biological state may be represented using a biological state feature vector. Each biological state feature may represent a biological marker. In one or more examples, the biological markers may include genomic data, proteomic data, transcriptomic data, physiological data, phenotypical data, or other biological markers, or combinations thereof.

[0106] In some embodiments, for each sample collected from the biological system, a sample- specific biological state may be determined. The sample- specific biological states may be combined, via reduction subsystem 1010, to obtain a combined biological state. In one or more examples, the biological state comprising the subset of biological state features may be determined based on the combined biological state. For instance, the sample- specific biological state of each of the plurality of samples may be represented using a plurality of 1sample- specific biological state features each having a sample- specific expression value. In this example, combining the sample- specific biological states may include reduction subsystem 1010 averaging the sample- specific expression value of each of the plurality of sample- specific biological state features to obtain an average expression value for a given biological state feature. The expression value of each biological state feature of the subset of the plurality of biological state features may comprise the averaged sample- specific expression value.

[0107] As an example, with reference to FIG. 11, a plurality of sample- specific biological states, biological states 11, 21, 31, ... LI, may be obtained, describing a first biological system, biological system 1. Each of biological states 11, 21, ... LI may include a plurality of biological state features, X0 - X(K+Z).

[0108] As mentioned previously, for example with reference to FIGS. 2A-2B, the plurality of biological state features may be grouped into one or more sets of biological state features (e.g., first set of biological state features 202 corresponding to RNA expression data, second set of biological state features 204 corresponding to protein expression data, third set of biological state features 206 corresponding to methylation markers, and / or fourth set of biological state features 208 corresponding to phenotype data). Each biological state feature of first set of biological state features 202 may comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature (e.g., Gene 1, Gene 2, Gen 3, ... Gene O and an expression level of the gene within a sample of the biological system. Each biological state feature of second set of biological state features 204 may comprise: a label indicating a corresponding protein from a set of proteins represented by the biological state feature (e.g., Protein 1, Protein 2, Protein 3, ... Protein M) and an expression level of the protein within the sample of the biological system. Each biological state feature of third set of biological state features 206 may comprise: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature (e.g., Marker 1, Marker 2, Marker 3, ... Marker K) and an expression level of the methylation marker within the sample of the biological system. Each biological state feature of fourth set of biological state features 208 comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature (e.g., Phenotype ID 1, Phenotype ID 2, Phenotype ID 3, ... Phenotype ID Z) and an expression level / confidence score of the phenotype within the sample of the biological system. In some examples, for instance biological state feature vector 201, may represent each biological state feature as a numericalvalue (e.g., a number between 0.0 and approximately 1.0, a number between 0.0 and 100, or using another numbering convention). For example, first set of biological state features 202 may be represented using variables X0-XN (also referred to interchangeably as “tokens”) in biological state feature vector 201, second set of biological state features 204 may be represented using variables X(N+1)-X(N+M), third set of biological state features 206 may be represented using variables X(M+1)-X(M+K), and fourth set of biological state features 208 may be represented using variables X(K+1)-X(K+Z). A total number of biological state features represented by biological state feature vector 201, and thus a total number of biological markers included in biological state 200 may be 103or more, 105or more, 107or more, or other values. In some examples, the set of genes includes between about 10A3 and about 10A6 genes. In some embodiments, a quantity of biological state features used to represent a biological state may be 10A6 or more biological state features, 10A7 or more biological state features, 10A8 or more biological state features, 10A9 or more biological state features, and the like.

[0109] Returning to FIG. 11, the corresponding biological state features of the plurality of biological state features across the sample- specific biological states (e.g., biological states 11- Ll) may be averaged. For instance, an expression value for the first biological state feature X0 of each of biological states 11 -LI may be averaged together. The averaged expression values for a biological state feature may be stored in average biological state 1. For example, the average expression value for biological state feature X0 may be represented by average biological state 1. A similar process may be performed for each biological state feature of the plurality of biological state features to obtain the entire plurality of biological state features represented by average biological state 1. Similarly, the aforementioned processes may be performed for each of a plurality of biological systems. For example, if there are Y biological systems to be analyzed, biological samples for each of the Y biological systems may be obtained and the expression values for each biological state feature represented by the biological states of each biological sample may be averaged together to obtain the corresponding averaged biological state. Therefore, Y averaged biological states may be used to represent each of the Y biological systems.

[0110] In some embodiments, for each of the sample-specific plurality of biological state features, reduction subsystem 1010 may determine whether the averaged sample- specific expression value satisfies a threshold condition. For example, for each biological state feature of the plurality of biological state features representing one of the averaged biologicalstates, a determination may be made as to whether the expression value, which has been averaged across a plurality of samples for that biological system, may be compared to a threshold expression value. If the average sample- specific expression value satisfies the threshold condition, then this may indicate that the averaged sample- specific expression value is greater than or equal to the threshold expression value. If the average sample- specific expression value fails to satisfy the threshold condition, then this may indicate that the averaged sample- specific expression value is less than a threshold expression value. In one or more examples, the threshold expression value may be about 1.0, however other values may be used. In some cases, the threshold expression value may be an adjustable hyperparameter of the machine learning model that will be used to process the resulting tokenized representations of each biological state. Furthermore, the threshold expression value may be pre-selected by a user based on historical threshold expression values and / or may be adjusted.

[0111] If reduction subsystem 1010 determines that the averaged sample- specific expression value of a given sample- specific biological state feature fails to satisfy the threshold condition (e.g., the averaged sample-specific expression value is less than the threshold expression value), then reduction subsystem 1010 may be configured to remove the sample- specific biological state feature from the plurality of biological state features. The result of the removal of this / these biological state features may produce a subset biological state comprising a subset of the plurality of biological state features. Here, the subset of the plurality of biological state features corresponds to the remaining biological state features of the plurality of biological state features after the one or more biological state features that fail to satisfy the threshold condition have been removed. If, however, reduction subsystem 1010 determines that the averaged sample- specific expression value of a given sample- specific biological state features satisfies the threshold condition (e.g., the averaged sample- specific expression value is greater than or equal to the threshold expression value), then reduction subsystem 1010 may be configured to keep the sample-specific biological state feature included in the plurality of biological state features. Therefore, the subset of the plurality of biological state features is formed based on whether a corresponding biological state feature of the sample- specific plurality of biological state features was removed or kept.

[0112] As an example, with reference to FIG. 12, each of averaged biological states 1-Y may be analyzed to determine whether any of biological state features X0-X(K+Z) have an expression value that is less than the threshold expression value. For instance, for averaged biological state 1, the first biological state feature, X0, may have an expression value of “0.”If the threshold expression value is 1.0, for example, then this may indicate that biological state feature X0 can be removed from the plurality of biological state features representing biological system l’s biological state. For instance, subset biological state 1 may comprise a subset of the plurality of biological state features this expression value included in averaged biological state 1 (e.g., the biological state feature X0 has been removed). At the end of this process, a total of Y subset biological states (e.g., subset biological state 1-Y) may be obtained, each including a subset of the plurality of biological state features stored by the corresponding averaged biological state.

[0113] Returning to FIG. 10, transformation subsystem 1012 may be configured to compute a logarithmic expression value for each biological state feature of the subset of biological state features (e.g., after biological state features that fail to satisfy a threshold condition have been removed). In one or more examples, the logarithmic expression may be computed by calculating a log2(n+l) value for each expression value included within the subset of the plurality of biological state features represented by the subset biological state. As an example, with reference to FIG. 13, a logarithmic transformation may be performed to transform each expression value included within the subset of biological state features (e.g., biological state features XI, X3, ... X(K+Z)) of subset biological state 1300 (e.g., subset biological state 1). Transformation subsystem 1012 may output a transformed biological state 1310 including the subset of the plurality of biological state features, where each biological state feature has a transformed (e.g., log-transform) expression value.

[0114] Returning to FIG. 10, discretization subsystem 1014 may be configured to discretize the logarithmic expression value of each biological state feature of the subset of the plurality of biological state features into a discretized expression value of the biological state feature. In some embodiments, discretization subsystem 1014 may be configured to categorize the logarithmic expression values into equal-length ranges. For example, with reference again to FIG. 13, discretized biological state 1320 may represent transformed biological state 1310 after the expression values have been sorted into equal-length ranges. In some embodiments, the ranges may have a same “size.” For instance, each range (also referred to as a “bin”) may cover a range of expression values. For example, biological state feature XI of discretized biological state 1320 may span a first range (e.g., a first expression value to a second expression value), biological state feature X2 may span a second range (e.g., the second expression value to a third expression value), and so on.

[0115] Returning again to FIG. 10, structuring subsystem 1018 may be configured to generate a tokenized representation of the biological system. In one or more examples, structuring subsystem 1018 may generate the tokenized representation such that it includes the subset of the plurality of biological state features structured using an ordering function based on the discretized expression value of each biological state feature. For example, with reference again to FIG. 13, tokenized representation 1330 may comprise the subset of the plurality of biological state features re-ordered such that those biological state features with greater discretized expression values are at the beginning of tokenized representation 1330, while those having smaller discretized expression values are at the end of tokenized representation 1330. In contrast, each of the previously mentioned biological states (e.g., subset biological state 1300, transformed biological state 1310, discretized biological state 1320) include the subset of the plurality of biological state features ordered based on a corresponding label associated with that biological state feature. For example, the biological states may be structured such that the expression values for “Gene 1” are positioned at the beginning of the biological state, and Gene N (e.g., corresponding to the K+Z-th biological state feature) are positioned at the end of the biological state.

[0116] In some embodiments, the ordering function may indicate an order for the subset of the plurality of biological state features included by the tokenized representation. For example, the ordering function may determine the order of the subset of the plurality of biological state features should be an ascending or descending order. Other ordering function may be used as well, and the aforementioned technique is not limited to just ascending and descending.

[0117] Returning again to FIG. 10, model training subsystem 1020 may be configured to generate training data for training a machine learning model to output a predicted biological state, where the training data may comprise tokenized representations of biological states. In some embodiments, model training subsystem 1020 may additionally or alternatively be configured to train a machine learning model to output a predicted biological state using training data comprising tokenized representations of biological states.

[0118] In some embodiments, training subsystem 112 may be configured to train aspects of the machine learning model using publicly available data and then fine-tune the model using training data stored in training data database 146. For example, the models may be trained using publicly available gene expression data, and then fine-tuned on curated geneexpression data. In some embodiments, only some components of the machine learning model may include a fine-tuning step in the training process.

[0119] In some embodiments, model training subsystem 1020, when generating training data, may be configured to retrieve a plurality of tokenized representations each describing a biological system of a sample of a plurality of samples. The tokenized representation may be generated using the aforementioned processes performed using one or more of subsystem 1010-1018. Model training subsystem 1020 may generate the training data using the plurality of tokenized representations. In some embodiments, the machine learning model to be trained comprises an encoder-decoder architecture. In some embodiments, the machine learning model comprises a neural network, LSTM, and / or transformer.

[0120] The tokenized representation of a biological state may be provided as input to the machine learning model. For example, the tokenized representation may be input to an encoder. The encoder may be trained to generate an embedding representing the tokenized representation in an embedding space. A decoder may receive, as input, the embedding, and may use the embedding to generate a prediction of a biological state. In one or more examples, the machine learning model may be trained to predict an expression value of a gene from a cell, predict a cell classification, predict a chronological development of a cell, or perform other predictions, or combinations thereof.

[0121] FIG. 14 illustrates a flowchart of an example method 1400 for generating a tokenized representation of a biological state, in accordance with various embodiments. In some embodiments, method 1400 may begin at step 1402.

[0122] At step 1402, a biological state describing a biological system may be obtained. The biological state may comprise a plurality of biological state features. Each biological state feature may have an expression value indicating a level with which that biological state feature (e.g., gene, protein, methylation marker, etc.) is expressed within a biological sample of the biological system. At step 1404, a biological state feature from a subset of the biological state features may be selected. In some embodiments, steps 1402 and 1404 may be performed by a subsystem that is the same or similar as reduction subsystem 110.

[0123] At step 1406, a logarithmic expression value of the biological state feature may be computed based on its expression value. For example, a log2(Ei) calculation may be performed for each biological state feature to obtain the logarithmic expression value LEi. Here, LEi corresponds to the logarithmic expression value of gene z based on expressionvalue Ei. In some embodiments, step 1406 may be performed by a subsystem that is the same or similar to transformation subsystem 112.

[0124] At step 1408, the logarithmic expression value may be discretized to obtain a discretized expression value of the feature. The discretization process may transform each logarithmic expression value into an equal bins. For example, the discretization process may categorize LEi into equal length ranges using LEBi = [LEi * b], where b indicates a size of the ranges. In the example of FIG. 13, for instance, b is set equal to 3. In some embodiments, step 1408 may be performed by a subsystem that is the same or similar as discretization subsystem 1014.

[0125] At step 1410, a determination may be made as to whether any more biological features are to be analyzed. If so, method 1400 may return to step 1404 where another biological state feature may be selected from the subset of the biological state features. If not, method 1400 may proceed to step 1412. At step 1412, a tokenized representation of the biological state may be generated. The tokenized representation may include the plurality of biological state features structured using an ordering function based on the discretized expression value of each biological state feature. In some embodiments, steps 1410 and 1412 may be performed by a subsystem that is the same or similar to structuring subsystem 1016.III. Example System for Longitudinal Prediction

[0126] FIG. 15 illustrates an example system 1500 for determining a future biological state of a biological system, in accordance with various embodiments. Some components of system 1500 may be the same or similar to those of system 100 of FIG. 1 and those of system 1000 of FIG. 10, and the previous descriptions of those components apply here. However, system 1500 may include one or more other components not previously described by system 1500. That is to say, some or all of the components of system 1500 may be used in conjunction with the components of system 100 and / or system 1000.

[0127] System 1500 may include a computing system 1502, user devices 130, databases 140 (e.g., biological state database 142, training data database 144, model database 146), or other components. In some embodiments, components of system 1500 may communicate with one another using network 150, such as the Internet.

[0128] It should be noted that while one or more operations are described herein as being performed by particular components of computing system 1502, those operations may, in some embodiments, be performed by other components of computing system 1502 or othercomponents of system 1500. As an example, while one or more operations are described herein as being performed by components of computing system 1502, those operations may, in some embodiments, be performed by aspects of user devices 130. It should also be noted that, although some embodiments are described herein with respect to machine learning models, other prediction models (e.g., statistical models or other analytics models) may be used in lieu of or in addition to machine learning models (e.g., a statistical model replacing a machine-learning model and a non-statistical model replacing a non-machine-leaming model in one or more embodiments). Still further, although a single instance of computing system 1502 is depicted within system 1500, additional instances of computing system 1502 may be included (e.g., computing system 1502 may comprise a distributed computing system).

[0129] Computing system 1502 may include a future biological state prediction subsystem 1510, a model training subsystem 1512, or other components. Each of subsystems 1510-1512 may be configured to communicate with one another, one or more other devices, systems, and / or servers, using network 150 (e.g., the Internet, an Intranet). This disclosure anticipates the use of one or more of each type of system and component thereof without necessarily deviating from the teachings of this disclosure. Furthermore, although not illustrated, other intermediary devices (e.g., data stores of a server connected to computing system 1502) can also be used.

[0130] The components of system 1500 of FIG. 15 can be used in a variety of contexts where scanning and evaluating digital pathology images, such as whole slide images, or biological samples, are essential components of the work. As an example, system 1500 can be associated with a clinical environment where a user is evaluating cells for drug discovery and disease monitoring. The user can review an image using user device 130 prior to providing the image to computing system 1502. The user can provide additional information to computing system 1502 that can be used to guide or direct the analysis of the image. Persons of ordinary skill in the art will recognize that, in some examples, no user review of the image may be needed.

[0131] In some embodiments, subsystem 1510 may be configured to determine a first biological state and a second biological state. The first biological state and the second biological state may both describe a same biological system. However, first biological state may describe the biological system at a first time and the second biological state may describe the biological system at a second time. The first time can be earlier than the secondtime, or vice versa. In some embodiments, additional biological state describing the biological system at other times may also be determined.

[0132] In one or more examples, the first biological state may be represented using a first plurality of biological state features each having an expression value and the second biological state may be represented using a second plurality of biological state features each having an expression value. In one or more examples, receiving the first biological state and the second biological state may comprise receiving expression data comprising a plurality of samples obtain for the biological system at different times. The biological system, for example, may correspond to a subject (e.g., a human subject, an animal subject, etc.). The biological samples may therefore include a plurality of tissue samples and / or blood samples obtained from the subject. The biological state may represent a health state of the subject at a given time (when the sample was collected).

[0133] In some embodiments, the plurality of biological state features representing each of the plurality of measured biological states may comprise an n-di mcnsional biological state feature vector. In one or more examples, the biological state features may include one or more sets of biological state features. For example, the biological state features may include a first set of biological state features which may correspond to RNA expression data, a second set of biological state features which may correspond to protein expression data, and / or a third set of biological state features which may correspond to methylation markers. The biological state features may, optionally, include a fourth set of biological state features corresponding to phenotype data. The biological state may be determined using a biological sample obtained from a biological system (e.g., a subject). For example, the biological sample may be a blood sample from which RNAseq data is extracted. In some embodiments, the biological state may be represented using a feature vector. Each biological state feature may represent a biological marker. In one or more examples, the biological markers may include genomic data, proteomic data, transcriptomic data, physiological data, phenotypical data, or other biological markers, or combinations thereof.

[0134] In some examples, each biological state feature of the first set of biological state features may comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature (e.g., Gene 1, Gene 2, Gen 3, ... Gene and an expression level of the gene within a sample of the biological system. The expression level is represented using a color bar (e.g., red for biological markers that are expressed more, green for biological markers expressed less) and / or using a numerical value (e.g., a value between0.0 and 1.0). In one or more examples, each biological state feature of the second set of biological state features may comprise: a label indicating a corresponding protein from a set of proteins represented by the biological state feature (e.g., Protein 1, Protein 2, Protein 3, ... Protein Af) and an expression level of the protein within the sample of the biological system. In one or more examples, each biological state feature of the third set of biological state features may comprise: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature (e.g., Marker 1, Marker 2, Marker 3, ... Marker K) and an expression level of the methylation marker within the sample of the biological system. In one or more examples, each biological state feature of the fourth set of biological state features may comprise: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature (e.g., Phenotype ID 1, Phenotype ID 2, Phenotype ID 3, ... Phenotype ID Z) and an expression level of the phenotype within the sample of the biological system. Similar to the first set of biological state features, the expression value of the second set of biological state features, the third set of biological state features, and fourth set of biological state features may be represented using a color bar and / or numerically. In some embodiments, the masked biological state features may be selected from at least one of the first set of biological state features, the second set of biological state features, the third set of biological state features, or the fourth set of biological state features.

[0135] In one or more examples, each biological state may be derived from RNA sequencing data and / or single cell RNA sequencing data obtained from the biological system. For example, RNA sequencing data may be obtained from a biological sample (e.g., blood sample) of the biological system. In one or more examples, the plurality of biological state features may comprise a set of biological state features each corresponding to a gene of a set of genes. In one or more examples, each gene of the set of genes may comprise an identifier for identifying the gene in one or more biological databases.

[0136] As mentioned previously, for example with reference to FIGS. 2A-2B, the plurality of biological state features may be grouped into one or more sets of biological state features (e.g., first set of biological state features 202 corresponding to RNA expression data, second set of biological state features 204 corresponding to protein expression data, third set of biological state features 206 corresponding to methylation markers, and fourth set of biological state features 208 corresponding to phenotype data). Each biological state feature of first set of biological state features 202 may comprises: a label indicating a correspondinggene from a set of genes represented by the biological state feature (e.g., Gene 1, Gene 2, Gen 3, ... Gene Q and an expression level of the gene within a sample of the biological system. Each biological state feature of second set of biological state features 204 may comprise: a label indicating a corresponding protein from a set of proteins represented by the biological state feature (e.g., Protein 1, Protein 2, Protein 3, ... Protein Af) and an expression level of the protein within the sample of the biological system. Each biological state feature of third set of biological state features 206 may comprise: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature (e.g., Marker 1, Marker 2, Marker 3, ... Marker K) and an expression level of the methylation marker within the sample of the biological system. Each biological state feature of fourth set of biological state features 208 comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature (e.g., Phenotype ID 1, Phenotype ID 2, Phenotype ID 3, ... Phenotype ID Z) and an expression level of the phenotype within the sample of the biological system. In some examples, for instance biological state feature vector 201, may represent each biological state feature as a numerical value (e.g., a number between 0.0 and approximately 1.0, a number between 0.0 and 100, or using another numbering convention). For example, first set of biological state features 202 may be represented using variables X0-XN in biological state feature vector 201, second set of biological state features 204 may be represented using variables X(N+1)-X(N+M), third set of biological state features 206 may be represented using variables X(M+1)-X(M+K), and fourth set of biological state features 208 may be represented using variables X(K+1)- X(K+Z). A total number of biological state features represented by biological state feature vector 201, and thus a total number of biological markers included in biological state 200 may be 103or more, 105or more, 107or more, or other values. In some examples, the set of genes includes between about 10A3 and about 10A6 genes. In some examples, the plurality of biological state features include 10A6 or more biological state features, 10A7 or more biological state features, 10A8 or more biological state features, 10A9 or more biological state features, and the like.

[0137] In some embodiments, the subject may be a human subject, an animal subject, or another subject. In some examples, the biological state may be derived from a biological sample obtained from the subject. For instance, the sample may comprise a tissue sample or a cellular sample. In some embodiments, the biological state of the biological system may be captured at a first time, and the future biological state of the biological system corresponds toa second time occurring after the first time. In some embodiments, multiple biological states of the biological system, each captured at a different time may be received and used to predict the future biological state.

[0138] Subsystem 1510 may further be configured to generate, using one or more machine learning models, a first embedding representing the first biological state at the first time and a second embedding representing the second biological state at a second time. In some embodiments, the trained machine learning model comprises an encoder, a generator, and a decoder. In some examples, the machine learning model may comprise a neural network, an LSTM, and / or a transformer model.

[0139] As an example, with reference to FIG. 16A, machine learning model 1610 may comprise an encoder, a generator, and a decoder. FIG. 16A illustrates an example process 1600 for using a machine learning model to obtain a prediction of a future biological state, in accordance with various embodiments. In process 1600, subsystem 1510 may provide, as input 1602, the first biological state describing the biological system at time tl and the second biological state describing the biological system at time t2 to the encoder portion of machine learning model 1610. The encoder may generate the first embedding representing the first biological state and the second embedding representing the second biological state. The first embedding and the second embedding may comprise a compressed version of the first and second biological states in a lower-dimensional embedding space.

[0140] Subsystem 1510 may further be configured to predict, using the one or more machine learning models, a future embedding representing a future biological state of the biological system at a future time. The future time may correspond to any time occurring temporally after the first time and the second time. In some embodiments, the future time may be determined based on the first and second time. For example, if the first time is tl and the second time is t2, then the difference between time tl and t2 is T = |tl-t2|. The future time may then by tl + j T or t2+ j T, depending on which of times tl or t2 occur later. The variable j may correspond to a multiplicative factor that can be applied. For example, the future time may be tl + 2T or t2+2T.

[0141] In some embodiments, subsystem 1510 may predict the future embedding using the generator component of machine learning model 1610. For example, the first embedding and the second embedding may be input to the generator, which outputs a predicted future embedding representing a future biological state of the biological system at the future time. In some embodiments, an indication of the future time (for which the prediction is to bemade) may be input to the generator with the first embedding and the second embedding. In some examples, the future time may be determined based on the times of the biological states included within input 1602 and / or based on predefined settings for machine learning model 1610. In some embodiments, the future time may be a configurable and / or learned parameter of machine learning model 1610.

[0142] Subsystem 1510 may still further be configured to generate, using the one or more machine learning models, the prediction of the future biological state describing the biological system at the future time based on the future embedding. In some embodiments, the future embedding may be input to a decoder component of machine learning model 1610. The decoder may decode the future embedding into a future biological state. The predicted future biological state may be included as output 1620 from machine learning model 1610. For example, the future biological state may correspond to a prediction of a biological system’s biological state at a future time t+T. In some embodiments, output 1620 may include multiple future biological states, each

[0143] In some embodiments, subsystem 1510 may further be configured to determine, using the machine learning models, a prediction of a disease state of the biological system at the future time based on the prediction of the future biological state of the biological system. For example, the prediction of the future biological state may serve as an input to a trained classifier to determine a predicted disease state of the biological system at the future time. As an example, with reference to FIG. 16B, the predicted future biological state at the future time may be used as input 1660 to classifier 1670. Classifier 1670 may be a multi-class classifier or a binary classifier — however one or more alternative classifiers may be used too. Each class may correspond to a disease state. Classifier 1670 may generate an output 1680 comprising a predicted disease station classification 1682. some embodiments, subsystem 1510 may be configured to determine one or more therapeutic interventions for the biological system based on the prediction of the disease state of the biological system. For example, therapeutic interventions may be selected and recommended based on disease state classification 1682.

[0144] In some embodiments, the prediction of the future biological state (e.g., output 1620) may comprise a first prediction of a first future biological state of the biological system at a first future time (e.g., at time T(tl+T)). Machine learning model 1610 may further be configured to output at least a second prediction of a second future biological state describing the biological system at a second future time (e.g., at time T(tl+T+Tl). In someembodiments, the disease state classification of the future biological state of the biological system comprises a first disease classification of the first future biological state of the biological system. Subsystem 1510 may be configured to determine, using machine learning model 1610, a second disease state classification of the second future biological state of the biological system. The second future biological state may also be determined based on input 1602. In this example, subsystem 1510 may be configured to determine a predicted phenotype classification based on the first phenotype classification and the second phenotype classification.

[0145] In some embodiments, output 1680 may include multiple disease classifications, each associated with a different future time. For example, output 1620 of FIG. 16A may include a prediction of one or more future biological states of the biological system at one or more respective future times. Each predicted future biological state may be input to classifier 1670 to obtain one or more disease state classifications.

[0146] FIG. 17 illustrates an example of a training process 1700 for training a machine learning model to predict a future biological state of a biological system, in accordance with various embodiments. In some embodiments, training process 1700 may be performed by training subsystem 1512. In some embodiments, training subsystem 1512 may be configured to train aspects of the machine learning model using publicly available data and then finetune the model using training data stored in training data database 146. For example, the models may be trained using publicly available gene expression data, and then fine-tuned on curated gene expression data. In particular, the encoder, generator, and decoder may be pretrained on publicly available gene expression data. In some embodiments, only some components of the machine learning model may include a fine-tuning step in the training process. For example, in addition to pre-training the generator on publicly available gene expression data, training subsystem 1512 may also perform fine-tuning to the model using longitudinal data (e.g., biological states captured at different temporal intervals for a same biological state) that has been collected.

[0147] In some embodiments, at least a first biological state 1702 describing a biological system at time T1 and a second biological state 1702 describing the biological system at time T2 may be input to a machine learning model 1710. As illustrated in FIG. 17, machine learning model 1710 may include an encoder 1712, a generator 1714, and a decoder 1716. First biological state 1702 and second biological state 1704 may be input to encoder 1712, which may generate embeddings 1718. In particular, an embedding may be generated foreach input biological state. The embeddings, for example, a first embedding representing first biological state 1702 and a second embedding representing second biological state 1704, may be input to generator 1714. Generator 1714 may predict a future embedding 1720 representing a future biological state of the biological system at a future time T. Future embedding 1720 may serve as input for decoder 1716 to generate future biological state 1706 describing the biological system at future time T.

[0148] FIG. 18 illustrates a flowchart of an example method 1800 for generating a tokenized representation of a biological state, in accordance with various embodiments. In some embodiments, method 1800 may be performed by a subsystem that is the same or similar to subsystem 1510. In some embodiments, method 1800 may begin at step 1802. At step 1802, a first biological state and a second biological state may be determined. The first biological state may describe a biological system at a first time and the second biological state may describe the biological system at a second time. At step 1804, a first embedding representing the first biological state and a second embedding representing the second biological state may be generated using one or more machine learning models. At step 1806, a future embedding representing a future biological state of the biological system at a future time may be predicted using the machine learning models. At step 1808, a prediction of the future biological state at the future time may be generated based on the future embedding.

[0149] In some embodiments, the machine learning techniques that can be used in the systems / subsystems / modules described herein may include, but are not limited to (which is not to suggest that any other list is limiting), any of the following: Ordinary Least Squares Regression (OLSR), Linear Regression, Logistic Regression, Stepwise Regression, Multivariate Adaptive Regression Splines (MARS), Locally Estimated Scatterplot Smoothing (LOESS), Instance-based Algorithms, k-Nearest Neighbor (KNN), Learning Vector Quantization (LVQ), Self-Organizing Map (SOM), Locally Weighted Learning (LWL), Regularization Algorithms, Ridge Regression, Least Absolute Shrinkage and Selection Operator (LASSO), Elastic Net, Least-Angle Regression (LARS), Decision Tree Algorithms, Classification and Regression Tree (CART), Iterative Dichotomizer 3 (ID3), C4.5 and C5.0 (different versions of a powerful approach), Chi- squared Automatic Interaction Detection (CHAID), Decision Stump, M5, Conditional Decision Trees, Naive Bayes, Gaussian Naive Bayes, Causality Networks (CN), Multinomial Naive Bayes, Averaged One-Dependence Estimators (AODE), Bayesian Belief Network (BBN), Bayesian Network (BN), k-Means, k- Medians, K-cluster, Expectation Maximization (EM), Hierarchical Clustering, AssociationRule Learning Algorithms, A-priori algorithm, Eclat algorithm, Artificial Neural Network Algorithms, Perceptron, Back-Propagation, Hopfield Network, Radial Basis Function Network (RBFN), Deep Learning Algorithms, Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN), Convolutional Neural Network (CNN), Deep Metric Learning, Stacked Auto-Encoders, Dimensionality Reduction Algorithms, Principal Component Analysis (PCA), Principal Component Regression (PCR), Partial Least Squares Regression (PLSR), Collaborative Filtering (CF), Latent Affinity Matching (LAM), Cerebri Value Computation (CVC), Multidimensional Scaling (MDS), Projection Pursuit, Linear Discriminant Analysis (LDA), Mixture Discriminant Analysis (MDA), Quadratic Discriminant Analysis (QDA), Flexible Discriminant Analysis (FDA), Ensemble Algorithms, Boosting, Bootstrapped Aggregation (Bagging), AdaBoost, Stacked Generalization (blending), Gradient Boosting Machines (GBM), Gradient Boosted Regression Trees (GBRT), Random Forest, Computational intelligence (evolutionary algorithms, etc.), Computer Vision (CV), Natural Language Processing (NLP), Recommender Systems, Reinforcement Learning, Graphical Models, or separable convolutions (e.g., depth-separable convolutions, spatial separable convolutions).IV. Example Method for Longitudinal Prediction

[0150] An example method for longitudinal prediction is described herein, in accordance with the disclosed methods. Namely, the present example describes the use of principle component analysis (PCA) to reduce the dimensionality of longitudinal gene expression data, e.g., RNA sequencing (RNAseq) data. For example, given an input matrix of time by genes, where the elements of the input matrix describe gene expression levels, PCA can be done across the genes, e.g., the column vectors of the input matrix, to result in an embedding or latent space representation that can be described, for example, as a matrix of time by principle components (PCs). In some instances, the embedding or latent space representation derived by PCA can be used for similar applications or outputs as those based on the embedding derived by an encoder from e.g., an autoencoder, as depicted in embedding 1718, shown in FIG. 17. From the latent space representation, such as a matrix of time by PCs for a subject, a trajectory of the data over time can be observed in PC space, such as the PC space visualized by projecting the RNAseq data onto the first two or first three PCs. The temporal relationship of the projected data in PC space can be visualized by connecting the datapoints in temporal order, e.g., via lines or arrows, as observed in FIG. 19D. The trajectory of the gene expression data in PC space, as it corresponds to the longitudinal RNA seq data, can beused to predict the gene expression profiles of subjects, in response to a stimulus, such as a therapeutic intervention, e.g., a drug, as well as the recovery from the response to the stimulus, e.g., once the effects of the drug wear off the subject. The present example describes the trajectory of gene expression data in PC space, after the subject is exposed to a drug, including the subject’s recovery from the drug. The trajectory of the subject’s gene expression data in PC space, in response to an intervention, e.g., a drug, can be used to predict another subject’s gene expression data in response to the same or even similar type of intervention, e.g., to a drug that is biosimilar to the drug provided for the subject for which the trajectory in PC space is available. The PC space where the trajectory of gene expression data occurs can be considered an embedding, e.g., a first embedding or a second embedding. Each datapoint of the trajectory in the embedding, including an imputed or an extrapolated datapoint, based on the trajectory, can represent a biological state, such as a first biological state or a second biological state. That is, a single embedding can represent multiple biological states, such as first or second biological states, in the sense that a single embedding can express the space in which the biological states change, including changes over time, and / or in response to a therapeutic intervention, such as a drug. The first biological state or a second biological state can correspond to a biological state at a point in time, such as a first time, a first future time, a second time, or a second future time.

[0151] The present example describes longitudinal daily RNA sequencing on rats dosed with tetracyline, isoniazid, carbon tetrachloride, or valproate. 4,302 temporally variable genes (TVGs) with statistically strong change in expression following dosing were identified. Projecting TVG expressions into a 3-dimensional PC space revealed consistent trajectories for recovery following dosing, and enabled separation of healthy from recovering states with 91% to 99% area under the receiver operator curve. Finally, Fast Recovery vs. Slow Recovery rats were observed, and the Fat Recovery vs. Slow Recovery rats exhibited distinct temporal expression patterns in the TVGs, suggesting that individual variations could be potentially captured via longitudinal RNAseq analysis.

[0152] Living organisms are dynamic systems. The RNA, protein, and metabolite concentrations of living systems change in response to changes in the health and environment of those living systems. Of these, RNA expression is an affordable method of comprehensive biological profiling, due to advancements in high throughput next generation sequencing (NGS) technology over the past decade. By comparing the RNA expression profiles of different populations of individuals, persons skilled in the art can identify unique RNAsignatures for a variety of diseases and can use the unique RNA signatures as predictive and prognostic biomarkers for therapy response. This type of cross-sectional analysis allows identification of differentially expressed genes (DEGs) across two groups, but can fail to capture the temporal dynamics of genes affected by perturbations to the individual, and can also risk significant false discoveries due to the noise intrinsic in the RNAseq sample preparation and data collection.

[0153] The present example shows that high frequency longitudinal RNAseq analysis of blood samples can provide a significantly deeper understanding of gene expression temporal dynamics in response to drug perturbations, and can also significantly improve the discovery accuracy of affected genes. In the present example, RNAseq on 829 blood samples collected from 84 Sprague-Dawley lab rats (Rattus norvegicu ) are performed. Other than control aniamls, each rat was each dosed with one of four different small molecules: tetracycline, isoniazid, valproate, and carbon tetrachloride, with known liver toxicity. Custom bioinformatics analysis tools were developed to identify temporally varying genes (TVGs).Experimental Design Summary

[0154] After acclimating the rats to their new environments for 3 days, daily blood draws were initiated, with day 1 being defined as the first day of blood draws (FIG. 19A). Each blood draw was 200 pL; this blood volume was selected as a balance between (1) obtaining sufficient biospecimen for analysis and banking and (2) not inducing undue harm to the rats from the bleeding process. The 200 pL blood volume was decided considering of the potential accumulated effects of daily blood draws, and followed animal ethics approved protocols. To ensure reproducibility of results and minimization of the effects of potential confounding factors, all blood draws were performed at approximately 10:00am local time each day. The relevant rats dosed with the respective drug intravenously on their day 3 or day 8, about 30 minutes prior to the next blood draw. Consequently, the first blood sample after dosing likely did not reflect the full effects of each molecule, based on the pharmacokinetics properties of each molecule. Following the end of each experiment, rat liver necropsy samples were analyzed via standard histopathology.

[0155] RNA was extracted from whole blood within 6 hours of collection. Subsequently, bead-based solid phase enrichment of mRNA was used from the total RNA sample using poly-T probes to remove non-coding RNA and ribosomal RNA. The enriched mRNA sample was then reverse transcribed using random hexamer primers, and the cDNA was sequenced using Illumina NGS at a depth of between 10M and 20M reads per sample. After filteringand alignment, the raw RNA expression profile was obtained, which was subsequently normalized based on the number of reads in each NGS library.

[0156] As a preview of the collected and analyzed data, FIG. 19C shows the temporal expression dynamics for two example genes, Eif4h and lnts8. Both genes showed temporally stable expression for the control rats, but the lnts8 gene shows a 4-fold drop in expression on day 4 for all 3 rats before returning on pre-dose levels on day 5. FIG. 19D shows a projection of the tetracycline-dosed rats' RNA expression on each data in a reduced 2-dimensional principal component parameter space. The trajectory of the rats through this parameter space shows that the rats took roughly 4 days to fully recover from the dose, suggesting that the lnts8 gene is an "early responder" gene. The dynamics of the rats' recovery back to full health follows similar trajectories for all the molecules tested, although important individual variabilities in the magnitude of TVG RNA perturbation and the speed at which the rats recovered, were observed.Results(i) Baseline and Bleeding Effects Characterization

[0157] Before conducting drug dosing experiments, RNAseq on daily blood draw was performed for 21 days on N=3 male and N=3 female rats, to understand technical reproducibility, baseline gene expression variability, and the effects of daily blood draws on rats. This allowed for the subsequent and more confident identifying of the genes for which temporal variations were due to the specific drugs being dosed.

[0158] Simple comparison of expression levels between pairs of rats on the same day confirmed the technical reproducibility of the results. Next, the RNA expression profiles of the 6 rats on Day 2 were compared vs. Day 1. Because Day 1 was the first day that blood was drawn, no bleeding or clotting related gene pathways were expected have been activated. Consequently, Day 1 data acted as true baseline control RNA profiles. The standard DESeq2 analysis tool (Love et al., (2014) Genome Biology. 15(12): 1-21) was used to identify potential differentially expressed genes (DEGs) in Day 2 vs. Day 1 (Fig. 2a). The standard p=0.05 significance cutoff for calling DEGs resulted in many false positives, and was insufficient to identify true DEGs.

[0159] To improve the accuracy of DEG calling, a metric comprising a non-linear combination of adjusted p-value and log2 fold change, with a single fitting parameter k, was developed and used:DEG iff [Abs(log2fold)-1] * Max{[-log(pval) + log(0.05)], 0} > k

[0160] Setting k= \ removed the vast majority of false positive findings from DESeq2 analysis (gray dots in FIG. 20A). Using this approach, 49 up-regulated and 453 down- regulated DEGs in Day 2 were identified. These 502 genes corresponded to roughly 2.1% of all 23,987 rat genes, and roughly 5.6% of the 9,044 rat genes with significant non-zero expression were observed.

[0161] The 21 -day longitudinal blood RNAseq data collected allowed for more sophisticated analysis and identification of TVGs. First, TVGs were identified through DEGs that were identified via pairwise comparison of each day's RNAseq profiles against the day 1 data (FIG. 2B). Unexpectedly, the number of identified DEGs varied by more than 10-fold, depending on the day that was being analyzed. Furthermore, the number of DEGs appeared to follow a smooth pattern with a 2 week period, rather than being independently distributed. Visual analysis of temporal gene expression confirmed the periodic behavior of gene expression for a subset of genes (Tra2a and Map2k3 in FIG. 20D). This finding indicates the first experimental report of such time-dependent gene expression phenomenon at the observed scales, e.g., at scales affecting more than 1000 genes.

[0162] Next, the number of days that each gene's out-day expression significantly varied from day 1 expression (FIG. 20C) was considered. The definition of TVGs being those genes that were called as DEGs with frequency greater than >3 out of 20 (DEGF > 3), based on the 20 pairwise comparisons vs. day 1, were tested. Comparing the genes with DEGF > 3 against the DEGs called from day 2 / day 1 showed that the latter generated many false positives and false negatives (FIG. 20D). False positives could be partially due to measurement noise (e.g. Hsphl), and false negatives could be due to delayed activation or periodic expression genes not significantly different in expression across consecutive days. These findings highlight the importance and contribution of high frequency longitudinal RNAseq collection.

[0163] Visual spot check inspection of the temporal expression profiles of the genes with DEGF > 3 suggested, however, that this approach also generated significant false positives, primarily because Day 1 data was overweighed. After considering and evaluating multiple approaches for defining TVGs, a TVG score based on the p-values of DEG analysis across all pairwise comparisons across different days was decided.TVG Score = Sum_(i > j) Min (Max{[-log(pval(i,j)) + log(0.05)] , 0], 20)where pval(i,j) is the DESeq2 generated p-value of the gene comparing day i vs. day j.

[0164] FIG. 20E shows the contributing p-value components of the Klhl25 gene, and its temporal gene expression profile, and FIG. 20F shows the distribution of TVG scores for all expressed rat genes. Based on visual inspection of the temporal gene expression profile of the 6 rats, the 300 genes with highest TVG scores were decided to be reasonably qualified as true TVGs. All 300 of the TVGs identified in this way exhibited DEGF >3, but only 8 of them were identified as DEGs by comparing day 2 versus day 1 (FIG. 20G). These observations suggest that a large majority of the TVGs identified fluctuate naturally or are delayed responses to bleeding, as opposed to acute bleeding responses. FIG. 20H shows the gene expression profiles of two TVGs. Of these genes, Gvinl was also identified as a day 1 / day 2 DEG and Abca4 was not.(ii) Liver Toxicity Responses

[0165] The rats were dosed with four different small molecules: tetracycline, isoniazid, valproate, and carbon tetrachloride. The first three dosed small molecules are FDA approved drugs, and the last is a formerly used drug (Necatorina) that is now a standard molecule for testing liver damage. The results of the studies suggest that the highest number of TVGs were in response to tetracycline dosing.

[0166] N=3 rats were dosed at each of 4 different concentrations of tetracycline (4, 15,50, and 200 mg / kg), and unsurprisingly, the highest dose elicited the strongest gene expression response. Consequently, the 200mg / kg dataset was used for identifying TVGs, following the same method as those used to generate the results shown in FIGS. 20E and 20F. In this analysis, the 300 bleeding / baseline TVGs that were previously called were excluded, and as a result, 3,206 TVGs associated with tetracycline response were identified (FIG. 21A). Temporal expression profiles for the TVGs Psmb4 and Tpm4, as well as non-TVG Grk6, are shown in FIG. 21B. Tpm4 responded quickly to tetracycline, with the strongest gene downregulation on day 4. Psmb4, in contrast, shows the highest up-regulation on Day 5, suggesting potentially either delayed gene activation or slower deactivation.

[0167] To identify the function of the genes, unsupervised clustering was performed to group the TVGs based on their temporal expression profiles (FIG. 21C). Of these three clusters of TVGs, cluster 2 showed upregulation on days 3-5, suggesting potential role in liver damage recovery. Analysis of the cluster 2 TVGs using ClusterProfiler (Yu el al., (2012) Omics: A Journal of Integrative Biology. 16(5):284-287) showed several groups ofassociated gene pathways. To further confirm that the associated gene pathways were real response signals, rather than potential false positives from testing a large number of gene hypotheses, the impact of tetracycline dose on the magnitude of the gene response was examined (FIG. 21E). Consistent with the up-regulated TVGs contributing to drug response, the up-regulated TVGs were found to have weaker responses at lower doses. Furthermore, the different TVGs were activated at different drug doses. For example, Rps20-psl 1 was up- regulated on day 5 even at the lowest dose of 4mg / kg, but LOC100911177 was up-regulated only at the highest 200mg / kg dose.

[0168] Analyzing the other 3 drug molecules, 1,742 TVGs for isoniazid, 1,233 TVGs for carbon tetrachloride, and 1,068 TVGs for valproate were identified. The union of these TVGs with the 3,206 tetracycline TVGs yielded a total of 4,302 TVGs. Of these, 186 genes were identified as TVGs for all 4 drugs, including the Lrpl gene (FIG. 21F). Given the large number of TVGs identified specific to different subsets of drugs, it is likely that these drugs activated multiple different pathways for recoveries.(iii) Principal Component Space Trajectories

[0169] In addition to the bottom-up analyses thus far, macroscopic insights were obtained from the longitudinal RNA expression data from the top down. The 4,302 TVGs (for any of the four molecules) and data from 482 blood samples (corresponding to the control and the experiments at 200 mg / kg tetracycline, 500 mg / kg valproate, 300 mg / kg isoniazid, and 2000 mg / kg carbon tetrachloride) were subject to PCA (FIG. 22A). Tetracycline, valproate, isoniazid, and carbon tetrachloride, or a combination thereof, can be considered one or more therapeutic interventions. Plotting the trajectory of the three rats dosed with 200mg / kg of tetracycline on the 2-dimensional principal component parameter space (using the top 2 principal components PCI and PC2) revealed highly similar trajectories (colored points in FIG. 22B) that moved to the upper right region, before returning to the healthy region in the lower left. The control rats shown in gray remained in the healthy region throughout the entirety of the 21-day study. The sorted distribution of the linear weights of the 4,302 TVGs for PCI and PC2 are shown in FIG. 22C. Similar trajectories were observed for isoniazid, valproate, and carbon tetrachloride.

[0170] Based on the PC space trajectories, it appears that days 4 and 5 typically provide the most dramatic expression change relative to control, followed by a moderate expression change in day 6. Taking expression from days 4, 5, and 6 as "positives" for drug response perturbation, and all other days as "negatives" for normal health, it was found that there was adividing line separating the positives from the negatives with good sensitivity and specificity. This separating line can be represented as the 0 isocline for a ToxScore that is perpendicular to the separating line:ToxScore = PCI + A * PC2 + B * PC3 + C Predict Positive iff ToxScore > 0

[0171] In FIG. 22D, the projection of the gene expressions into 2-dimensional PC space using only PCI and PC2 are shown; the green dotted separating line shows A=5, B=0, and C=-50. By varying the intercept C, different tradeoffs can be achieved between sensitivity and specificity, generating a receiver operator curve (ROC) for each of the four dosed molecules (FIG. 22E). The area under the ROC curve (AUROC) were moderately high for all four molecules (70%, 83%, 85%, and 100%), with valproate exhibiting the lowest AUROC. By expanding the projection to include PC3 and considering each drug molecule independently, separating hyperplanes that improves the AUROC to between 91% and 99% were found. Of note, the TVG expression data fed into the PCA were not specifically labeled and trained for the task of separating days 4, 5, 6 from the remaining days and the control rats.

[0172] FIG. 22F shows the trajectory of the TVG expression levels for the four drugs in the two dimensional PC1 / PC2 space. Trajectories for all 4 drugs followed a movement to the upper right, followed by movement back to the lower right. The post-dosing TVG expression levels for rats subjected to lower doses of the molecules provided less separation in parameter space (FIG. 22G), consistent with the findings of lowered numbers of TVGs as shown in FIG. 21E.

[0173] Because all four of the molecules dosed are known to cause liver toxicity, these results suggested that the ToxScore could be potentially used as a universal diagnostic biomarker of liver damage response. To test this hypothesis, the ToxScore versus the traditional liver biomarker aspartate aminotransferase (AST) values (Schomaker el al., (2012) Toxicological Sciences. 132(2):276-283) were compared for the rats from each blood sample (Fig. 22H). There appeared to be a weak correlation between ToxScore and AST, driven primarily by the large number of blood samples with high ToxScore but low AST levels. Additional longitudinal studies dosing animal models with additional liver toxic and non-liver toxic molecules could potentially validate a refined version of ToxScore based on TVGs as a more sensitive liver biomarker than AST.(iv) Individual Variability in Drug Response

[0174] In the first set of experiments dosing the rats with 500mg / kg valproate, 1 of the 3 rats died within one day of dosing and the other two recovered after 1 day and 2 days. The variability in the outcome occurred despite the rats' similarity in age (10 weeks), weight (220- 240g), and sex (male) of the rats. To confirm that this result was not an operator technical error, a follow-up experiment was performed using an additional 15 male rats, all dosed with 500 mg / kg. Additionally, to confirm that minor differences in age were not the cause of the divergent phenotypic results, 3 different ages for the rats were selected: 3-4 weeks, 8 weeks, and 8-9 months (FIG. 23 A). The outcomes of the experiments confirmed the individual variability in the outcomes (FIG. 23B): 2 / 15 rats died 1 and 2 days after valproate injection (Death group), 9 / 15 rats showed phenotypic changes on day 8 (Slow Recovery group), and 4 / 15 rats exhibited no symptoms (Fast Recovery group).

[0175] Given these results, whether the differences in phenotypic outcomes could be recapitulated through the valproate TVG genes were assessed. For ease of data analysis, a new PCA on only the 1,068 valproate TVGs was performed, and new PCI and PC2 weights were obtained only on these genes. The expression trajectories of the Slow Recovery group deviated significantly further on day 9 and day 10 from the healthy region of the PC1 / PC2 parameter space, compared to the Fast Recovery group. This result suggested that the phenotypic symptoms could be effectively captured through blood TVG expression. FIG. 23D showed the different temporal expression profiles of the Sema4f gene for the Fast Recovery and Slow Recovery groups.

[0176] The Fast Recovery and Slow Recovery animals appeared in slightly different subregions of the healthy region in PC1 / PC2 parameter space. To further analyze potential pre-dosing gene expression differences that may be prognostic of phenotypic outcomes, DESeq2 analysis on the full RNA expression profiles from Day 7 were performed, just before the animals were dosed (FIG. 23E). 546 DEGs comparing the Slow Recovery and Fast Recovery groups were identified and 106 DEGs comparing the Death and Fast Recovery groups were identified. Some of the identified DEGs may reflect age-based differences in expression.Discussion

[0177] Through the course of this set of experiments, roughly 5Tb of Fastq NGS data from 829 RNAseq samples were collected. To facilitate analysis of this large longitudinal RNA expression data, several ad hoc bioinformatics methods were developed and utilized to (1) minimize the number of false positive DEGs called from the large number of potential pairwise differential expression analyses, and (2) identify TVGs from longitudinal data. Although these tools have not be rigorously optimized for performance, spot check evaluation results suggests that the TVG calling is reasonably accurate and catches a significant number of false positive DEGs called based on cross-sectional tools.

[0178] Several qualitatively unexpected findings were made. First, even in the control group, there were roughly 300 TVGs, and many of these were part of what appeared to be an oscillatory expression pattern with a 2 week period. Second, many TVGs were activated or repressed at different drug dose thresholds, suggesting that rat biology can include multiple layers of response mechanisms activated at different levels of toxicity severity. Third, the relatively small number of genes that were called as TVGs for all 4 drugs suggested that there may be multiple separate recovery or detoxication mechanism biologically in rats. Finally, the same drug dose can elicit very different phenotypic outcomes (including Death, Slow Recovery, and Fast Recovery) in different individuals, and that these phenotypic outcomes could be reflected in the degree and duration of response in TVGs. Importantly, all of these observations required high frequency longitudinal data, and were not visible from a single before / after comparison. The findings in the present example, advocate for broader adoption of high frequency longitudinal RNA expression profiling.

[0179] In addition to biological findings, the results of the present example also suggest the strong potential of longitudinal RNAseq to serve as molecular diagnostic tests. The ToxScore metric, a linear combination of the top 3 principal components from PCA provided drug response AUROC values between 91% and 99% for the 4 drug molecules. One immediate application may be in the prognosis of individualized patient responses to drugs with known liver toxicity, to inform personalized therapy selection or dosage.

[0180] Eongitudinal RNA expression studies can be costly to conduct because the large number of RNAseq libraries that need to be constructed and sequenced. In the experiments shown in the present examplee, 10 or more daily blood samples per animal were typically taken, corresponding to 5x more samples than a simple before / after dosing analysis. New technologies and methods for improving the total cost of longitudinal RNAseq studies wouldbe greatly synergistic with and beneficial to the adoption of longitudinal RNAseq-based diagnostics.

[0181] The data analysis methods used in the present example are primarily human- driven, using traditional tools based on linear combinations of gene expression levels. Given the depth and complexity of RNA expression data, it is reasonable to assume that a well- trained deep neural network would be able to better utilize the "long tail" of information inherent within gene expression variations that are below our stringent cutoffs needed to avoid false positives. Deep neural networks, such as those based on transformers, can use a large amount of data to achieve performance significantly superior to traditional machine learning models. In our dataset of 829 samples, roughly 9,000 genes were expressed at nonzero levels in each sample, so the total size of this dataset is roughly 7.46 million tokens.7.46M tokens is large for biological datasets, but just a small early step in comparison to large language models (LLMs).V. Computer System

[0182] FIG. 24 illustrates an example computer system 2400. In some embodiments, one or more computer systems 2400 perform one or more steps of one or more methods described or illustrated herein. In some embodiments, one or more computer systems 2400 provide functionality described or illustrated herein. In some embodiments, software running on one or more computer systems 2400 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein.Particular embodiments include one or more portions of one or more computer systems 2400. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate.

[0183] This disclosure contemplates any suitable number of computer systems 2400. This disclosure contemplates computer system 2400 taking any suitable physical form. As example and not by way of limitation, computer system 2400 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these. Where appropriate, computer system 2400 may include one or more computer systems 2400; be unitary or distributed; spanmultiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 2400 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example, and not by way of limitation, one or more computer systems 2400 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems 2400 may perform at various times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

[0184] In some embodiments, computer system 2400 includes a processor 2402, memory 2404, storage 2406, an input / output (I / O) interface 2408, a communication interface 2410, and a bus 2412. Although this disclosure describes and illustrates a particular computer system having a particular number of components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0185] In some embodiments, processor 2402 includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor 2402 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 2404, or storage 2406; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 2404, or storage 2406. In some embodiments, processor 2402 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 2402 including any suitable number of any suitable internal caches, where appropriate. As an example, and not by way of limitation, processor 2402 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 2404 or storage 2406, and the instruction caches may speed up retrieval of those instructions by processor 2402. Data in the data caches may be copies of data in memory 2404 or storage 2406 for instructions executing at processor 2402 to operate on; the results of previous instructions executed at processor 2402 for access by subsequent instructions executing at processor 2402 or for writing to memory 2404 or storage 2406; or other suitable data. The data caches may speed up read or write operations by processor 2402. The TLBs may speed up virtual-address translation for processor 2402. In some embodiments, processor2402 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 2402 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 2402 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 2402. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0186] In some embodiments, memory 2404 includes main memory for storing instructions for processor 2402 to execute or data for processor 2402 to operate on. As an example, and not by way of limitation, computer system 2400may load instructions from storage 2406 or another source (such as, for example, another computer system 2400) to memory 2404. Processor 2402 may then load the instructions from memory 2404 to an internal register or internal cache. To execute the instructions, processor 2402 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 2402 may write one or more results (which may be intermediate or final) to the internal register or internal cache. Processor 2402 may then write one or more of those results to memory 2404. In some embodiments, processor 2402 executes only instructions in one or more internal registers or internal caches or in memory 2404 (as opposed to storage 2406 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 2404 (as opposed to storage 2406 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple processor 2402 to memory 2404. Bus 2412 may include one or more memory buses, as described below. In some embodiments, one or more memory management units (MMUs) reside between processor 2402 and memory 2404 and facilitate access to memory 2404 requested by processor 2402. In some embodiments, memory 2404 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory 2404 may include one or more memories 2404, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

[0187] In some embodiments, storage 2406 includes mass storage for data or instructions. As an example, and not by way of limitation, storage 2406 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetictape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage 2406 may include removable or non-removable (or fixed) media, where appropriate. Storage 2406 may be internal or external to computer system 2400, where appropriate. In some embodiments, storage 2406 is non-volatile, solid-state memory. In some embodiments, storage 2406 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage 2406 taking any suitable physical form. Storage 2406 may include one or more storage control units facilitating communication between processor 2402 and storage 2406, where appropriate. Where appropriate, storage 2406 may include one or more storages 2406. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0188] In some embodiments, RO interface 2408 includes hardware, software, or both, providing one or more interfaces for communication between computer system 2400and one or more RO devices. Computer system 2400may include one or more of these RO devices, where appropriate. One or more of these RO devices may enable communication between a person and computer system 2400. As an example, and not by way of limitation, an RO device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable RO device, or a combination of two or more of these. An RO device may include one or more sensors. This disclosure contemplates any suitable RO devices and any suitable RO interfaces 2408 for them. Where appropriate, RO interface 2408 may include one or more device or software drivers enabling processor 2402 to drive one or more of these RO devices. RO interface 2408 may include one or more RO interfaces 2408, where appropriate. Although this disclosure describes and illustrates a particular RO interface, this disclosure contemplates any suitable RO interface.

[0189] In some embodiments, communication interface 2410 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 2400 and one or more other computer systems 2400 or one or more networks. As an example, and not by way of limitation, communication interface 2410 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or awireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 2410 for it. As an example, and not by way of limitation, computer system 2400 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 2400 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WLMAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Computer system 2400may include any suitable communication interface 2410 for any of these networks, where appropriate. Communication interface 2410 may include one or more communication interfaces 2410, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.

[0190] In some embodiments, bus 2412 includes hardware, software, or both coupling components of computer system 2400 to each other. As an example and not by way of limitation, bus 2412 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI- Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus 2412 may include one or more buses 2412, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0191] Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field-programmable gate arrays (FPGAs) or application- specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs),magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.

[0192] The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, biological state feature, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, biological state features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend.Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.EXAMPLES

[0193] Some examples of the machine learning models performance for predicting unknown biological state features are described. The metrics used to evaluate the performance of the machine learning model, such as machine learning model 310 of FIG. 3, indicate how well the model can predict gene expression changes resulting from masking (or perturbations). Some example metrics include the Mean Squared Error (MSE) and the Pearson Correlation Coefficient (Pearson). The MSE measures the average squared difference between the predicted expression values and the actual expression values. A “lower” MSE indicates that the predicted changes are closer to the actual changes. The Pearson coefficient measures the linear correlation between two sets of data. The Pearsoncoefficient ranges from -1 (perfect negative correlation) to +1 (perfect positive correlation), with 0 indicating no linear correlation. Using the techniques described herein, in the context of RNAseq masking(perturbation) prediction, a Pearson coefficient of 0.82 was obtained, indicating a very strong positive linear correlation between the predicted and actual gene expression values. The MSE was also low, indicating accurate predictions of overall gene expression value changes. In particular, the Pearson coefficient value suggests that there is a strong positive linear correlation between the predicted and actual gene expression value changes, indicating that the model's predictions are in line with the actual data.

[0194] {'mse': 0.00633879785470548,

[0195] 'pearson': 0.8201751059128802}

[0196] Some examples of the tokenization process are also described. The tokenization process, as mentioned above with respect to FIG. 10, for example, provides an improved technique for training machine learning models to predict gene expression values or other biological state features. For example, models trained with the tokenization method described herein can predict which genes are expressed at what level of expression in a cell based on limited expression value information with over 88% accuracy. Additional downstream tasks, such as cell classification or chronological development prediction, can alsoEXAMPLE EMBODIMENTS

[0197] Embodiments disclosed herein may include:

[0198] Al. A method for generating a prediction of a future biological state of a biological system, the method comprising: determining a first biological state and a second biological state, wherein the first biological state describes the biological system at a first time and the second biological state describes the biological system at a second time; generating, using one or more machine learning models, a first embedding representing the first biological state at the first time and a second embedding representing the second biological state at the second time; predicting, using the one or more machine learning models, a future embedding representing a future biological state of the biological system at a future time; and generating, using the one or more machine learning models, the prediction of the future biological state describing the biological system at the future time based on the future embedding.

[0199] A2.The method of example Al, further comprising: determining, using the one or more machine learning models, a prediction of a disease state of the biological system at the future time based on the prediction of the future biological state of the biological system.

[0200] A3. The method of example A2, further comprising: determining one or more therapeutic interventions for the biological system based on the prediction of the disease state of the biological system.

[0201] A4.The method of any one of examples A1-A3, wherein the one or more machine learning models comprise an encoder, a generator, and a decoder.

[0202] A5.The method of example A4, wherein generating the first embedding and the second embedding comprises: inputting the first biological state and the second biological state into the encoder to generate the first embedding and the second embedding.

[0203] A6.The method of example A4 or example A5, wherein predicting the future embedding comprises: inputting the first embedding and the second embedding into the generator to generate the future embedding.

[0204] A7.The method of example A6, wherein an indication of the future time is input to the generator with the first embedding and the second embedding to generate the future embedding.

[0205] A8.The method of any one of examples A4-A7, wherein generating the prediction of the future biological state comprises: inputting the future embedding into the decoder to generate the prediction of the future biological state of the biological system at the future time.

[0206] A9.The method of any one of examples A1-A8, wherein the first biological state, the second biological state, and the future biological state are each an n-di mcnsional feature vector.

[0207] A10. The method of any one of examples A1-A9, wherein the first biological state, the second biological state, and the future biological state are each represented using a plurality of biological state features.

[0208] Al l. The method of example A 10, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least oneof: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

[0209] A 12. The method of any one of example A10 or Al 1, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

[0210] A13. The method of example A12, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of the biological system; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system.

[0211] A14. The method of example A13, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

[0212] A15. The method of example A 14, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

[0213] A 16. The method of any one of examples Al -A 15, wherein the one or more machine learning models comprise one or more first machine learning models, the first biological state comprises a first plurality of biological state features and the second biological state comprises a second plurality of biological state features, wherein at least one of: one or more of the first plurality of biological state features or one or more of the second plurality of biological state features are predicted using one or more second machine learning models.

[0214] A17. The method of example A16, wherein the first plurality of biological state features comprises one or more first known biological state features and the second plurality of biological state features comprises one or more second known biological state features.

[0215] Al 8. The method of any one of examples Al -A 16, further comprising: generating a first tokenized representation of the first biological state and a second tokenized representation of the second biological state, wherein the first tokenized representation and the second tokenized representation are input to the one or more machine learning models to generate the first embedding and the second embedding, respectively.

[0216] A 19. The method of example A 18, wherein the first tokenized representation comprises a first plurality of biological state features representing the first biological state and the second tokenized representation comprises a second plurality of biological state features represented second biological state, and wherein the first tokenized representation structure the first plurality of biological state features in an order determine based on an expression value of each of the first plurality of biological state features, and wherein the second tokenized representation structure the second plurality of biological state features in an order determine based on an expression value of each of the second plurality of biological state features.

[0217] A20. The method of example A 19, wherein the order of the first plurality of biological state features and the order of the second plurality of biological state features is from low expression value to high expression value or from high expression value to low expression value.

[0218] A21. The method of any one of examples A19-A20, wherein each biological state feature of the first plurality of biological state features and each biological state feature of the second plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype is expressed within a biological sample of the biological system; and at least one conditional attribute associated with the biological state feature.

[0219] A22. The method of example A21, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

[0220] A23. The method of any one of examples A1-A22, wherein the one or more machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

[0221] A24. The method of any one of examples A1-A23, wherein the future embedding representing the future biological state of the biological system at the future time comprises a first future embedding representing a first future biological state of the biological system at a first future time, the method further comprises: predicting, using the one or more machine learning models, a second future embedding representing a second future biological state of the biological system at a second future time; and generating, using the one or more machine learning models, a second prediction of the second future biological state describing the biological system at the second future time based on the second future embedding.

[0222] A25. The method of example A24, further comprising: determining, using the one or more machine learning models, a first prediction of a first disease state of the biological system at the first future time based on the first prediction of the first future biological state of the biological system; and determining, using the one or more machine learning models, a second prediction of a second disease state of the biological system at the second future time based on the second prediction of the second future state of the biological system.

[0223] A26. The method of example A25, further comprising: determining one or more therapeutic interventions for the biological system based on at least one of the first prediction of the first disease state of the biological system or the second prediction of the second disease state of the biological system.

[0224] A27. The method of any of examples A1-A26, further comprising: training the one or more machine learning models using training data comprising: for each of a plurality of biological systems: a first biological state describing the biological system at a first time and a second biological state describing the biological system at a second time; a first embedding representing the first biological state and a second embedding representing the second biological state; one or more future embeddings representing one or more future biological states of the biological system at one or more future times; andthe one or more future biological states of the biological system at the one or more future times.

[0225] B 1. A method for generating a tokenized representation of a biological state for a machine learning model, the method comprising: obtaining a biological state describing a biological system, wherein the biological state comprises a plurality of biological state features each having an expression value; for each of a subset of the plurality of biological state features: computing a logarithmic expression value of the biological state feature based on the expression value; and discretizing the logarithmic expression value into a discretized expression value; and generating a tokenized representation of the biological state comprising the plurality of biological state features structured using an ordering function based on the discretized expression value of each of the subset of the plurality of biological state features.

[0226] B2.The method of example Bl, wherein the biological state comprises a n- dimensional feature vector.

[0227] B3.The method of example B2, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

[0228] B4.The method of any one of examples B1-B3, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

[0229] B5.The method of example B4, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the sample of the biological system; andeach biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system.

[0230] B6.The method of example B4 or B5, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

[0231] B7.The method of example B6, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

[0232] B8.The method of example B7, wherein the plurality of biological state features comprises one or more unknown biological state features and one or more known biological state features.

[0233] B9.The method of example B8, wherein the one or more known biological state features comprise at least one of: one or more predicted biological state features or one or more measured biological state features.

[0234] B10. The method of example B8 or B9, wherein the one or more unknown biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the second set of biological state features, or one or more biological state features from the third set of biological state features.

[0235] B 11. The method of any one of examples B8-B 10, wherein the one or more unknown biological state features comprise at least one biological state feature from the fourth set of biological state features.

[0236] B12. The method of any one of examples Bl-Bl l, wherein obtaining the biological state describing the biological system comprises: receiving expression data comprising a plurality of biological samples obtain for the biological system, wherein each of the plurality of biological samples is represented using a sample- specific biological state; and combining the sample- specific biological state of each of the plurality of samples to obtain a combined biological state, wherein the biological state comprising the subset of the plurality of biological state features is determined based on the combined biological state.

[0237] B13. The method of example B12, wherein the sample- specific biological state of each of the plurality of biological samples is represented using a plurality of samplespecific biological state features each having a sample-specific expression value, combining comprises: averaging the sample- specific expression value of each of the plurality of samplespecific biological state features to obtain an average expression value, wherein the expression value of each of the subset of the plurality of features comprises the average expression value.

[0238] B 14. The method of example B 13, further comprising: for each of the plurality of sample- specific biological state features: determining whether the average expression value satisfies a threshold condition; and one of: removing the sample- specific biological state feature from the plurality of sample- specific biological state features based on the average expression value of the sample- specific biological state feature failing to satisfy the threshold condition; or keeping the sample- specific biological state feature included in the plurality of features based on the average expression value of the sample- specific biological state feature satisfying the threshold condition.

[0239] B15. The method of example B14, wherein the subset of the plurality of biological state features is formed based on whether a corresponding sample- specific biological state feature of the sample- specific plurality of features was removed or kept.

[0240] B16. The method of example B14 or B15, wherein the average expression value satisfying the threshold condition comprises the average expression value being greater than or equal to a threshold expression value.

[0241] B 17. The method of any one of examples B 14-B 16, wherein the average expression value failing to satisfy the threshold condition comprises the average expression value being less than a threshold expression value.

[0242] Bl 8. The method of example B 16 or B 17, wherein the threshold expression value is an adjustable parameter of the machine learning model.

[0243] B19. The method of example B 16 or B 17, wherein the threshold expression value is about 1.0.

[0244] B20. The method of any one of examples B 1-B 19, wherein the biological state is derived from RNA sequencing data obtained from the biological system.

[0245] B21. The method of any one of examples B1-B20, wherein the plurality of biological state features comprises a set of biological state features each corresponding to a gene of a set of genes.

[0246] B22. The method of example B21, wherein each gene of the set of genes comprises an identifier for identifying the gene in one or more biological databases.

[0247] B23. The method of example B21 or B22, wherein the set of genes includes between about 10A3 and about 10A6 genes.

[0248] B24. The method of any one of examples B 1-B23, wherein the plurality of biological state features comprise: 10A6 or more biological state features, 10A7 or more biological state features, 10A8 or more biological state features, or 10A9 or more biological state features.

[0249] B25. The method of any one of examples B 1-B24, wherein the ordering function indicates an order for the plurality of biological state features included by the tokenized representation.

[0250] B26. The method of example B25, wherein the order is ascending or descending.

[0251] B27. The method of any one of examples B 1-B26, wherein the biological state is derived from a biological sample of the biological system, wherein the biological sample comprises bulk RNA sequencing data or single cell RNA sequencing data.

[0252] B28. A method for generating training data for training a machine learning model to predict a biological state of a biological system, the method comprising: retrieving a plurality of tokenized representations each describing a biological system of a plurality of biological systems, wherein a biological sample is obtained from each of the plurality of biological systems to determine the biological state of the biological system, and wherein each tokenized representation is generated using the method of any one of examples B1-B27; and generating training data using the plurality of tokenized representations.

[0253] B29. A method for training a machine learning model to predict a biological state of a biological system, the method comprising: training a machine learning model using the generated training data of exampleB21.

[0254] B30. The method of B29, the machine learning model comprises at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

[0255] Cl. A method for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state, the method comprising: inputting a plurality of biological state features of the biological state into one or more machine learning models to generate an embedding representing the biological state, wherein the plurality of biological state features comprise the one or more known biological state features and the one or more unknown biological state features; and predicting, using the one or more machine learning models, at least one of the one or more unknown biological state features based on the embedding.

[0256] C2.The method of example Cl, wherein the biological state comprises a n- dimensional feature vector.

[0257] C3.The method of example C2, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

[0258] C4.The method of any one of examples C1-C3, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

[0259] C5.The method of example C4, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markersrepresented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system.

[0260] C6.The method of example C4 or C5, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

[0261] C7.The method of example C6, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

[0262] C8.The method of any one of examples C4-C7, wherein the one or more unknown biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the second set of biological state features, or one or more biological state features from the third set of biological state features.

[0263] C9.The method of example C6, wherein the one or more unknown biological state features comprise at least one biological state feature from the fourth set of biological state features.

[0264] CIO. The method of any one of examples C1-C9, further comprising: generating a tokenized representation of the biological state, wherein the tokenized representation is input to the one or more machine learning models to predict the at least one of the one or more unknown biological state features.

[0265] Cl 1. The method of example CIO, wherein the tokenized representation comprises the plurality of biological state features, including the one or more known biological state features and the one or more unknown biological state features, structured based on an expression value of each of the plurality of biological state features.

[0266] C12. The method of example Cl 1, wherein the plurality of biological state features are structured from low expression value to high expression value or from high expression value to low expression value.

[0267] C13. The method of any one of examples Cl -Cl 2, wherein each biological state feature of the plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype isexpressed within a biological sample of a biological system; and at least one conditional attribute associated with the biological state feature.

[0268] C14. The method of example C13, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

[0269] C15. The method of any one of examples Cl -Cl 4, wherein the one or more machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

[0270] Cl 6. The method of any one of examples Cl -Cl 4, wherein the prediction of the at least one of the one or more unknown biological state features comprises at least one of: an intra-omic prediction, an inter-omic prediction, or an inter-phenotype prediction.

[0271] C17. The method of any one of examples C1-C16, wherein the biological state describes a disease state of a biological system, the biological system comprising a subject, and the one or more known biological state features are measured using a biological sample obtained from the subject.

[0272] Cl 8. A method for training a machine learning model to predict one or more unknown biological state features, the method comprising: obtaining a plurality of measured biological states each represented by a plurality of biological state features; and for each of the plurality of measured biological states: masking one or more biological state features from the plurality of biological state features; inputting the plurality of biological state features including the one or more masked biological state features to a machine learning model, the machine learning model outputting a predicted biological state, wherein the predicted biological state comprises a plurality of predicted biological state features; computing a loss based on the plurality of biological state features and the plurality of predicted biological state features; and adjusting one or more parameters of the machine learning model based on the computed loss until a threshold training condition is satisfied.

[0273] Cl 9. The method of example Cl 8, wherein masking the one or more masked biological state features comprises: obtaining a selection of a set of biological state features to mask, wherein the oneor more masked biological state features of each measured biological state corresponds to the set of biological state features.

[0274] C20. The method of example C18 or C19, wherein inputting the plurality of biological state features including the one or more masked biological state features to the machine learning model comprises: for each of the plurality of measured biological states: inputting the plurality of biological state features including the one or more masked features into the encoder to obtain an embedding; and determining, using the decoder, the plurality of predicted biological state features.

[0275] C21. The method of any one of examples C18-C20, wherein the machine learning model comprises a first machine learning model, the one or more masked biological state features comprise one or more first masked biological state features, the predicted biological state comprises a first predicted biological state, the plurality of predicted biological state features comprises a first biological state features, the loss comprises a first loss, and the one or more parameters comprise one or more first parameters of the first machine learning model, the method further comprises: for each of the plurality of measured biological states: masking one or more second biological state features from the plurality of biological state features; inputting the plurality of biological state features including the one or more second masked biological state features to a second machine learning model to obtain a second predicted biological state, wherein the second predicted biological state comprises a second plurality of predicted biological state features; and computing a second loss based on the second plurality of biological state features and the second plurality of predicted biological state features; and adjusting one or more parameters of the second machine learning model based on the second computed loss until the threshold training condition is satisfied.

[0276] C22. The method of example C21, wherein an output of the first machine learning model comprises a first plurality of predicted biological states respectively corresponding to the plurality of measured biological states and an output of the second machine learning model comprises a second plurality of predicted biological states respectively corresponding to the plurality of measured biological states, the method furthercomprises: generating a first attention map using the first plurality of predicted biological states; determining a first relationship map representing first relationships between the plurality of biological state features using the first attention map; generating a second attention map using the second plurality of predicted biological states; and determining a second relationship map representing second relationships between the plurality of biological state features using the second attention map; and combining the first relationship map and the second relationship map to generate a gene regulatory network.

[0277] C23. The method of any one of examples C18-C22, wherein the plurality of predicted biological state features comprise predictions for the one or more masked biological state features.

[0278] C24. The method of example C23, wherein the plurality of predicted biological state features further comprise predictions of one or more unmasked biological state features.

[0279] C25. The method of any one of examples C18-C24, wherein each of the plurality of measured biological states is associated with a different biological system of a plurality of biological systems.

[0280] C26. The method of example C25, wherein, for each of the plurality of measured biological states, each feature of the plurality of biological state features comprises a measured expression value indicating a level with which a biological marker associated with the feature is expressed within a biological sample of a corresponding biological system.

[0281] C27. The method of any one of examples C18-C26, further comprising: for each of the plurality of measured biological states: generating a tokenized representation of the measured biological state, wherein the tokenized representation is input to the machine learning model, wherein the machine learning model outputs the plurality of predicted biological state features.

[0282] C28. The method of example C27, wherein the tokenized representation comprises the plurality of biological state features structured based on an expression value of each of the plurality of biological state features.

[0283] C29. The method of example C28, wherein the plurality of biological state features are structured from low expression value to high expression value or from high expression value to low expression value.

[0284] C30. The method of any one of examples C18-C29, wherein each measured biological state of the plurality of measured biological states comprises a n-dimcnsional feature vector, wherein the plurality of biological state features are stored using the n- dimensional feature vector.

[0285] C31. The method of example C30, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

[0286] C32. The method of any one of examples C18-C31, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

[0287] C33. The method of example C32, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system described by the measured biological state; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system.

[0288] C34. The method of example C32 or C33, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

[0289] C35. The method of example C34, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

[0290] C36. The method of any one of examples C32-C35, wherein the one or more masked biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the second set of biological state features, or one or more biological state features from the third set of biological state features.

[0291] C37. The method of example C35, wherein the one or more masked biological state features comprise at least one biological state feature from the fourth set of biological state features.

[0292] C38. The method of any one of examples C18-C37, wherein each biological state feature of the plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype is expressed within a biological sample of a biological system described by the measured biological state; and at least one conditional attribute associated with the biological state feature.

[0293] C39. The method of example C38, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

[0294] C40. The method of any one of examples C18-C39, wherein the machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

[0295] C41. The method of example C40, wherein the machine learning model comprises an encoder and decoder, wherein inputting comprises: inputting the plurality of biological state features including the one or more masked biological state feature to the encoder, the encoder outputting an embedding representing the plurality of biological state features; and inputting the embedding to the decoder, the decoder outputting plurality of predicted biological state features.

[0296] C42. The method of any one of examples C18-C41, wherein the prediction of the plurality of predicted biological state features comprises at least one of: an intra-omic prediction, an inter-omic prediction, or an inter-phenotype prediction.

[0297] C43. The method of any one of examples C18-C42, wherein the biological system comprises a subject, and the plurality of known biological state features are measured using a biological sample obtained from the subject.

[0298] DI. A system, comprising: memory storing computer program instructions; and one or more processors that execute the computer program instructions to perform the method of any one of examples A1-A27, B1-B30, or C1-C43.

[0299] D2. A non-transitory computer-readable medium storing computer program instructions that, when executed by one or more processors, effectuate operations comprising the method of any one of examples A1-A27, B1-B30, or C1-C43.

[0300] El . A method for generating a tokenized representation of a biological state for a machine learning model, the method comprising: obtaining a biological state describing a biological system, wherein the biological state comprises a plurality of biological state features each having an expression value; for each of a subset of the plurality of biological state features: computing a logarithmic expression value of the biological state feature based on the expression value; and discretizing the logarithmic expression value into a discretized expression value; and generating a tokenized representation of the biological state comprising the plurality of biological state features structured using an ordering function based on the discretized expression value of each of the subset of the plurality of biological state features.

[0301] E2. The method of example El, wherein the biological state comprises a n- dimensional feature vector.

[0302] E3. The method of example E2, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

[0303] E4. The method of any one of examples E1-E3, wherein the plurality of biological state features comprise at least one of: a first set of biological state featurescorresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

[0304] E5. The method of example E4, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the biological sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the biological sample of the biological system.

[0305] E6. The method of example E4 or E5, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

[0306] E7. The method of example E6, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

[0307] E8. The method of example E7, wherein the plurality of biological state features comprises one or more unknown biological state features and one or more known biological state features.

[0308] E9. The method of example E8, wherein the one or more known biological state features comprise at least one of: one or more predicted biological state features or one or more measured biological state features.

[0309] E10. The method of example E8 or E9, wherein the one or more unknown biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the secondset of biological state features, or one or more biological state features from the third set of biological state features.

[0310] El l. The method of any one of examples E8-E10, wherein the one or more unknown biological state features comprise at least one biological state feature from the fourth set of biological state features.

[0311] E12. The method of any one of examples El-El l, wherein obtaining the biological state describing the biological system comprises: receiving expression data comprising a plurality of biological samples obtain for the biological system, wherein each of the plurality of biological samples is represented using a sample- specific biological state; and combining the sample- specific biological state of each of the plurality of samples to obtain a combined biological state, wherein the biological state comprising the subset of the plurality of biological state features is determined based on the combined biological state.

[0312] E13. The method of example E12, wherein the sample- specific biological state of each of the plurality of biological samples is represented using a plurality of samplespecific biological state features each having a sample-specific expression value, combining comprises: averaging the sample- specific expression value of each of the plurality of samplespecific biological state features to obtain an average expression value, wherein the expression value of each of the subset of the plurality of biological state features comprises the average expression value.

[0313] E14. The method of example E13, further comprising: for each of the plurality of sample- specific biological state features: determining whether the average expression value satisfies a threshold condition; and one of: removing the sample- specific biological state feature from the plurality of sample- specific biological state features based on the average expression value of the sample- specific biological state feature failing to satisfy the threshold condition; or keeping the sample- specific biological state feature included in the plurality of sample- specific biological state features based on the average expression value of the sample- specific biological state feature satisfying the threshold condition.

[0314] E15. The method of example E14, wherein the subset of the plurality of biological state features is formed based on whether a corresponding sample- specific biological state feature of the plurality of sample- specific biological state features was removed or kept.

[0315] El 6. The method of example E14 or E15, wherein the average expression value satisfying the threshold condition comprises the average expression value being greater than or equal to a threshold expression value.

[0316] E17. The method of any one of examples E14-E16, wherein the average expression value failing to satisfy the threshold condition comprises the average expression value being less than a threshold expression value.

[0317] E18. The method of example E16 or E17, wherein the threshold expression value is an adjustable parameter of the machine learning model.

[0318] E19. The method of example E16 or E17, wherein the threshold expression value is about 1.0.

[0319] E20. The method of any one of examples El -El 9, wherein the biological state is derived from RNA sequencing data obtained from the biological system.

[0320] E21. The method of any one of examples E1-E20, wherein the plurality of biological state features comprises a set of biological state features each corresponding to a gene of a set of genes.

[0321] E22. The method of example E21, wherein each gene of the set of genes comprises an identifier for identifying the gene in one or more biological databases.

[0322] E23. The method of example E21 or E22, wherein the set of genes includes between about 10A3 and about 10A6 genes.

[0323] E24. The method of any one of examples E1-E23, wherein the plurality of biological state features comprise: 10A6 or more biological state features, 10A7 or more biological state features, 10A8 or more biological state features, or 10A9 or more biological state features.

[0324] E25. The method of any one of examples E1-E24, wherein the ordering function indicates an order for the plurality of biological state features included by the tokenized representation.

[0325] E26. The method of example E25, wherein the order is ascending or descending.

[0326] E27. The method of any one of examples E1-E26, wherein the biological state is derived from a biological sample of the biological system, wherein the biological sample comprises bulk RNA sequencing data or single cell RNA sequencing data.

[0327] E28. A method for generating training data for training a machine learning model to predict a biological state of a biological system, the method comprising: retrieving a plurality of tokenized representations each describing a biological system of a plurality of biological systems, wherein a biological sample is obtained from each of the plurality of biological systems to determine the biological state of the biological system, and wherein each tokenized representation is generated using the method of any one of examples E1-E27; and generating training data using the plurality of tokenized representations.

[0328] E29. A method for training a machine learning model to predict a biological state of a biological system, the method comprising: training a machine learning model using the generated training data of example E27.

[0329] E30. The method of E29, the machine learning model comprises at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

[0330] E31. A system, comprising: memory storing computer program instructions; and one or more processors that execute the computer program instructions to perform the method of any one of examples E1-E30.

[0331] E32. A non-transitory computer-readable medium storing computer program instructions that, when executed by one or more processors, effectuate operations comprising the method of any one of examples E1-E30.

[0332] Fl. A method for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state, the method comprising: inputting a plurality of biological state features of the biological state into one or more machine learning models to generate an embedding representing the biological state, wherein the plurality of biological state features comprise the one or more known biological state features and the one or more unknown biological state features; and predicting, using the one or more machine learning models, at least one of the one or more unknown biological state features based on the embedding.

[0333] F2. The method of example Fl, wherein the biological state comprises a n- dimensional feature vector.

[0334] F3. The method of example F2, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

[0335] F4. The method of any one of examples F1-F3, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

[0336] F5. The method of example F4, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the biological sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the biological sample of the biological system.

[0337] F6. The method of example F4 or F5, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

[0338] F7. The method of example F6, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

[0339] F8. The method of any one of examples F4-F7, wherein the one or more unknown biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the secondset of biological state features, or one or more biological state features from the third set of biological state features.

[0340] F9. The method of example F6, wherein the one or more unknown biological state features comprise at least one biological state feature from the fourth set of biological state features.

[0341] F10. The method of any one of examples F1-F9, further comprising: generating a tokenized representation of the biological state, wherein the tokenized representation is input to the one or more machine learning models to predict the at least one of the one or more unknown biological state features.

[0342] Fl 1. The method of example F10, wherein the tokenized representation comprises the plurality of biological state features, including the one or more known biological state features and the one or more unknown biological state features, structured based on an expression value of each of the plurality of biological state features.

[0343] F12. The method of example Fl 1, wherein the plurality of biological state features are structured from low expression value to high expression value or from high expression value to low expression value.

[0344] F13. The method of any one of examples Fl -Fl 2, wherein each biological state feature of the plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype is expressed within a biological sample of a biological system; and at least one conditional attribute associated with the biological state feature.

[0345] F14. The method of example F13, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

[0346] F15. The method of any one of examples Fl -Fl 4, wherein the one or more machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

[0347] Fl 6. The method of any one of examples Fl -Fl 4, wherein the prediction of the at least one of the one or more unknown biological state features comprises at least one of: an intra-omic prediction, an inter-omic prediction, or an inter-phenotype prediction.

[0348] F17. The method of any one of examples F1-F16, wherein the biological state describes a disease state of a biological system, the biological system comprising a subject, and the one or more known biological state features are measured using a biological sample obtained from the subject.

[0349] Fl 8. A method for training a machine learning model to predict one or more unknown biological state features, the method comprising: obtaining a plurality of measured biological states each represented by a plurality of biological state features; and for each of the plurality of measured biological states: masking one or more biological state features from the plurality of biological state features; inputting the plurality of biological state features including the one or more masked biological state features to a machine learning model, the machine learning model outputting a predicted biological state, wherein the predicted biological state comprises a plurality of predicted biological state features; computing a loss based on the plurality of biological state features and the plurality of predicted biological state features; and adjusting one or more parameters of the machine learning model based on the computed loss until a threshold training condition is satisfied.

[0350] Fl 9. The method of example Fl 8, wherein masking the one or more masked biological state features comprises: obtaining a selection of a set of biological state features to mask, wherein the one or more masked biological state features of each measured biological state corresponds to the set of biological state features.

[0351] F20. The method of example F18 or F19, wherein inputting the plurality of biological state features including the one or more masked biological state features to the machine learning model comprises: for each of the plurality of measured biological states: inputting the plurality of biological state features including the one or more masked features into the encoder to obtain an embedding; and determining, using the decoder, the plurality of predicted biological state features.

[0352] F21. The method of any one of examples F18-F20, wherein the machine learning model comprises a first machine learning model, the one or more masked biological state features comprise one or more first masked biological state features, the predicted biological state comprises a first predicted biological state, the plurality of predicted biological state features comprises a first biological state features, the loss comprises a first loss, and the one or more parameters comprise one or more first parameters of the first machine learning model, the method further comprises: for each of the plurality of measured biological states: masking one or more second biological state features from the plurality of biological state features; inputting the plurality of biological state features including the one or more second masked biological state features to a second machine learning model to obtain a second predicted biological state, wherein the second predicted biological state comprises a second plurality of predicted biological state features; and computing a second loss based on the second plurality of biological state features and the second plurality of predicted biological state features; and adjusting one or more parameters of the second machine learning model based on the second computed loss until the threshold training condition is satisfied.

[0353] F22. The method of example F21, wherein an output of the first machine learning model comprises a first plurality of predicted biological states respectively corresponding to the plurality of measured biological states and an output of the second machine learning model comprises a second plurality of predicted biological states respectively corresponding to the plurality of measured biological states, the method further comprises: generating a first attention map using the first plurality of predicted biological states; determining a first relationship map representing first relationships between the plurality of biological state features using the first attention map; generating a second attention map using the second plurality of predicted biological states; and determining a second relationship map representing second relationships between the plurality of biological state features using the second attention map; andcombining the first relationship map and the second relationship map to generate a gene regulatory network.

[0354] F23. The method of any one of examples F18-F22, wherein the plurality of predicted biological state features comprise predictions for the one or more masked biological state features.

[0355] F24. The method of example F23, wherein the plurality of predicted biological state features further comprise predictions of one or more unmasked biological state features.

[0356] F25. The method of any one of examples F18-F24, wherein each of the plurality of measured biological states is associated with a different biological system of a plurality of biological systems.

[0357] F26. The method of example F25, wherein, for each of the plurality of measured biological states, each feature of the plurality of biological state features comprises a measured expression value indicating a level with which a biological marker associated with the feature is expressed within a biological sample of a corresponding biological system.

[0358] F27. The method of any one of examples F18-F26, further comprising: for each of the plurality of measured biological states: generating a tokenized representation of the measured biological state, wherein the tokenized representation is input to the machine learning model, wherein the machine learning model outputs the plurality of predicted biological state features.

[0359] F28. The method of example F27, wherein the tokenized representation comprises the plurality of biological state features structured based on an expression value of each of the plurality of biological state features.

[0360] F29. The method of example F28, wherein the plurality of biological state features are structured from low expression value to high expression value or from high expression value to low expression value.

[0361] F30. The method of any one of examples F18-F29, wherein each measured biological state of the plurality of measured biological states comprises a n-dimcnsional feature vector, wherein the plurality of biological state features are stored using the n- dimensional feature vector.

[0362] F31. The method of example F30, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least oneof: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

[0363] F32. The method of any one of examples F18-F31, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

[0364] F33. The method of example F32, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system described by the measured biological state; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system.

[0365] F34. The method of example F32 or F33, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

[0366] F35. The method of example F34, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

[0367] F36. The method of any one of examples F32-F35, wherein the one or more masked biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the second set of biological state features, or one or more biological state features from the third set of biological state features.

[0368] F37. The method of example F35, wherein the one or more masked biological state features comprise at least one biological state feature from the fourth set of biological state features.

[0369] F38. The method of any one of examples F18-F37, wherein each biological state feature of the plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype is expressed within a biological sample of a biological system described by the measured biological state; and at least one conditional attribute associated with the biological state feature.

[0370] F39. The method of example F38, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

[0371] F40. The method of any one of examples F18-F39, wherein the one or more machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

[0372] F41. The method of example F40, wherein the one or more machine learning models comprises an encoder and decoder, wherein inputting comprises: inputting the plurality of biological state features including the one or more masked biological state feature to the encoder, the encoder outputting an embedding representing the plurality of biological state features; and inputting the embedding to the decoder, the decoder outputting plurality of predicted biological state features.

[0373] F42. The method of any one of examples F18-F41, wherein the prediction of the plurality of predicted biological state features comprises at least one of: an intra-omic prediction, an inter-omic prediction, or an inter-phenotype prediction.

[0374] F43. The method of any one of examples F18-F42, wherein the plurality of known biological state features are measured using a biological sample, the biological sample being obtained from the biological system, and wherein the biological system comprise comprises a subject.

[0375] F44. A system, comprising: memory storing computer program instructions; andone or more processors that execute the computer program instructions to perform the method of any one of examples F1-F43.

[0376] F45. A non-transitory computer-readable medium storing computer program instructions that, when executed by one or more processors, effectuate operations comprising the method of any one of examples F1-F43.

Claims

CLAIMSWhat is claimed is:

1. A method for generating a prediction of a future biological state of a biological system, the method comprising: determining a first biological state and a second biological state, wherein the first biological state describes the biological system at a first time and the second biological state describes the biological system at a second time; generating, using one or more machine learning models, a first embedding representing the first biological state at the first time and a second embedding representing the second biological state at the second time; predicting, using the one or more machine learning models, a future embedding representing a future biological state of the biological system at a future time; and generating, using the one or more machine learning models, the prediction of the future biological state describing the biological system at the future time based on the future embedding.

2. The method of claim 1, further comprising: determining, using the one or more machine learning models, a prediction of a disease state of the biological system at the future time based on the prediction of the future biological state of the biological system.

3. The method of claim 2, further comprising: determining one or more therapeutic interventions for the biological system based on the prediction of the disease state of the biological system.

4. The method of any one of claims 1-3, wherein the one or more machine learning models comprise an encoder, a generator, or a decoder.

5. The method of any of claims 1-4, wherein the one or more machine learning models comprises an autoencoder.

6. The method of claim 5, wherein generating the first embedding and the second embedding comprises: inputting the first biological state and the second biological state into the encoder to generate the first embedding and the second embedding.

7. The method of claim 5 or claim 6, wherein predicting the future embedding comprises: inputting the first embedding and the second embedding into the generator to generate the future embedding.

8. The method of claim 7, wherein an indication of the future time is input to the generator with the first embedding and the second embedding to generate the future embedding.

9. The method of any one of claims 4-8, wherein generating the prediction of the future biological state comprises: inputting the future embedding into the decoder to generate the prediction of the future biological state of the biological system at the future time.

10. The method of any one of claims 1-9, wherein the first biological state, the second biological state, and the future biological state are each an n-dimcnsional feature vector.

11. The method of any one of claims 1-10, wherein the first biological state, the second biological state, and the future biological state are each represented using a plurality of biological state features.

12. The method of claim 11, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

13. The method of claim 11 or 12, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

14. The method of claim 13, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of the biological system; each biological state feature of the second set of biological state featurescomprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system.

15. The method of claim 14, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

16. The method of claim 15, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

17. The method of any one of claims 1-16, wherein the one or more machine learning models comprise one or more first machine learning models, the first biological state comprises a first plurality of biological state features and the second biological state comprises a second plurality of biological state features, wherein at least one of: one or more of the first plurality of biological state features or one or more of the second plurality of biological state features are predicted using one or more second machine learning models.

18. The method of claim 17, wherein the first plurality of biological state features comprises one or more first known biological state features and the second plurality of biological state features comprises one or more second known biological state features.

19. The method of any one of claims 1-18, further comprising: generating a first tokenized representation of the first biological state and a second tokenized representation of the second biological state, wherein the first tokenized representation and the second tokenized representation are input to the one or more machine learning models to generate the first embedding and the second embedding, respectively.

20. The method of claim 19, wherein the first tokenized representation comprises a first plurality of biological state features representing the first biological state and the second tokenized representation comprises a second plurality of biological state features represented second biological state, and wherein the first tokenized representation structures the firstplurality of biological state features in an order determine based on an expression value of each of the first plurality of biological state features, and wherein the second tokenized representation structures the second plurality of biological state features in an order determine based on an expression value of each of the second plurality of biological state features.

21. The method of claim 20, wherein the order of the first plurality of biological state features and the order of the second plurality of biological state features is from low expression value to high expression value or from high expression value to low expression value.

22. The method of claim 20 or 21, wherein each biological state feature of the first plurality of biological state features and each biological state feature of the second plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype is expressed within a biological sample of the biological system; and at least one conditional attribute associated with the biological state feature.

23. The method of claim 22, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

24. The method of any one of claims 1-23, wherein the one or more machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

25. The method of any one of claims 1-24, wherein the future embedding representing the future biological state of the biological system at the future time comprises a first future embedding representing a first future biological state of the biological system at a first future time, the method further comprises: predicting, using the one or more machine learning models, a second future embedding representing a second future biological state of the biological system at a second future time; and generating, using the one or more machine learning models, a second prediction ofthe second future biological state describing the biological system at the second future time based on the second future embedding.

26. The method of claim 25, further comprising: determining, using the one or more machine learning models, a first prediction of a first disease state of the biological system at the first future time based on the first prediction of the first future biological state of the biological system; and determining, using the one or more machine learning models, a second prediction of a second disease state of the biological system at the second future time based on the second prediction of the second future state of the biological system.

27. The method of claim 26, further comprising: determining one or more therapeutic interventions for the biological system based on at least one of the first prediction of the first disease state of the biological system or the second prediction of the second disease state of the biological system.

28. The method of any of claims 1-27, wherein the first embedding represents the first biological state at the first time and the second biological state at the second time.

29. The method of claim 28, wherein the second embedding is the first embedding.

30. The method of any of claims 1-29, wherein the first embedding or the second embedding is generated by a linear dimensionality reduction.

31. The method of claim 30, wherein the linear dimensionality reduction is principal component analysis (PCA).

32. The method of any of claims 1-31, wherein the first embedding represents a reduced dimensionality space for which the biological system changes over time.

33. The method of claim 32, wherein the reduced dimensionality space is based on gene expression data.

34. The method of any of claims 1-33, wherein the biological system changes over time, based, at least in part, on the one or more therapeutic interventions or a second one or more therapeutic interventions.

35. The method of any of claims 1-34, wherein the first embedding is used to predict a second future biological state of a second biological system.

36. The method of claim 35, wherein the second future biological state is a response to the one or more therapeutic interventions or to the second or more therapeutic interventions.

37. The method of any of claims 1-36, further comprising: training the one or more machine learning models using training data comprising: for each of a plurality of biological systems: a first biological state describing the biological system at a first time and a second biological state describing the biological system at a second time; a first embedding representing the first biological state and a second embedding representing the second biological state; one or more future embeddings representing one or more future biological states of the biological system at one or more future times; and the one or more future biological states of the biological system at the one or more future times.

38. The method of claim 37, wherein the first embedding represents the first biological state and the second biological state.

39. The method of claim 37 or 38, wherein the method comprises a first embedding and not a second embedding or one or more future embeddings.

40. The method of any of claims 37-39, wherein the first biological state and the second biological state are based on the gene expression data.

41. The method of any of claims 1-40, wherein the gene expression data comprises differentially expressed genes (DEGs).

42. The method of claim 41, wherein the DEGs are determined based on a non-linear combination of an adjusted p-value, a log2 fold change, and a single fitting parameter, k.

43. The method of claim 41 or 42, wherein the DEGs are determined according toDEG iff [Abs(log2fold)-1] * Max{[-log(pval) + log(0.05)J, 0} > kwherein, k is the single fitting parameter and log2fold is a base logarithm fold change in gene expression of a gene.

44. The method of any of claims 1-43, wherein the gene expression data comprises at least one temporally variable gene (TVG).

45. The method of claim 44, wherein the TVG is determined based, at least in part, on a TVG score.

46. The method of claim 45, wherein the TVG score is determined according toTVG Score = Sum_(i > j) Min (Max{[-log(pval(i,j)) + log(0.05)] , 0}, 20) wherein pval(i,j) is a generated p-value of a gene comparing day i versus day j.

47. The method of any of claims 44-46, wherein the TVG is determined, in accordance with a determination that the TVG score is greater than a predetermined TVG score threshold.

48. A method for generating a tokenized representation of a biological state for a machine learning model, the method comprising: obtaining a biological state describing a biological system, wherein the biological state comprises a plurality of biological state features each having an expression value; for each of a subset of the plurality of biological state features: computing a logarithmic expression value of the biological state feature based on the expression value; and discretizing the logarithmic expression value into a discretized expression value; and generating a tokenized representation of the biological state comprising the plurality of biological state features structured using an ordering function based on the discretized expression value of each of the subset of the plurality of biological state features.

49. The method of claim 48, wherein the biological state comprises a n-dimcnsional feature vector.

50. The method of claim 49, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

51. The method of any one of claims 48-50, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

52. The method of claim 51, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the biological sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the biological sample of the biological system.

53. The method of claim 51 or 52, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

54. The method of claim 53, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

55. The method of claim 54, wherein the plurality of biological state features comprises one or more unknown biological state features and one or more known biological state features.

56. The method of claim 55, wherein the one or more known biological state features comprise at least one of: one or more predicted biological state features or one or more measured biological state features.

57. The method of claim 55 or 56, wherein the one or more unknown biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the second set of biological state features, or one or more biological state features from the third set of biological state features.

58. The method of any one of claims 55-57, wherein the one or more unknown biological state features comprise at least one biological state feature from the fourth set of biological state features.

59. The method of any one of claims 48-58, wherein obtaining the biological state describing the biological system comprises: receiving expression data comprising a plurality of biological samples obtain for the biological system, wherein each of the plurality of biological samples is represented using a sample- specific biological state; and combining the sample- specific biological state of each of the plurality of samples to obtain a combined biological state, wherein the biological state comprising the subset of the plurality of biological state features is determined based on the combined biological state.

60. The method of claim 59, wherein the sample-specific biological state of each of the plurality of biological samples is represented using a plurality of sample- specific biological state features each having a sample- specific expression value, combining comprises: averaging the sample- specific expression value of each of the plurality of samplespecific biological state features to obtain an average expression value, wherein the expression value of each of the subset of the plurality of biological state features comprises the average expression value.

61. The method of claim 60, further comprising: for each of the plurality of sample- specific biological state features: determining whether the average expression value satisfies a threshold condition; andone of: removing the sample- specific biological state feature from the plurality of sample- specific biological state features based on the average expression value of the sample- specific biological state feature failing to satisfy the threshold condition; or keeping the sample- specific biological state feature included in the plurality of sample- specific biological state features based on the average expression value of the sample- specific biological state feature satisfying the threshold condition.

62. The method of claim 61, wherein the subset of the plurality of biological state features is formed based on whether a corresponding sample- specific biological state feature of the plurality of sample- specific biological state features was removed or kept.

63. The method of claim 61 or 62, wherein the average expression value satisfying the threshold condition comprises the average expression value being greater than or equal to a threshold expression value.

64. The method of any one of claims 61-63, wherein the average expression value failing to satisfy the threshold condition comprises the average expression value being less than a threshold expression value.

65. The method of claim 63 or 64, wherein the threshold expression value is an adjustable parameter of the machine learning model.

66. The method of claim 63 or 64, wherein the threshold expression value is about 1.0.

67. The method of any one of claims 48-66, wherein the biological state is derived from RNA sequencing data obtained from the biological system.

68. The method of any one of claims 48-67, wherein the plurality of biological state features comprises a set of biological state features each corresponding to a gene of a set of genes.

69. The method of claim 68, wherein each gene of the set of genes comprises an identifier for identifying the gene in one or more biological databases.

70. The method of claim 68 or 69, wherein the set of genes includes between about 10A3 and about 10A6 genes.

71. The method of any one of claims 48-70, wherein the plurality of biological state features comprise: 10A6 or more biological state features, 10A7 or more biological state features, 10A8 or more biological state features, or 10A9 or more biological state features.

72. The method of any one of claims 48-71, wherein the ordering function indicates an order for the plurality of biological state features included by the tokenized representation.

73. The method of claim 72, wherein the order is ascending or descending.

74. The method of any one of claims 48-73, wherein the biological state is derived from a biological sample of the biological system, wherein the biological sample comprises bulk RNA sequencing data or single cell RNA sequencing data.

75. A method for generating training data for training a machine learning model to predict a biological state of a biological system, the method comprising: retrieving a plurality of tokenized representations each describing a biological system of a plurality of biological systems, wherein a biological sample is obtained from each of the plurality of biological systems to determine the biological state of the biological system, and wherein each tokenized representation is generated using the method of any one of claims 48- 74; and generating training data using the plurality of tokenized representations.

76. A method for training a machine learning model to predict a biological state of a biological system, the method comprising: training a machine learning model using the generated training data of claim 75.

77. The method of claim 76, the machine learning model comprises at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

78. A method for predicting one or more unknown biological state features of a biological state from one or more known biological state features of the biological state, the method comprising: inputting a plurality of biological state features of the biological state into one or more machine learning models to generate an embedding representing the biological state, wherein the plurality of biological state features comprise the one or more known biological state features and the one or more unknown biological state features; andpredicting, using the one or more machine learning models, at least one of the one or more unknown biological state features based on the embedding.

79. The method of claim 78, wherein the biological state comprises a n-dimcnsional feature vector.

80. The method of claim 79, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

81. The method of any one of claims 78-80, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

82. The method of claim 81, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the biological sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the biological sample of the biological system.

83. The method of claim 81 or 82, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

84. The method of claim 83, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

85. The method of any one of claims 81-84, wherein the one or more unknown biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the second set of biological state features, or one or more biological state features from the third set of biological state features.

86. The method of claim 85, wherein the one or more unknown biological state features comprise at least one biological state feature from the fourth set of biological state features.

87. The method of any one of claims 78-86, further comprising: generating a tokenized representation of the biological state, wherein the tokenized representation is input to the one or more machine learning models to predict the at least one of the one or more unknown biological state features.

88. The method of claim 87, wherein the tokenized representation comprises the plurality of biological state features, including the one or more known biological state features and the one or more unknown biological state features, structured based on an expression value of each of the plurality of biological state features.

89. The method of claim 88, wherein the plurality of biological state features are structured from low expression value to high expression value or from high expression value to low expression value.

90. The method of any one of claims 78-89, wherein each biological state feature of the plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype is expressed within a biological sample of a biological system; and at least one conditional attribute associated with the biological state feature.

91. The method of claim 90, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

92. The method of any one of claims 78-91, wherein the one or more machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

93. The method of any one of claims 78-92, wherein the prediction of the at least one of the one or more unknown biological state features comprises at least one of: an intra-omic prediction, an inter-omic prediction, or an inter-phenotype prediction.

94. The method of any one of claims 78-93, wherein the biological state describes a disease state of a biological system, the biological system comprising a subject, and the one or more known biological state features are measured using a biological sample obtained from the subject.

95. A method for training a machine learning model to predict one or more unknown biological state features, the method comprising: obtaining a plurality of measured biological states each represented by a plurality of biological state features; and for each of the plurality of measured biological states: masking one or more biological state features from the plurality of biological state features; inputting the plurality of biological state features including the one or more masked biological state features to a machine learning model, the machine learning model outputting a predicted biological state, wherein the predicted biological state comprises a plurality of predicted biological state features; computing a loss based on the plurality of biological state features and the plurality of predicted biological state features; and adjusting one or more parameters of the machine learning model based on the computed loss until a threshold training condition is satisfied.

96. The method of claim 95, wherein masking the one or more masked biological state features comprises: obtaining a selection of a set of biological state features to mask, wherein the one or more masked biological state features of each measured biological state corresponds to the set of biological state features.

97. The method of claim 95 or 96, wherein inputting the plurality of biological state features including the one or more masked biological state features to the machine learning model comprises: for each of the plurality of measured biological states: inputting the plurality of biological state features including the one or more masked features into the encoder to obtain an embedding; and determining, using the decoder, the plurality of predicted biological state features.

98. The method of any one of claims 95-97, wherein the machine learning model comprises a first machine learning model, the one or more masked biological state features comprise one or more first masked biological state features, the predicted biological state comprises a first predicted biological state, the plurality of predicted biological state features comprises a first biological state features, the loss comprises a first loss, and the one or more parameters comprise one or more first parameters of the first machine learning model, the method further comprises: for each of the plurality of measured biological states: masking one or more second biological state features from the plurality of biological state features; inputting the plurality of biological state features including the one or more second masked biological state features to a second machine learning model to obtain a second predicted biological state, wherein the second predicted biological state comprises a second plurality of predicted biological state features; and computing a second loss based on the second plurality of biological state features and the second plurality of predicted biological state features; and adjusting one or more parameters of the second machine learning model based on the second computed loss until the threshold training condition is satisfied.

99. The method of claim 98, wherein an output of the first machine learning model comprises a first plurality of predicted biological states respectively corresponding to the plurality of measured biological states and an output of the second machine learning model comprises a second plurality of predicted biological states respectively corresponding to the plurality of measured biological states, the method further comprises: generating a first attention map using the first plurality of predicted biological states;determining a first relationship map representing first relationships between the plurality of biological state features using the first attention map; generating a second attention map using the second plurality of predicted biological states; and determining a second relationship map representing second relationships between the plurality of biological state features using the second attention map; and combining the first relationship map and the second relationship map to generate a gene regulatory network.

100. The method of any one of claims 95-99, wherein the plurality of predicted biological state features comprise predictions for the one or more masked biological state features.

101. The method of claim 100, wherein the plurality of predicted biological state features further comprise predictions of one or more unmasked biological state features.

102. The method of any one of claims 95-101, wherein each of the plurality of measured biological states is associated with a different biological system of a plurality of biological systems.

103. The method of claim 102, wherein, for each of the plurality of measured biological states, each feature of the plurality of biological state features comprises a measured expression value indicating a level with which a biological marker associated with the feature is expressed within a biological sample of a corresponding biological system.

104. The method of any one of claims 95-103, further comprising: for each of the plurality of measured biological states: generating a tokenized representation of the measured biological state, wherein the tokenized representation is input to the machine learning model, wherein the machine learning model outputs the plurality of predicted biological state features.

105. The method of claim 104, wherein the tokenized representation comprises the plurality of biological state features structured based on an expression value of each of the plurality of biological state features.

106. The method of claim 105, wherein the plurality of biological state features are structured from low expression value to high expression value or from high expression value to low expression value.

107. The method of any one of claims 95-106, wherein each measured biological state of the plurality of measured biological states comprises a n-dimcnsional feature vector, wherein the plurality of biological state features are stored using the n-di mcnsional feature vector.

108. The method of claim 107, wherein the plurality of biological state features comprise biological markers, wherein the biological markers comprise at least one of: genomic data, proteomic data, transcriptomic data, physiological data, or phenotypical data.

109. The method of any one of claims 95-108, wherein the plurality of biological state features comprise at least one of: a first set of biological state features corresponding to RNA expression data, a second set of biological state features corresponding to protein expression data, or a third set of biological state features corresponding to methylation markers.

110. The method of claim 109, wherein: each biological state feature of the first set of biological state features comprises: a label indicating a corresponding gene from a set of genes represented by the biological state feature and an expression value indicating a level with which that gene is expressed within a biological sample of a biological system described by the measured biological state; each biological state feature of the second set of biological state features comprises: a label indicating a corresponding protein from a set of proteins represented by the biological state feature and an expression value indicating a level with which that protein is expressed within the sample of the biological system; and each biological state feature of the third set of biological state features comprises: a label indicating a corresponding methylation marker of a set of methylation markers represented by the biological state feature and an expression value indicating a level with which that methylation marker is expressed within the sample of the biological system.

111. The method of claim 109 or 110, wherein the plurality of biological state features further comprises a fourth set of biological state features corresponding to phenotype data.

112. The method of claim 111, wherein each biological state feature of the fourth set of biological state features comprises: a label indicating a corresponding phenotype of a set of phenotypes represented by the biological state feature.

113. The method of any one of claims 109-112, wherein the one or more masked biological state features comprise at least one of: one or more biological state features from the first set of biological state features, one or more biological state features from the second set of biological state features, or one or more biological state features from the third set of biological state features.

114. The method of claim 113, wherein the one or more masked biological state features comprise at least one biological state feature from the fourth set of biological state features.

115. The method of any one of claims 95-114, wherein each biological state feature of the plurality of biological state features comprises: a label indicating that the biological state feature corresponds to a gene, a protein, a methylation marker, or a phenotype; an expression value associated with the biological state feature, the expression value indicating a level with which that gene, protein, methylation marker, or phenotype is expressed within a biological sample of a biological system described by the measured biological state; and at least one conditional attribute associated with the biological state feature.

116. The method of claim 115, wherein the at least one conditional attribute comprises at least one of: epigenetic markers or environmental variables.

117. The method of any one of claims 85-116, wherein the one or more machine learning models comprise at least one of: a neural network, a long-short term memory (LSTM) model, or a transformer model.

118. The method of claim 117, wherein the one or more machine learning models comprises an encoder and decoder, wherein inputting comprises: inputting the plurality of biological state features including the one or more masked biological state feature to the encoder, the encoder outputting an embedding representing the plurality of biological state features; andinputting the embedding to the decoder, the decoder outputting plurality of predicted biological state features.

119. The method of any one of claims 95-118, wherein the prediction of the plurality of predicted biological state features comprises at least one of: an intra-omic prediction, an inter-omic prediction, or an inter-phenotype prediction.

120. The method of any one of claims 95-119, wherein the plurality of known biological state features are measured using a biological sample, the biological sample being obtained from the biological system, and wherein the biological system comprise comprises a subject.

121. A system, comprising: memory storing computer program instructions; and one or more processors that execute the computer program instructions to perform the method of any one of claims 1-120.

122. A non-transitory computer-readable medium storing computer program instructions that, when executed by one or more processors, effectuate operations comprising the method of any one of claims 1-120.

Images