System for predicting genetic diseases and conditions using neural networks

Abstract

Disclosed is an apparatus, method and computer program product for training and utilizing neural networks.An exemplary method includes obtaining sequence variant sample data and using sequence variant sample data to train neural networks.An exemplary method further includes modifying the value of one or more loci for one or more samples in sequence variant sample data to create modified sequence variant sample data, and using the modified sequence variant sample data as input during training of neural network, so that neural network is trained to predict values ​​from unaltered sequence variant sample data and one or more reference genome maps.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 308,091, filed February 9, 2022, and U.S. Provisional Patent Application No. 63 / 324,536, filed March 28, 2022, both of which are incorporated by reference in their entireties herein.

[0002] Technical Field The present invention relates generally to a deep learning based system that receives genomic or multi-omic and clinical data inputs and classifies or predicts disease, and more specifically to a system that processes genomic or multi-omic sequences for long regions from one or more chromosomes. [Background technology]

[0003] background The size of the human genome obtained and the various experimental data obtained that are mapped to positions in the genome are more than 3 billion bases long. Even if longer k-mers (i.e., multiple consecutive loci or substrings) are represented as one token, for a fairly large subset of these loci in a deep learning model, the full attention matrix for all these tokens becomes completely impractical due to its size. For example, due to its complexity, this may require tetrabytes or petabytes of storage to calculate. Attempts to reduce this complexity in the prior art include clustering or focusing the attention mechanism on only a subset of the input, reducing the region of interest, and reducing the input sequence to only a small sequence of the locus of interest.

[0004] As a further example, the whole genome sequences of human cancer and normal tissues as input to a neural network, when represented in a densely linked (e.g., stacked) one-hot encoding format, require a matrix with approximately 3 billion columns and 10 or 20 rows. This assumes two or four groups of columns that specify the four nucleotides (A, C, G, T), as well as placeholders for both cancer and normal tissues at all positions and possibly both chromosome pairs for each tissue type that are unknown. In addition to this, multiple additional dimensions (rows) can track a wide variety of other genomic features, including but not limited to methylation sites, microsatellite variability, and / or insertions and deletions. As networks trained using stochastic gradient descent need to assemble multiple such configurations, memory and computational power have had to be multiplied during training. As a result, solutions found in the prior art are suboptimal as they focus on input with relatively small networks and may require further filtering of input loci that rely on knowledge of the underlying biology that is only partially understood. Summary of the Invention [Means for solving the problem]

[0005] overview The demonstrated embodiments show how to input a given set of reference genome feature mappings (such as multi-omics data and nucleotide bases) and different loci, and teach the network to integrate the reference genome mappings with the received data in an indirect, possibly selective, measurable manner during training. In other words, in one or more embodiments, the deep learning system is taught to internally store the reference data and mappings. Also, one or more of the described embodiments teach how to increase the receptive field of the neural network using a stacked / graph neural network structure so that every part of the input and integrated reference data can attend to every other part of the data during the self-supervised pre-learning and supervised or semi-supervised fine-tuning during the training process. This allows the network in some embodiments to evaluate the impact of genome-wide genomic changes and multi-omics in addition to the clinical data included in the embodiments. [Brief description of the drawings]

[0006] Having generally described certain exemplary embodiments above, reference is now made to the accompanying drawings, which are not necessarily drawn to scale. Some embodiments may include fewer or more components than shown in the drawings.

[0007] [Figure 1] FIG. 1 shows a high-level simplified schematic setup for a training, testing, and inference pipeline that may be used in accordance with some embodiments described herein.

[0008] [Diagram 2] FIG. 2 shows an illustrative diagram of a neural network graph composed of layers, nodes, functions, and transformations between graph levels that may be used in accordance with some embodiments described herein.

[0009] [Diagram 3]FIG. 3 illustrates a transformation from one level to another in a neural network graph structure that may be used in accordance with some embodiments described herein.

[0010] [Figure 4] FIG. 4 illustrates an example embodiment of injecting global metadata (e.g., cell type, gender, cohort cluster, etc.) at the lowest level in the graph to propagate root-level information to lower graph layers that may be used in accordance with some embodiments described herein.

[0011] [Diagram 5] FIG. 5 shows a flowchart of pre-learning logic that may be used to recreate or store integrated reference data and input data in a neural network that may be used in accordance with some embodiments described herein.

[0012] [Figure 6] FIG. 6 illustrates the use of a root node(s) for classification or regression analysis for example diseases and conditions that may be used in accordance with some embodiments described herein.

[0013] [Figure 7] FIG. 7 illustrates an example flowchart for training a neural network to generate one or more predictions according to some embodiments described herein.

[0014] [Figure 8] FIG. 8 shows an exemplary flowchart for utilizing a neural network to generate one or more predictions for a subject according to some embodiments described herein.

[0015] [Figure 9]FIG. 9 illustrates a schematic block diagram of an example device that may perform various operations in accordance with certain example embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Detailed Description Some exemplary embodiments will now be described more fully below with reference to the accompanying drawings. The embodiments are some, but not necessarily all, shown. Because the invention described herein may be embodied in many different forms, the invention is not intended to be limited to only the described embodiments; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements.

[0017] The term "computing device" refers to any one or all of programmable logic controllers (PLCs), programmable automation controllers (PACs), industrial computers, desktop computers, personal digital assistants (PDAs), laptop computers, tablet computers, smartbooks, palmtop computers, personal computers, smartphones, wearable devices (such as headsets or smartwatches), and similar electronic devices that include at least one processor and any other physical components necessary to perform the wide variety of operations described herein. Devices such as smartphones, laptop computers, tablet computers, and wearable devices are generally collectively referred to as mobile devices.

[0018] Considering now FIG. 1, an embodiment of a pipeline required to train a neural network on genome sequencing data and data mapping to genome coordinates (e.g., loci) is shown. The elements shown are a tangible computer-readable medium 10, such as a database or flat file system, that contains the results of sequencing a given tissue, optionally metadata about the sequencing and each example input, as well as additional multi-omics features of each or some of the resulting loci. The sequencing data may need to be processed by a first parser algorithm 11 and transferred to a second tangible computer-readable medium or computer memory 13 for efficient processing by the training pipeline shown. The first parser 11 can be instructed to format the data and store it as formatted data, such as TensorFlow records, or in another efficient format specialized for training the neural network and moving the data through the network. A second parser or conversion algorithm or both 12 may be used at training time to further process the data, and the parser may access one or more reference genomes and other reference data stored on medium 14. In some embodiments, for example, GRCh38.p13, T2T, or other reference genomes may be used. These reference genomes may be capable of reducing the size of genomes or features retrieved from data storage 10 or storage medium 13 by selectively erasing data already described by the reference data stored in storage medium 14. For example, the reference data may be combined into one or more sets by combining the reference genome with one or more epigenetic maps (such as deoxyribonucleic acid (DNA) methylation maps) or other suitable epigenetic maps to match the input feature requirements. Each set may be used to reduce the size of all or a subset of the input cases as determined by other metadata attributes (such as sex and cell type).In some embodiments, this step is not necessary as the genomes and data in data storage 10 already contain only variants and changes from the reference data.

[0019] As part of the processing performed by the second parser 12, one or more genomes or data aligned to said genomes, such as from different samples obtained from tissues or genomic material from the same individual, may be aligned as tensors for input to the network, as an example. In some embodiments, sparse tensors (e.g., tables with tensor matrices and tensor values) or other data types are used and arranged in a dictionary representing an example of training. The dictionary may also include multi-omics data from the data storage 10, which may also be read and processed by the first parser 11. The multi-omics data may include data such as methylation sites found in the respective tissues or genetic material, and / or chromatin accessibility information associated with or experimentally obtained for the tissue. If the model is to be fine-tuned or otherwise trained in a supervised manner, labels may be further extracted from the data storage (e.g., data storage 10 and / or storage medium 13). For pre-learning, some embodiments use a masked training schema, as this is becoming common in the field of training large-scale foundational models (such as the Bidirectional Encoder Representation (BERT) model). In some embodiments, masking is performed by selecting limited-size random slices from the genome and optimizing a model (e.g., using associated weights) to predict the integrated reference sequence and input sequence within a particular layer of the masked sequence's location (e.g., locus). Self-supervised pre-learning such as masking can also be assigned to other optimization tasks in some embodiments, including predicting genomic modifications, chromatin accessibility sites, epigenomic features, and / or somatic or germline variant locations. To achieve this, a set of labels and masks is collected as part of the processing performed by the second parser 12, and these labels and masks are added to a dictionary or sparse tensor before batching multiple examples together for the training process.

[0020] During training, an iterator algorithm 15 may be used to loop through multiple epochs of training data and split the data into training, validation, and test data sets. In some embodiments, the training process 16 optimizes a self-supervised pre-learning task. In some embodiments, the training process 16 optimizes the neural network using supervised learning based on labels such as disease types, phenotypes, or epigenomic features. Some embodiments optimize both together. Some embodiments save the trained weights for one or more times in a storage medium 17 both after training and during training based on set conditions such as early stopping. Before training begins, some embodiments may load existing weights from a previous training session as initial weights instead of randomizing the initial values ​​of the weights according to standard deep learning approaches.

[0021] FIG. 2 shows a schematic diagram of an example of a neural network graph composed of layers that may be used in some embodiments for analysis of large data sets such as multi-omics data mapped to genomic coordinates / locuses. In particular, FIG. 2 shows a stacked transformer graph structure, similar to that used in the BERT model and / or other deep learning kernels. In one example, the input data to the network for one input case or input example is shown as a matrix at the leaf level 21, which may include one or more stacked one-hot encoding matrices aligned to human genome coordinates, the one-hot encoding matrices indicating deviations from the reference data in the storage medium 14 for each of the stacked matrices. Since the matrix may contain mostly zeros, the matrix may be efficiently stored as a sparse matrix representation with most or all zeros removed. For example, the matrix may be stored as a list of pairs of coordinates of the matrix and the non-zero or mostly non-zero values ​​of the matrix, as well as other metadata (such as the dense tensor or shape of the matrix). This can be done because the nature of one-hot encoding data and the sparsity of differences from the reference genome in common humans and individuals of other species are well understood.

[0022] In some embodiments, most or all of the genomic data that aligns with the reference values ​​in the storage medium 14 is simply represented by columns of zero values ​​and is therefore ignored by the sparse matrix representation of the matrix. In other embodiments, a non-zero encoding is used to represent the reference data. In some embodiments, the sparsity is such that the input matrix is ​​reduced based on the variants found in each individual case itself, and in some other embodiments, a set of standard variant positions can be used. Thus, even in examples where the size of the matrix is ​​n rows and m columns (m is greater than 3 billion), in some embodiments, the sparse matrix representation is efficiently stored as a list (e.g., a sparse matrix). This is also true in other embodiments for smaller matrices, for example when restricted to only exome sequences or a subset of genes or a combination of sequences from exon and intron regions and sequences from non-coding regions, in which case the matrix structure is equally efficiently stored as a sparse matrix of variant data only.

[0023] The data in the matrix in leaf level 21 is mapped to individual tokens in level 3 of group 23.

number

[0024] For illustrative purposes, m in level 3 of group 21 can be set to 3 billion, n can be set to 10, k (e.g., number of columns for sparse submatrix) can be set to 1000, d in level 3 of group 23 can be set to 1000, and the stride length can be set to 1000, so as not to overlap. In the above example, the number of tokens created in level 3 of group 23 is 3 million, and each token has a length (dimensionality) of 1000. In this exemplary setup, each token / embedding dimension 1000 represents 1000 consecutive bases in the genome, even though only variations from the reference data in storage medium 14 are included in this first stage. The tokens are shown as nodes in the graph at the third level of group 23. During processing by the training algorithm, the matrices have a wide variety of shapes determined for computation using well-known tensor reshaping methods available in neural network training frameworks such as TensorFlow. Additional batch dimensions (not shown, but contemplated) may also be included during training, allowing multiple examples forming a mini-batch of cases to be included in one optimization step. In some embodiments, a non-linear mapping may be used in transformation function 22 instead of the trainable linear mapping, including a transformer-based mapping (such as that shown in FIG. 3).

[0025] In some embodiments, the tokens in level 3 are assigned to a trainable or fixed positional encoding (e.g., see positional encoding operation 43 in FIG. 3 for further details) so that each token can represent a genomic interval in the previous level (e.g., leaf level 21) from before. The tokens in level 3 are subdivided into successive groups of tokens, and each group g of tokens is passed through a trainable multi-head attention mechanism (e.g., a transformer encoder) further illustrated in FIG. 3 shared at that level in some embodiments. Further tokens are created through the attention mechanism to represent a sequence of one or more output or parent node tokens at the next level and assigned to a positional encoding included in the multi-head attention mechanism, as further described in FIG. 3.

[0026] In some embodiments, one or more additional input type tokens are added and shared per token group as trainable or fixed tokens for distinguishing different input cases, such as tokens representing input cells, tissue types, sex, ancestry, or other covariates or cohort groups, as further explained by FIG. 4. If more than one parent token node is indicated, some embodiments concatenate (e.g., by a concat function) the resulting parent node tokens before placing them at the next level in the graph (toward the root node). Furthermore, the embedding dimension d can be increased at the next level accordingly to accommodate the embedding with increased length. Note that for simplicity, FIG. 2 and FIG. 3 show only one additional token. However, those skilled in the art will recognize that one or more additional tokens may be added to represent the parent token node. The one or more additional tokens can then be processed by a transformer encoder at the corresponding level and then returned as a representative of the group as one or more parent token nodes in the graph at the coarser next level (e.g., toward the root node on FIG. 2 and according to the operations described in FIG. 3).

[0027] Focusing on the first group in level 3 of group 23, one or more further tokens initialized with positional encoding are added to the first group, and then all tokens from this first group and further tokens (e.g. both further output (parent node) tokens and input type tokens) are mapped by the transformer encoder 24 (e.g. T2). The mappings resulting from the further output (e.g. parent node) tokens are forwarded to the next level constructed (e.g. level 2 of group 25 as parent nodes in the case of a child node in the first group in level 3 of group 23 in the graph) as they represent a complete group.

[0028] In some embodiments, the results of the application of the multi-head attention mechanism to the swarm (e.g., output 54 in FIG. 3) are further processed and used to create a loss function 31 for the layer, which is then used with other loss contributions to optimize the layer and neural network (weights) for prediction using the masked training schema, reference data, labels, or genome (e.g., reference data stored by storage medium 14 merged with input data from the matrix). In some embodiments, the loss function 31 is used to reshape the merged inherited genotype after merging the reference genome with the variants. In other embodiments, the loss function is also used to predict the multi-omics data after the merge of the reference data. In some embodiments, the self-supervised masked training implemented by the loss function 31 is implemented according to the operations described in FIG. 5. This training of the loss function teaches the network to selectively represent data excluded from the input as reference data through the multi-head self-attention mechanism to the next level, and gives the neural network the ability to work with the merged input data and reference data moving forward in the direction of the root node of the group 29. With this and other pre-training, in some embodiments, the large-scale network thereby learns to memorize a unified representation of the sparse input data and the reference data.

[0029] To illustrate the transformer encoder 24 shown in FIG. 2, let us again assume for the sake of illustration that the first group initially contains only 1000 tokens, then a further token initialized only with positional encoding is created, for a total of 1001 tokens passed through the shared transformer encoder 24 used for transformation. The 1001 vectors resulting from this process are then split into two parts: (i) the results of the original 1000 tokens after processing by the transformer used to establish a loss function that can internally predict the combined incoming data and reference data (as opposed to the input of this data) in the network, and (ii) the final 1001st token that goes forward as an embedding for the group that represents the full extent of the genome that the group covers. At each iteration, only a portion of the 1000 tokens, randomly and iteratively selected, is used to determine the loss value of the loss function. In some embodiments, this portion can be in the range of 15% to 20% of the tokens.

[0030] The above process of grouping tokens and passing the grouped tokens through a transformer kernel (e.g., transformer encoder 24 and transformer encoder 26 (e.g., T1)) is repeated as many times as necessary until the number of remaining tokens or the height of the graph (e.g., number of levels) reaches a desired limit. In some embodiments, as shown in FIG. 2, the process is repeated one or two more times using independent multi-headed self-attention, such as with transformer encoder 26 and transformer encoder 28 (e.g., T0). This results in multiple tokens at level 1 of group 27 and a final single token at the root node of group 29.

number

[0031] Figure 3 is referred to several times in the above description and described in more detail herein. Figure 3 shows an example of an encoder derived from a transform network that may be used in some embodiments. The encoder described in Figure 3 may be similar to a transform encoder that is commonly used and available in the BERT model. Although Figure 3 shows one particular implementation of a transform encoder, it is recognized that these transform encoders may include variations such as additional layer normalization, dropout, activation, regularizer, and other additional methods and network layers.

[0032] As shown in Fig. 3, a given group g 41 of tokens is taken as input, containing one or more further tokens 42 in addition to the tokens in the group. These tokens are assigned to a trainable or fixed positional encoding using an encoding operation 43. Fig. 3 then shows elements from a standard transform encoder design, such as a key K matrix, a query Q matrix, and a value V matrix 44, each formed to repeat and divided into h heads. The K, Q, V matrices 44 are then used by a multi-head self-attention mechanism 45. A first residual connection 46 and layer normalization 47 are provided to a feedforward network 48. Residual connections 49 are also shown along with further layer normalization 50. The process is repeated N times as shown by operation 51, and the final output 52 is the p variables 54 used in the prediction and the parent tokens.

number

[0033] FIG. 4 illustrates the use of learned input type tokens to convey input types to the lowest grouping level in a neural network graph structure. The top matrix shows an input-type A matrix 60, listed as input type "A". In some embodiments, this input type corresponds to a specific cell type, tissue type, exome sequence, whole genome sequence, or identifies species "A" or any other metadata associated with the example input. AA token denoted as “I” is added to each of the groups 62 at the lowest group level 61 (e.g., level 3 group). Assuming that different input type matrices 63 belong to different input types (e.g., different cell types, tissue types, exome sequences, whole genome sequences, or species), a token denoted as “I” is added to each of the groups 62 at the lowest group level 64 (e.g., level 3 group). B This results in different input type tokens, shown as "."

[0034] In some embodiments, the input type token "I A ","I B" etc. are d-dimensional trainable vectors. In some embodiments, these input type tokens are constant vectors. In some embodiments, multiple input type tokens per input are used to represent multiple input attributes. For example, one input type token may be used to represent cell type while another is used to represent gender. Encoding the input type representation in this manner conveys root-level (e.g., example-based) information to the lowest group level, allowing the network to perform input type-specific calculations at each grouping level if such information is hypothesized or shown to improve the network's performance and is available at inference time. In some embodiments, the input type tokens represent groups of input data based on clustering (such as clustering representing different ancestral cohorts). This may be the case if the input data can be meaningfully reduced by having different reference genomes or reference data for each cluster. In some embodiments, the input embeddings represent continuous variables (such as scores from a wide variety of other databases and algorithms). For example, input embeddings may include collective REVEL scores for specific regions, EVE scores for genes and other variables used to measure pathogenicity or loss of function of protein and DNA sequences, or available polygenic risk scores (PRS) for individuals analyzed or screened for a given disease. In some embodiments, continuous variables are mapped by binning ranges (e.g., from zero to one), so that different bins are represented by different embeddings that are learned or fixed. In some embodiments, continuous variables are embedded directly using a linear transformation starting from a set of one or more such values.

[0035] FIG. 5 shows a high-level flowchart of a part of the training process used by some embodiments to learn the integrated genomic or multi-omics data for individual training cases during pre-learning or fine-tuning. The training described in FIG. 5 can make the neural network learn the integrated genomic and / or multi-omics data even if, for example, the only data that differs from the reference data is the input to the neural network. At the same time, the masking optimization described requires the network to learn the frequency and likelihood of the functional genomics represented to the network as part of the input data by masking and modifying the input. In the present context, a part of the input is masked from both the variant data (e.g., the variant for a particular stack from the matrix and the reference data from the storage medium 14). In particular, operation 71 collects data during training to add loss contributions to the neural network training, and forces the network to internally reshape the integrated genomic or multi-omics input data after the data is integrated with the reference data from the storage medium 14. This is done by masking a subset of the inputs, varying some input or reference values, and having the network predict either the masked values ​​or the exact values ​​using a corresponding loss function (such as loss function 31).

[0036] In some embodiments, this is done by a neural network training iterator that invokes sub-process operations 72, 73, 74, and 75. In sub-process operation 72, the neural network training iterator requests an operation to mask one or more leaf nodes represented by a given output token, input token, and training example. Some embodiments generate or are fed one or more random token graph positions (e.g., from level 3 of group 23 in the case of loss function 31) that identify a subset of leaf nodes from the matrix.

[0037] In sub-process operation 73, the neural network training iterator determines which leaf nodes to mask and / or modify. In some embodiments, the neural network training iterator randomly selects leaf nodes to mask or modify (e.g., a subset of 15% of the leaf nodes). In some embodiments, the input leaf nodes are one or more stacked one-hot encoding matrices, and the masking goes to one or more of these stacked matrices.

[0038] In sub-process operation 74, the neural network training iterator reshapes the integrated inputs for the leaf nodes to mask or modify, and saves them as labels to mask, to create a set of corresponding weights. These training labels are generated because label training is required for mask loss. In some embodiments, these training labels are generated by procedurally reshaping the integrated input data and reference data for the selected stacked matrix. The labels can then be stored in memory or otherwise transiently removed from the input matrix for one round of training (e.g., epoch) in advance for the corresponding input data for a given training case.

[0039] In sub-process operation 75, the neural network training iterator may replace the masked or modified input nodes with a reference data encoding (eg, zeros) or a modified encoding.

[0040] In some embodiments, in addition to masking a portion (e.g., 5%) of the leaf nodes from the leaf node subset, a portion of the leaf nodes are alternatively modified by replacing their one-hot encoding with a different modified one-hot encoding, or left unchanged and still have the loss function (e.g., loss function 31) predict the correct class encoding during classification. Masking may require one or more weight matrices to be created and stored for use as further inputs to the loss function to instruct the loss function to only consider the predictions of the randomly selected leaf nodes.

[0041] In operation 76, once sufficient data is available, the training iterator may map selected tokens for the one-hot encoded input matrix to the input structure logits, compute a masked loss function, and add to determine the total loss. In some embodiments, mini-batches of masked examples are repeatedly created, and once the batches are ready for processing, the loss function is applied using the logits directly, or a softmax function is applied to normalize the encoding predictions. In some embodiments, predictions (logits) are generated by a trainable, shared linear map that maps the last forward pass tokens encoded from their positions in the token graph (e.g., level 3 of group 23 in the case of loss function 31 by each transformer encoder 24) to a matrix structure that fits the selected leaf node from the stacked matrix selected as part of the input. A loss function (e.g., loss function 31) then compares the network's predictions using the linear map to each stored label previously generated using the stored loss weights (e.g., in operation 74). In some embodiments, a classification cross-entropy loss function is used for this purpose, adding appropriate weights to the total loss of the network. In some embodiments, masked or modified leaf nodes are internally reshaped using one or more of the loss functions at a higher level (e.g., loss function 32 and / or loss function 33) similarly using the higher level tokens and encoders, respectively.

[0042] 6 illustrates the use of a root node for classification and / or regression analysis for exemplary diseases and conditions. In particular, FIG. 6 illustrates an operation 81 that maps one or more root tokens from the root node of a group 29 to a structure that fits a desired classification or regression template for training and inference. In some embodiments, the template 82 is an array of values ​​86. In some embodiments, the template 82 includes a single value that represents a logit or normalized likelihood (e.g., a softmax value) or a binary value 83. In some embodiments, the template 82 includes one or more other values ​​that result from a forward pass of input data through the neural network illustrated in FIG. 2.

[0043] The classification or regression templates are used to train the neural network (e.g., using loss function 34) during fine-tuning of the network or during both training phases (fine-tuning and pre-training). The classification or regression labels 88 used during training are available during training and may be extracted from data storage 10 and / or data storage 13 for each or some training examples. In some embodiments, the labels 88 are processed during training in operations 89 to fit the templates and / or to inject noise into the training.

[0044] For clarity, an example of such processing is the mapping of a label to a precise age bucket representing the onset or duration of a medical condition. In some embodiments, the classification template is composed of one or more age buckets 84, each undergoing a likelihood prediction representative of the likelihood of a disease or medical condition for the patient undergoing analysis or screening. In some embodiments, when predicting the (polygenic) likelihood of breast cancer from genetic and functional data obtained by sequencing and other methods, the age buckets 84 are used as a template to score the likelihood of the patient's condition in each age range defined by the age buckets. Similarly, in some embodiments, including when determining a cardiovascular disease risk score (risk of developing schizophrenia or type 1 or type 2 diabetes), the likelihood variables of the age buckets 86 or lifetime risk 85 are used to predict the risk of an individual.

[0045] In some embodiments, inputs to the neural network include variants from whole exome or whole genome sequencing, and inputs may be restricted to genomic regions that contain genes and other sequences hypothesized to be involved in disease development. Other phenotypes may be predicted using suitable categorical classes. In some embodiments, when predicting categorical values, a cross-entropy loss function is used as the loss criterion by loss function 34, and when predicting outputs in multiple buckets (e.g., age buckets), the template generates multiple binary scores per training example. In some embodiments, processing during operation 89 generates a loss weight for each bucket. The loss weights are used in some embodiments to eliminate buckets when information about the disease state of individuals in a given age range is not available. Additionally, the loss weights may be used to address bias and balance datasets to represent specific techniques related to oversampling or undersampling. In some embodiments, measurements of cellular expression (e.g., mRNA), and other quantitative traits may be predicted using the array of values ​​86 template while matching available labels. In some embodiments, when predicting quantitative traits, a Poisson or least squares loss function is used as the loss function criterion for loss function 34.

[0046] Some embodiments use human exome sequencing data as input. These embodiments may map a small number of genomic positions (e.g., 1-128) from the matrix to each individual token at the third level group 23 described with respect to FIG. 2. Some embodiments one-hot encode into constructed genotypes (phased or unphased) and treat homozygous genotypes that match the reference genome as reference genotypes, in which case only variants from the reference need then be included in the input sparse matrix representation of the matrix. Some embodiments include only single nucleotide polymorphisms, while others also one-hot encode insertions and deletions as variants. Some embodiments include larger regions that include exome data as input, for example to capture splicing variants, while some embodiments include large slivers of non-coding regions that may include regulatory regions. Labels used by some embodiments include the onset of diseases (such as breast cancer, schizophrenia, cardiovascular disease, type 1 and type 2 diabetes, etc.) and other diseases with genetic components from large cohorts of participants in genetic studies. Additionally, some embodiments construct a label using a combination of the disease phenotype and other attributes (such as age at onset or not affected by the disease or condition) (e.g., negative or positive samples).

[0047] Some embodiments use several additional techniques well known in the art, such as additional linear layers, regularization, convolution, dropout, tokenizers, specific input layers, activation functions, weight initialization, weight regularizers, momentum, different types of optimizers, variations of stochastic gradient descent, and different types of one-hot encoding schemes. Other embodiments of the invention, such as training on significantly larger portions of genomes or exomes instead of complete genomes, should be considered within the scope of the invention.

[0048] Considering FIG. 7, an exemplary flow chart is shown including exemplary operations implemented by various embodiments contemplated herein. In particular, FIG. 7 illustrates exemplary operations that may be performed to train a neural network to generate one or more predictions. The operations illustrated in FIG. 7 may be performed, for example, by the device 900 shown and described in FIG. 9. To perform the operations described below, the device 900 may utilize one or more of a processor 902, a memory 904, communication hardware 906, other components, and / or combinations thereof. It is understood that a user may directly interact with the device 900 via input / output circuitry (not shown) or, alternatively, may be facilitated by the device to interact with the device 900.

[0049] As indicated by operation 702, the device 900 includes means, such as a processor 902, memory 904, or communication hardware 906, for acquiring sequencing variant sample data. The sequencing variant sample data may be based on a sequencing sample dataset and one or more reference genome maps. In some embodiments, the sequencing variant sample data is acquired similarly to the above process / operations described in FIG. 1. As described above, the sequencing variant sample data may include sequencing data for loci and / or data describing multi-omics features from one or more samples for a corresponding subject. In some embodiments, a subject may refer to an entity, such as a human, a cell type, a cancer type, or other biological entity. A sample may be any biological material that can be sequenced, including, but not limited to, a tissue sample, a blood sample, a fluid sample, and / or a cell-specific sample (e.g., a sample taken from a malignant tumor), etc. In some embodiments, the identity of the sample and / or the sequencing performed may be described as associated sequencing variant sample metadata.

[0050] In some embodiments, the sequencing variant sample data may be obtained directly from a storage, such as storage medium 13, and may not require further processing of the data. In some embodiments, the sequencing variant sample data may include variants obtained from whole genome sequencing or whole exome sequencing. The particular variants obtained may depend on whether the corresponding sample was sequenced using a whole genome sequencing process or a whole exome sequencing process. In some embodiments, the sequencing variant sample data is subjected to one-hot encoding as described above with reference to FIG. 1.

[0051] In some embodiments, the sequencing variant sample data may be obtained by generating the sequencing variant sample data. In particular, the device 900 may receive the reference data and the formatted sequencing sample data. In some embodiments, the reference data may be received by identifying one or more reference datasets (e.g., if stored in the storage medium 14) and generating the combined reference data by combining the one or more reference datasets with one or more epigenetic maps. Here, the one or more epigenetic maps may include a DNA methylation map. The combined reference data may then be received by the device 900, and then used as the reference data. In some embodiments, the formatted sequencing sample data may be received by accessing the raw sequencing sample data (e.g., stored in the data storage 10) and formatting the raw sequencing sample data to create the formatted raw sequencing sample data. For example, the raw sequencing sample data may be formatted by storing it in the data storage 10 and then formatting it by the first parser 11 into a format suitable for subsequent processing by the neural network (e.g., TensorFlow report). The formatted raw sequencing sample data may then be received by the device 900 and used as the formatted sequencing sample data. Obtaining the sequencing variant sample data is described in more detail above with reference to FIG.

[0052] The device 900 can then identify shared values ​​between the reference data and the formatted sequencing sample data. Once these shared values ​​are identified, the device 900 can remove one or more shared values ​​between the reference data and the formatted sequencing sample data to generate sequencing variant sample data. As such, the device 900 can selectively erase data that has already been described by the reference data.

[0053] As indicated by operation 704, the device 900 includes a means for acquiring phenotypic data (such as a processor 902, memory 904, or communication hardware 906). The phenotypic data may include disease onset labels and / or control labels for one or more of the conditions known in the corresponding subject. For example, the one or more conditions may include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or combinations thereof associated with the sample data. In some embodiments, the phenotypic data may also describe associated PRS scores for the subject's known conditions. The phenotypic data may be stored in the data storage 10 and / or storage medium 13, etc., together with the sequencing variant sample data and / or the raw sequencing sample data.

[0054] As shown by operation 706, the device 900 includes a means (such as a processor 902, memory 904, or communication hardware 906) for training a neural network using the sequencing variant sample data. In some embodiments, the device 900 may store the neural network and corresponding attributes (e.g., weights, loss functions, hyperparameters, etc.) in a connected memory (such as memory 904), storage medium 17, or remote data storage. When a training program is initiated and / or received by the device 900, the device 900 may access the stored neural network. As described above, the neural network may be trained to generate one or more predictions for a subject, and may further be trained using the sequencing variant sample data obtained in operation 802 as input for the neural network. The neural network may be trained to predict values ​​from the unaltered sequencing sample data and one or more reference genome maps. Training the neural network can include modifying the values ​​at one or more loci for one or more samples in the sequence variant sample data to create modified sequence variant sample data, and using the modified values ​​to train the neural network.Furthermore, one or more layers of the neural network can be trained to predict phenotypic data associated with the sequence variant sample data.As will be further detailed with respect to Figure 6, classification or regression level can be constructed based on phenotypic data, so that the labeled phenotypic data for the sequence variant sample data can be used to train the neural network.

[0055] In particular, a neural network may be trained by training one or more layers of the neural network by performing one or more training iterations. As described above and shown in FIG. 2, the neural network may include one or more layers arranged in a hierarchy of attention encoder mechanisms. Each level in the hierarchy is then subdivided into one or more groups (e.g., submatrices). Then, for each group, one or more parent tokens and in some embodiments further tokens are input to the attention encoder mechanism associated with the particular group and used as input tokens for the attention encoder mechanism as a subsequent level. Now, as shown in FIG. 2 and further described, the graph structure of the neural network moves toward the root node level.

[0056] In some embodiments, input type tokens are added to one or more groups.As mentioned above, input type tokens can correspond to one or more of cell type, tissue type, exome sequence, whole genome sequence, or species.These input type tokens can be added to the lowest grouping level in neural network.

[0057] During each training iteration, one or more actual values ​​of the sequencing variant sample data from one or more loci and one or more samples corresponding to the sequencing variant sample data may be modified. This modified sequencing variant sample data may be used as input for the neural network so that the neural network is trained to predict values ​​from the unaltered sequencing variant sample data and one or more reference genome maps. The neural network may then generate one or more predicted values ​​for these one or more loci and one or more samples, and the neural network may be adjusted based on the one or more predicted values ​​and the one or more actual values. In particular, the loss weights of one or more loss functions in the neural network may be adjusted according to the operation described in FIG. 6. In some embodiments, the value modifying operation may be performed by masking a portion of the values ​​of the sequencing variant sample data (e.g., 10-15 percent of the sequencing variant sample data). In some embodiments, the value-changing operation may be performed by masking a portion of the sequencing variant sample data values ​​(e.g., 10-15 percent of the sequencing variant sample data) and replacing a portion of the sequencing variant sample data values ​​(e.g., 5-10 percent of the sequencing variant sample data).

[0058] The operations described in FIG. 7 above may be repeated any number of times to train or maintain the neural network. In some embodiments, subsequent training of the neural network may update the attributes (e.g., weights, hyperparameters, etc.) of the neural network. These updated attributes and neural network model may be recorded and stored in a corresponding memory (e.g., memory 904) of the device 900. When a model is to be trained and / or retrained, the device 900 may be configured to query and / or access the most recent neural network and attributes so that the most recent model is used.

[0059] Further, although operation 7 describes operations 702 through 706 being performed by apparatus 900, it is recognized that each operation may be performed by a separate and / or remote apparatus. For example, operation 702 for obtaining sequencing variant sample data may be performed by apparatus A, while operation 706 for training the neural network may be performed by apparatus B. Furthermore, data storage 10, storage medium 13, storage medium 14, and / or storage medium 17 of FIG. 1 may each be separate and / or remote storage media or may be shared storage, as described in FIG. 7 above.

[0060] Turning now to Figure 8, an exemplary flow chart is shown including exemplary operations implemented by various embodiments contemplated herein. The operations shown in Figure 8 may be performed, for example, by the device 900 shown and described with respect to Figure 9. To perform the operations described below, the device 900 may utilize one or more of a processor 902, memory 904, communications hardware 906, other components, and / or combinations thereof. It is understood that a user may interact with the device 900 directly via input / output circuitry (not shown) or, alternatively, may be facilitated by a device to interact with the device 900.

[0061] As shown by operation 802, the device 900 includes a means (such as a processor 902, memory 904, or communication hardware 906) for acquiring subject sequencing variant sample data corresponding to a subject. As described in operation 702 above, a subject may refer to an entity such as a human being, a cell type, a cancer type, or other biological entity. In particular, acquiring subject sequencing variant sample data may include sequencing a sample from the subject to create raw sequenced sample data. The sequenced data may be acquired from the subject through whole genome sequencing or whole exome sequencing of the sample from the subject, or the like. The sample may be any biological material that can be sequenced, including, but not limited to, a tissue sample, a blood sample, a fluid sample, and / or a cell-specific sample (e.g., a sample taken from a malignant tumor), and the like. The raw data of the sample may be formatted (e.g., into a TensorFlow record) to create the sequencing sample data of the subject.

[0062] Upon acquiring the sample data, the device 900 may receive the reference data. Similarly, as described above for operation 702, the reference data may be received by identifying one or more reference data sets (e.g., if stored in the storage medium 14) and combining the one or more reference data sets with one or more epigenetic maps to generate combined reference data. Here, the one or more epigenetic maps may include DNA methylation maps. The combined reference data may be received by the device 900, and then used as reference data. The device 900 may then identify values ​​shared between the reference data and the sample data for the subject. The device 900 may then remove one or more values ​​shared between the reference data and the sample data to generate subject sequencing variant sample data for the subject.

[0063] As shown by operation 804, the device 900 includes a means (such as a processor 902, memory 904, or communication hardware 906) for generating a predicted value of a genome map for a subject. In particular, the device 900 may input subject sequencing variant sample data to a neural network that has been trained, such as through the operations described in FIG. 7 above. The device 900 may access and utilize the neural network from associated storage (such as memory 904, storage medium 17, or remote data storage). The neural network may be configured to predict a value of a genome map for a subject based on the input sequencing variant sample data. As such, the neural network may be configured to infer a value of a genome map, even though only variant data is provided to the neural network as input.

[0064] As shown by operation 806, the device 900 includes a means (such as a processor 902, memory 904, or communication hardware 906) for generating one or more predicted states for the subject. In some embodiments, the neural network may also be configured to generate one or more predicted states for the subject based on the prediction values. For example, as described in more detail in FIG. 6 above, the neural network may be trained using a classification or regression template to generate one or more predicted states by scoring the likelihood of states for each subject. In particular, if a subject is determined to have a score that meets a score threshold for a particular state, the neural network may predict a state for the subject. In some embodiments, the neural network may be configured to generate a predicted polygenic risk score of a predicted state for the subject. The neural network may generate a predicted polygenic risk score for a state based on one or more predicted risk scores for the state.

[0065] In some embodiments, the neural network may generate and output one or more predictions and / or predictive values ​​of the genomic map for the subject as a report and transmit to one or more computing devices (e.g., computing devices) of a clinician, geneticist, researcher, patient, etc. In some embodiments, the neural network may generate and output one or more predictions and / or predictive values ​​of the genomic map for the subject as input to other downstream models.

[0066] Exemplary Implementation System The methods described herein can be implemented in various systems.For example, in some embodiments, the system can be used to obtain sequence variant sample data, obtain phenotype data, train neural networks, obtain subject sequence variant sample data, generate predictions of genome maps for subjects, determine one or more predicted states for subjects, etc.The following device 900 can be used to perform any combination of these operations.In some embodiments, one or more of the above devices (such as the training iterator described in Figure 5) can be implemented as device 900.

[0067] As shown in device 900 in FIG. 9, the system may include one or more system devices, which may be embodied by one or more computing devices or servers. As shown in FIG. 9, device 900 may include a processor 902, memory 904, and communication hardware 906, each of which is described in further detail below. Although FIG. 9 only illustrates a variety of components connected to device 900, it is understood that device 900 may further include a bus (not explicitly shown in FIG. 9) for communicating information between any combination of the various components of device 900. Device 900 may be configured to perform a variety of operations described above.

[0068] The processor 902 (and / or any other processors that aid or otherwise associate with the processor) may communicate with the memory 904 via a bus for communicating information between components of the apparatus. The processor 902 may be embodied in a number of different ways, and may include, for example, one or more processing devices configured to perform independently. Additionally, the processor may include one or more processors arranged in tandem via a bus to enable independent execution of software instructions, pipelining, and / or multithreading. Use of the term "processor" may be understood to include a single core processor, a multicore processor, multiple processors in the apparatus 900, a remote or "cloud" processor, or any combination thereof.

[0069] The processor 902 may be configured to execute software instructions stored in memory 904 or otherwise accessible to the processor (e.g., software instructions stored in a separate storage device). In some cases, the processor may be configured to execute hard-coded functionality. As such, whether configured by hardware or software methods, or a combination of hardware and software, the processor 902 represents an entity (e.g., physically embodied in circuitry) that can be appropriately configured to perform operations in accordance with various embodiments of the present invention. Alternatively, as another example, if the processor 902 is embodied as an executor of software instructions, the software instructions may specifically configure the processor 902 to perform the algorithms and / or operations described herein when the software instructions are executed.

[0070] The memory 904 may be non-transitory, for example, including one or more volatile and / or non-volatile memories. In other words, for example, the memory 904 may be an electronic storage device (e.g., a computer-readable storage medium). The memory 904 may be configured to store information, data, content, applications, or software instructions, etc., to enable the device to perform a wide variety of functions according to the exemplary embodiments contemplated herein. Furthermore, the data storage 10, the storage medium 13, the storage medium 14, and / or the storage medium 17 of FIG. 1 may be embodied by the memory 904. The data storage 10, the storage medium 13, the storage medium 14, and / or the storage medium 17 may each be separate and / or remote storage media, or may be shared storage.

[0071] The communications hardware 906 may be any means, such as devices or circuits embodied in either hardware or a combination of hardware and software, configured to communicate with the apparatus 900 to receive and / or send data from / to a network and / or any other device, circuit, or module. In this regard, the communications hardware 906 may include, for example, a network interface for enabling communication with a wired or wireless communications network. For example, the communications hardware 906 may include one or more network interface cards, antennas, buses, switches, routers, modems, and supporting hardware and / or software, or any other devices suitable for enabling communication over a network. Additionally, the communications hardware 906 may include processing circuitry for handling receipt of signals received from a network or forwarding such signals to the network.

[0072] The communications hardware 906 may be configured to provide output to a user and, in some embodiments, to receive user input indications. The communications hardware 906 may include a user interface (such as a display) and may further include components that manage the use of the user interface (such as a web browser, a mobile application, or a dedicated user device). In some embodiments, the communications hardware 906 may include a keyboard, a mouse, a touch screen, a touch area, soft keys, a microphone, a speaker, and / or other input / output mechanisms. The communications hardware 906 may utilize the processor 902 to control one or more functions of one or more of these user interface elements via software instructions (e.g., application software and / or system software, e.g., firmware) stored in a memory (e.g., memory 904) accessible to the processor 902.

[0073] conclusion Many modifications and other embodiments of the inventions described herein will occur to those skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing description and the associated drawings. It is therefore to be understood that the invention is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, while the foregoing description and the associated drawings describe exemplary embodiments in the context of certain exemplary combinations of elements and / or functions, it should be recognized that different combinations of elements and / or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, it is also contemplated that different combinations of elements and / or functions may be set forth in some of the appended claims than those set forth above. Although specific terms have been used herein, these terms are used in a generic and descriptive sense only and not for purposes of limitation.

[0074] Item 1. A method comprising: obtaining sequencing variant sample data, wherein the sequencing variant sample data is based on sequencing sample data and one or more reference genome maps; and training a neural network using the sequencing variant sample data, wherein training the neural network comprises altering values ​​of one or more loci for one or more samples in the sequencing variant sample data to create altered sequencing variant sample data, and using the altered sequencing variant sample data as input during training of the neural network, such that the neural network is trained to predict values ​​from unaltered sequencing variant sample data and the one or more reference genome maps.

[0075] Item 2. The method of any of items 1, wherein training the neural network comprises: training one or more layers of the neural network by performing one or more training iterations, each training iteration modifying one or more actual values ​​of the sequencing variant sample data from the one or more loci and the one or more samples corresponding to the sequencing variant sample data; generating one or more predicted values ​​for the one or more loci and the one or more samples using the neural network, and adjusting the neural network based on the one or more predicted values ​​and the one or more actual values.

[0076] Item 3. The method according to any one of items 1 to 2, further comprising obtaining phenotype data associated with the sequencing variant sample data; and training one or more layers of the neural network to predict the phenotype data.

[0077] Item 4. A method according to any of Items 1 to 3, wherein (i) the one or more layers are arranged in a hierarchy of attention encoder mechanisms, (ii) the levels in the hierarchy are subdivided into one or more groups, and (iii) for each group, one or more parent tokens from another attention encoder mechanism associated with the group are used as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

[0078] Item 5. The method of any one of items 1 to 4, further comprising adding an input type token to the one or more groups.

[0079] Item 6. The method of any one of Items 1 to 5, wherein the input type token corresponds to an input type, and the input type corresponds to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

[0080] Item 7. The method of any one of Items 1 to 6, wherein the sequencing variant sample data comprises variants obtained from whole exome sequencing.

[0081] Item 8. The method of any one of items 1 to 7, wherein the sequencing variant sample data includes variants obtained from whole genome sequencing.

[0082] Item 9. The method of any of items 1 to 8, wherein the phenotypic data comprises disease incidence and control labels for one or more conditions.

[0083] Item 10. The method of any of items 1-9, wherein the one or more conditions comprises breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0084] Item 11. The method according to any one of Items 1 to 10, wherein modifying the value further comprises masking a portion of the value of the sequencing variant sample data.

[0085] Item 12. The method of any one of Items 1 to 11, wherein modifying the value further comprises masking a first portion of the value of the sequencing variant sample data and replacing a second portion of the value of the sequencing variant sample data.

[0086] Item 13. The method according to any one of Items 1 to 12, wherein the sequencing variant sample data is one-hot encoded.

[0087] Item 14. The method of any one of items 1 to 13, wherein obtaining the sequencing variant sample data further comprises receiving reference data and formatted sequencing sample data; identifying one or more values ​​shared between the reference data and the formatted sequencing sample data; and removing one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data.

[0088] Item 15. The method of any one of Items 1 to 14, wherein receiving the reference data further comprises: identifying one or more reference datasets; and generating combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data.

[0089] Item 16. The method according to any one of Items 1 to 15, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

[0090] Item 17. The method of any of Items 1 to 16, wherein receiving the formatted sequencing sample data further comprises: accessing raw sequencing sample data; and formatting the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted raw sequencing sample data is used to obtain the formatted sequencing sample data and create the sequencing variant sample data.

[0091] Item 18. The method according to any one of Items 1 to 17, further comprising obtaining subject sequencing variant sample data corresponding to sequenced sample data for a subject; and using the neural network to generate a predicted value of a genome map for the subject based on the subject sequencing variant sample data.

[0092] Item 19. The method of any one of items 1 to 18, further comprising determining one or more predicted conditions for the subject.

[0093] Item 20. The method of any of items 1 to 19, wherein the one or more predicted conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0094] Item 21. The method of any of Items 1 to 20, wherein the one or more predicted conditions comprise a polygenic risk score determined for a particular condition of the subject, and the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

[0095] Item 22. The method of any one of Items 1 to 21, further comprising sequencing a sample from the subject to generate raw sequenced sample data; and formatting the raw sequenced sample data to generate the sequenced sample data for the subject.

[0096] Item 23. The method of any one of items 1 to 22, further comprising receiving reference data; identifying one or more shared values ​​between the reference data and the sample data for the subject; and removing one or more of the one or more shared values ​​between the reference data and the formatted sequencing sample data to obtain the subject sequencing variant sample data for the subject.

[0097] Item 24. The method according to any one of items 1 to 23, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

[0098] Item 25. An apparatus comprising a processor and a memory storing software instructions, which when executed by the processor, causes the apparatus to perform the steps of: obtaining sequencing variant sample data, wherein the sequencing variant sample data is based on sequencing sample data and one or more reference genome maps; and training the neural network using the sequencing variant sample data, wherein training the neural network comprises modifying values ​​of one or more loci for one or more samples in the sequencing variant sample data to create modified sequencing variant sample data, and using the modified sequencing variant sample data as input during training of the neural network, such that the neural network is trained to predict values ​​from unaltered sequencing variant sample data and the one or more reference genome maps.

[0099] Item 26. The apparatus of item 25, wherein the software instructions, when executed by the processor, train a neural network, further cause the apparatus to train one or more layers of the neural network by performing one or more training iterations, wherein each performing each training iteration causes the apparatus to further modify one or more actual values ​​of sequencing variant sample data from the one or more loci or one or more samples corresponding to the sequencing variant sample data; generate one or more predicted values ​​for the one or more loci and the one or more samples using a neural network, and adjust the neural network based on the one or more predicted values ​​and the one or more actual values.

[0100] Item 27. The apparatus of any one of items 25 to 26, wherein the software instructions, when executed by the processor, further cause the apparatus to obtain phenotype data associated with the sequencing variant sample data; and train one or more layers of a neural network to predict the phenotype data.

[0101] Item 28. An apparatus described in any of Items 25 to 27, (i) arranging the one or more layers into a hierarchy of attention encoder mechanisms, (ii) subdividing levels in the hierarchy into one or more groups, and (iii) for each group, using one or more parent tokens from another attention encoder mechanism associated with the group as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

[0102] Item 29. An apparatus according to any one of items 25 to 28, wherein the software instructions, when executed by the processor, further cause the apparatus to append an input type token to the one or more groups.

[0103] Item 30. The apparatus of any of items 25 to 29, wherein the input type token corresponds to an input type, and the input type corresponds to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

[0104] Item 31. The apparatus of any of Items 25 to 30, wherein the sequencing variant sample data includes variants obtained from whole exome sequencing.

[0105] Item 32. The apparatus of any one of items 25 to 31, wherein the sequencing variant sample data includes variants obtained from whole genome sequencing.

[0106] Item 33. The device of any of items 25 to 32, wherein the phenotype data includes disease onset and control labels for one or more conditions.

[0107] Item 34. The device of any of items 25-33, wherein the one or more conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0108] Item 35. An apparatus described in any of items 25 to 34, wherein the software instructions, when executed by the processor and when changing the value, further cause the apparatus to mask a portion of the value of the sequencing variant sample data.

[0109] Item 36. An apparatus described in any of Items 25 to 35, wherein the software instructions, when executed by the processor and when modifying the value, further cause the apparatus to mask a first portion of the value of the sequencing variant sample data and replace a second portion of the value of the sequencing variant sample data.

[0110] Item 37. The apparatus according to any one of items 25 to 36, wherein the sequencing variant sample data is one-hot encoded.

[0111] Item 38. The apparatus of any one of items 25 to 37, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the apparatus to receive reference data and formatted sequencing sample data; identify one or more values ​​shared between the reference data and the formatted sequencing sample data; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data.

[0112] Item 39. The apparatus of any one of Items 25 to 38, wherein the software instructions, when executed by the processor and when receiving the reference data, further cause the apparatus to identify one or more reference datasets; and generate combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data.

[0113] Item 40. The apparatus of any of items 25 to 39, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

[0114] Item 41. An apparatus described in any of items 25 to 40, wherein the software instructions, when executed by the processor and when receiving the formatted sequencing sample data, further cause the apparatus to access raw sequencing sample data; format the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted sequencing sample data is obtained using the formatted raw sequencing sample data to create the sequencing variant sample data.

[0115] Item 42. The apparatus of any one of Items 25 to 41, wherein when the software instructions are executed by the processor and when the sequencing variant sample data is acquired, the apparatus further causes the apparatus to acquire subject sequencing variant sample data, where the subject sequencing variant sample data corresponds to sequenced sample data for a subject; and uses the neural network to generate a predicted value of a genome map for the subject based on the subject sequencing variant sample data.

[0116] Item 43. The device of any one of items 25 to 42, wherein the software instructions, when executed by the processor and acquiring the sequencing variant sample data, further cause the device to determine one or more predicted states for the subject.

[0117] Item 44. The device of any of items 25-43, wherein the one or more predicted conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0118] Item 45. The device of any of items 25 to 44, wherein the one or more predicted conditions include a polygenic risk score determined for a particular condition of the subject, and the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

[0119] Item 46. The apparatus of any one of items 25 to 45, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the apparatus to sequence a sample from the subject to generate raw sequenced sample data; and format the raw sequenced sample data to generate sequenced sample data for the subject.

[0120] Item 47. The apparatus of any one of Items 25 to 46, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the apparatus to receive reference data; identify one or more values ​​shared between the reference data and the sample data for the subject; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data to generate subject sequencing variant sample data for the subject.

[0121] Item 48. The apparatus of any of items 25 to 47, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

[0122] Item 49. A computer program product comprising at least one non-transitory computer-readable storage medium having stored thereon software instructions, which when executed by an apparatus causes the apparatus to obtain sequencing variant sample data, where the sequencing variant sample data is based on sequencing sample data and one or more reference genome maps; train a neural network using the sequencing variant sample data, where training the neural network comprises modifying values ​​of one or more loci for one or more samples in the sequencing variant sample data to create modified sequencing variant sample data, and using the modified sequencing variant sample data as an input during training of the neural network, such that the neural network is trained to predict values ​​from unaltered sequencing variant sample data and the one or more reference genome maps.

[0123] Item 50. The computer program product of item 49, wherein the software instructions, when executed by the processor, further cause the device to train one or more layers of the neural network by performing one or more training iterations, where each performing each training iteration causes the device to further modify one or more actual values ​​of sequencing variant sample data from the one or more loci or one or more samples corresponding to the sequencing variant sample data; generate one or more predicted values ​​for the one or more loci and the one or more samples using a neural network, and adjust the neural network based on the one or more predicted values ​​and the one or more actual values.

[0124] Item 51. The computer program product of any one of items 49 to 50, wherein the software instructions, when executed by the processor, further cause the device to obtain phenotype data associated with the sequencing variant sample data; and train one or more layers of a neural network to predict the phenotype data.

[0125] Item 52. A computer program product described in any of Items 49 to 51, wherein (i) the one or more layers are arranged in a hierarchy of attention encoder mechanisms, (ii) the levels in the hierarchy are subdivided into one or more groups, and (iii) for each group, one or more parent tokens from another attention encoder mechanism associated with the group are used as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

[0126] Item 53. The computer program product of any of items 49 to 52, wherein the software instructions, when executed by the processor, further cause the device to append an input type token to the one or more groups.

[0127] Item 54. The computer program product of any of Items 49 to 53, wherein the input type token corresponds to an input type, and the input type corresponds to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

[0128] Item 55. The computer program product of any of Items 49 to 54, wherein the sequencing variant sample data includes variants obtained from whole exome sequencing.

[0129] Item 56. The computer program product of any of Items 49 to 55, wherein the sequencing variant sample data includes variants obtained from whole genome sequencing.

[0130] Item 57. The computer program product of any of items 49 to 56, wherein the phenotype data comprises disease onset and control labels for one or more conditions.

[0131] Item 58. The computer program product of any of items 49 to 57, wherein the one or more conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0132] Item 59. A computer program product according to any one of items 49 to 58, wherein the software instructions, when executed by the processor and when modifying the value, further cause the device to mask a portion of the value of the sequencing variant sample data.

[0133] Item 60. The computer program product of any of Items 49 to 59, wherein the software instructions, when executed by the processor and when modifying the value, further cause the device to mask a first portion of the value of the sequencing variant sample data and replace a second portion of the sequencing variant sample data.

[0134] Item 61. The computer program product according to any one of items 49 to 60, wherein the sequencing variant sample data is one-hot encoded.

[0135] Item 62. The computer program product of any one of items 49 to 61, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the device to receive reference data and formatted sequencing sample data; identify one or more values ​​shared between the reference data and the formatted sequencing sample data; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data.

[0136] Item 63. The computer program product of any one of Items 49 to 62, wherein the software instructions, when executed by the processor and when receiving the reference data, further cause the device to identify one or more reference datasets; and generate combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data.

[0137] Item 64. The computer program product of any of items 49 to 63, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

[0138] Item 65. The computer program product of any of Items 49 to 64, wherein the software instructions, when executed by the processor and when receiving the formatted sequencing sample data, further cause the device to access raw sequencing sample data; format the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted sequencing sample data is obtained using the formatted raw sequencing sample data to create the sequencing variant sample data.

[0139] Item 66. The computer program product of any one of Items 49 to 65, wherein when the software instructions are executed by the processor and when the sequencing variant sample data is acquired, the software instructions further cause the device to acquire subject sequencing variant sample data, where the subject sequencing variant sample data corresponds to sequenced sample data for a subject; and generate a genome map prediction for the subject based on the subject sequencing variant sample data using the neural network.

[0140] Item 67. The computer program product of any one of items 49 to 66, wherein the software instructions, when executed by the processor and acquiring the sequencing variant sample data, further cause the device to determine one or more predicted states for the subject.

[0141] Item 68. The computer program product of any of items 49 to 67, wherein the one or more predicted conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0142] Item 69. The computer program product of any of items 49 to 68, wherein the one or more predicted conditions comprise a polygenic risk score determined for a particular condition of the subject, and the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

[0143] Item 70. The computer program product of any one of items 49 to 69, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the device to sequence a sample from the subject to generate raw sequenced sample data; and format the raw sequenced sample data to generate sequenced sample data for the subject.

[0144] Item 71. The computer program product of any of items 49 to 70, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the device to receive reference data; identify one or more values ​​shared between the reference data and the sample data for the subject; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data to generate subject sequencing variant sample data for the subject.

[0145] Item 72. The computer program product of any of items 49 to 71, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

[0146] Item 73. A method comprising: obtaining subject sequencing variant sample data, the subject sequencing variant sample data corresponding to sequenced sample data for a subject; generating a predicted value of a genome map for the subject based on the subject sequencing variant sample data using a neural network trained to predict values ​​of a genome map generated from sample data and one or more reference genome maps; and determining one or more predicted states for the subject based on the predicted value.

[0147] Item 74. The method of item 73, further comprising sequencing a sample from the subject to generate raw sequenced sample data; and formatting the raw sequenced sample data to generate sequenced sample data for the subject.

[0148] Item 75. The method of any of Items 73-74, further comprising receiving reference data; identifying one or more shared values ​​between the reference data and the sample data for the subject; and removing one or more of the one or more shared values ​​between the reference data and the formatted sequencing sample data to generate subject sequencing variant sample data for the subject.

[0149] Item 76. The method of any of items 73 to 75, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

[0150] Item 77. The method of any of items 73 to 76, wherein the one or more predicted conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0151] Item 78. The method of any of items 73 to 77, wherein the one or more predicted conditions comprise a polygenic risk score determined for a particular condition of the subject, and the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

[0152] Item 79. The method of any of items 73 to 78, further comprising: acquiring sequencing variant sample data on the apparatus, the sequencing variant sample data being based on sequencing sample data and one or more reference genome maps; and training the neural network using the sequencing variant sample data, wherein training the neural network comprises modifying values ​​of one or more loci for one or more samples in the sequencing variant sample data to create modified sequencing variant sample data, and using the modified sequencing variant sample data as input during training of the neural network, such that the neural network is trained to predict values ​​from unaltered sequencing variant sample data and the one or more reference genome maps.

[0153] Item 80. The method of any of items 73-79, wherein training the neural network comprises training one or more layers of the neural network by performing one or more training iterations, each training iteration modifying one or more actual values ​​of sequencing variant sample data from the one or more loci or one or more samples corresponding to the sequencing variant sample data; generating one or more predicted values ​​for the one or more loci and the one or more samples using a neural network, and adjusting the neural network based on the one or more predicted values ​​and the one or more actual values.

[0154] Item 81. The method of any one of items 73 to 80, further comprising obtaining phenotype data associated with the sequencing variant sample data; and training one or more layers of a neural network to predict the phenotype data.

[0155] Item 82. A method according to any of Items 73 to 81, wherein (i) the one or more layers are arranged in a hierarchy of attention encoder mechanisms, (ii) the levels in the hierarchy are subdivided into one or more groups, and (iii) for each group, one or more parent tokens from another attention encoder mechanism associated with the group are used as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

[0156] Item 83. The method of any one of items 73 to 82, further comprising adding an input type token to the one or more groups.

[0157] Item 84. The method of any of Items 73 to 83, wherein the input type token corresponds to an input type, and the input type corresponds to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

[0158] Item 85. The method of any one of items 73 to 84, wherein the sequencing variant sample data comprises variants obtained from whole exome sequencing.

[0159] Item 86. The method of any one of items 73 to 85, wherein the sequencing variant sample data comprises variants obtained from whole genome sequencing.

[0160] Item 87. The method of any of items 73 to 86, wherein the phenotypic data comprises disease incidence and control labels for one or more conditions.

[0161] Item 88. The method of any of items 73-87, wherein the one or more conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0162] Item 89. The method of any of items 73 to 88, wherein modifying the value further comprises masking a portion of the value of the sequencing variant sample data.

[0163] Item 90. The method of any of Items 73 to 89, wherein modifying the value further comprises masking a first portion of the value of the sequencing variant sample data and replacing a second portion of the value of the sequencing variant sample data.

[0164] Item 91. The method according to any one of items 73 to 90, wherein the sequencing variant sample data is one-hot encoded.

[0165] Item 92. The method of any of Items 73 to 91, wherein obtaining the sequencing variant sample data further comprises receiving reference data and formatted sequencing sample data; identifying one or more values ​​shared between the reference data and the formatted sequencing sample data; and removing one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data.

[0166] Item 93. The method of any of Items 73 to 92, wherein receiving the reference data further comprises: identifying one or more reference datasets; and generating combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data.

[0167] Item 94. The method of any of Items 73 to 93, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

[0168] Item 95. The method of any of Items 73 to 94, wherein receiving the formatted sequencing sample data further comprises: accessing raw sequencing sample data; and formatting the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted raw sequencing sample data is used to obtain the formatted sequencing sample data and create the sequencing variant sample data.

[0169] Item 96. An apparatus comprising a processor and a memory storing software instructions, which when executed by the processor, cause the apparatus to obtain subject sequencing variant sample data, where the subject sequencing variant sample data corresponds to sequenced sample data for a subject; generate a predicted value of a genome map for the subject based on the subject sequencing variant sample data using a neural network trained to predict values ​​of a genome map generated from sample data and one or more reference genome maps; and determine one or more predicted states for the subject based on the predicted value.

[0170] Item 97. The device of Item 96, wherein the memory storing the software instructions, when executed by the processor, further causes the device to sequence a sample from the subject to generate raw sequenced sample data; and format the raw sequenced sample data to generate sequenced sample data for the subject.

[0171] Item 98. The apparatus of any of items 96 to 97, wherein the memory storing the software instructions, when executed by the processor, further causes the apparatus to receive reference data; identify one or more values ​​shared between the reference data and sample data for the subject; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data to generate subject sequencing variant sample data for the subject.

[0172] Item 99. The apparatus of any of items 96 to 98, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

[0173] Item 100. The apparatus of any of items 96-99, wherein the one or more predicted conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0174] Item 101. The device of any of items 96-100, wherein the one or more predicted conditions include a polygenic risk score determined for a particular condition of the subject, and the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

[0175] Item 102. The device of any of items 96-101, wherein the memory storing software instructions, when executed by the processor, further causes the device to: obtain sequencing variant sample data, the sequencing variant sample data based on sequencing sample data and one or more reference genome maps; and train the neural network using the sequencing variant sample data, wherein training the neural network includes modifying values ​​of one or more loci for one or more samples in the sequencing variant sample data to create modified sequencing variant sample data, and using the modified sequencing variant sample data as input during training of the neural network, such that the neural network is trained to predict values ​​from unaltered sequencing variant sample data and the one or more reference genome maps.

[0176] Item 103. The apparatus of any of items 96-102, wherein the software instructions, when executed by the processor, train a neural network, further cause the apparatus to train one or more layers of the neural network by performing one or more training iterations, where each performing each training iteration causes the apparatus to further modify one or more actual values ​​of sequencing variant sample data from the one or more loci or one or more samples corresponding to the sequencing variant sample data; generate one or more predicted values ​​for the one or more loci and the one or more samples using a neural network, and adjust the neural network based on the one or more predicted values ​​and the one or more actual values.

[0177] Item 104. The apparatus of any of Items 96 to 103, wherein the software instructions, when executed by the processor, further cause the apparatus to obtain phenotype data associated with the sequencing variant sample data; and train one or more layers of a neural network to predict the phenotype data.

[0178] Item 105. An apparatus described in any of Items 96 to 104, (i) arranging the one or more layers into a hierarchy of attention encoder mechanisms, (ii) subdividing levels in the hierarchy into one or more groups, and (iii) for each group, using one or more parent tokens from another attention encoder mechanism associated with the group as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

[0179] Item 106. An apparatus described in any of Items 96 to 105, wherein the software instructions, when executed by the processor, further cause the apparatus to append an input type token to the one or more groups.

[0180] Item 107. The apparatus of any of items 96 to 106, wherein the input type token corresponds to an input type, the input type corresponding to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

[0181] Item 108. The apparatus of any of items 96 to 107, wherein the sequencing variant sample data includes variants obtained from whole exome sequencing.

[0182] Item 109. The apparatus of any of items 96 to 108, wherein the sequencing variant sample data includes variants obtained from whole genome sequencing.

[0183] Item 110. The apparatus of any of items 96-109, wherein the phenotype data includes disease onset and control labels for one or more conditions.

[0184] Item 111. The device of any of items 96-110, wherein the one or more conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0185] Item 112. An apparatus described in any of items 96 to 111, wherein the software instructions, when executed by the processor and when modifying the value, further cause the apparatus to mask a portion of the value of the sequencing variant sample data.

[0186] Item 113. An apparatus described in any of items 96 to 112, wherein the software instructions, when executed by the processor and when modifying the value, further cause the apparatus to mask a first portion of the value of the sequencing variant sample data and replace a second portion of the value of the sequencing variant sample data.

[0187] Item 114. The apparatus according to any one of items 96 to 113, wherein the sequencing variant sample data is one-hot encoded.

[0188] Item 115. The apparatus of any of items 96 to 114, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the apparatus to receive reference data and formatted sequencing sample data; identify one or more values ​​shared between the reference data and the formatted sequencing sample data; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data.

[0189] Item 116. The apparatus of any one of Items 96 to 115, wherein the software instructions, when executed by the processor and when receiving the reference data, further cause the apparatus to identify one or more reference datasets; and generate combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data.

[0190] Item 117. The apparatus of any of items 96 to 116, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

[0191] Item 118. An apparatus described in any of items 96 to 117, wherein the software instructions, when executed by the processor and when receiving the formatted sequencing sample data, further cause the apparatus to access raw sequencing sample data; format the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted sequencing sample data is obtained using the formatted raw sequencing sample data to create the sequencing variant sample data.

[0192] Item 119. A computer program product comprising at least one non-transitory computer-readable storage medium having stored thereon software instructions, which, when executed by an apparatus, causes the apparatus to obtain subject sequencing variant sample data, where the subject sequencing variant sample data corresponds to sequenced sample data for a subject; generate a predicted value of a genome map for the subject based on the subject sequencing variant sample data using a neural network trained to predict values ​​of a genome map generated from sample data and one or more reference genome maps; and determine one or more predicted states for the subject based on the predicted value.

[0193] Item 120. The computer program product of Item 119, wherein the memory storing the software instructions, when executed by the processor, further causes the device to sequence a sample from the subject to generate raw sequenced sample data; and format the raw sequenced sample data to generate sequenced sample data for the subject.

[0194] Item 121. The computer program product of any of items 119 to 120, wherein the memory storing the software instructions, when executed by the processor, further causes the device to receive reference data; identify one or more values ​​shared between the reference data and the sample data for the subject; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data to generate subject sequencing variant sample data for the subject.

[0195] Item 122. The computer program product of any of items 119 to 121, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

[0196] Item 123. The computer program product of any of items 119 to 122, wherein the one or more predicted conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0197] Item 124. The computer program product of any of items 119 to 123, wherein the one or more predicted conditions comprise a polygenic risk score determined for a particular condition of the subject, and the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

[0198] Item 125. The computer program product of any of items 119 to 124, wherein the memory storing software instructions, when executed by the processor, further causes the device to: obtain sequencing variant sample data, the sequencing variant sample data being based on sequencing sample data and one or more reference genome maps; and train the neural network using the sequencing variant sample data, wherein training the neural network includes modifying values ​​of one or more loci for one or more samples in the sequencing variant sample data to create modified sequencing variant sample data, and using the modified sequencing variant sample data as input during training of the neural network, such that the neural network is trained to predict values ​​from the unchanged sequencing variant sample data and the one or more reference genome maps.

[0199] Item 126. The computer program product of any of items 119 to 125, wherein when the software instructions are executed by the processor to train a neural network, the software instructions further cause the device to train one or more layers of the neural network by performing one or more training iterations, where each performing each training iteration causes the device to further modify one or more actual values ​​of sequencing variant sample data from the one or more loci or one or more samples corresponding to the sequencing variant sample data; generate one or more predicted values ​​for the one or more loci and the one or more samples using a neural network, and adjust the neural network based on the one or more predicted values ​​and the one or more actual values.

[0200] Item 127. The computer program product of any one of items 119 to 126, wherein the software instructions, when executed by the processor, further cause the device to obtain phenotype data associated with the sequencing variant sample data; and train one or more layers of a neural network to predict the phenotype data.

[0201] Item 128. A computer program product described in any of Items 119 to 127, wherein (i) the one or more layers are arranged in a hierarchy of attention encoder mechanisms, (ii) the levels in the hierarchy are subdivided into one or more groups, and (iii) for each group, one or more parent tokens from another attention encoder mechanism associated with the group are used as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

[0202] Item 129. A computer program product according to any one of items 119 to 128, wherein the software instructions, when executed by the processor, further cause the device to append an input type token to the one or more groups.

[0203] Item 130. The computer program product of any of Items 119 to 129, wherein the input type token corresponds to an input type, the input type corresponding to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

[0204] Item 131. The computer program product of any of items 119 to 130, wherein the sequencing variant sample data comprises variants obtained from whole exome sequencing.

[0205] Item 132. The computer program product of any of Items 119 to 131, wherein the sequencing variant sample data comprises variants obtained from whole genome sequencing.

[0206] Item 133. The computer program product of any of items 119 to 132, wherein the phenotype data comprises disease onset and control labels for one or more conditions.

[0207] Item 134. The computer program product of any of items 119 to 133, wherein the one or more conditions include breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

[0208] Item 135. A computer program product according to any one of items 119 to 134, wherein the software instructions, when executed by the processor and when modifying the value, further cause the device to mask a portion of the value of the sequencing variant sample data.

[0209] Item 136. A computer program product according to any one of items 119 to 135, wherein the software instructions, when executed by the processor and when modifying the value, further cause the device to mask a first portion of the value of the sequencing variant sample data and replace a second portion of the sequencing variant sample data.

[0210] Item 137. The computer program product according to any one of items 119 to 136, wherein the sequencing variant sample data is one-hot encoded.

[0211] Item 138. The computer program product of any one of items 119 to 137, wherein the software instructions, when executed by the processor and when obtaining the sequencing variant sample data, further cause the device to receive reference data and formatted sequencing sample data; identify one or more values ​​shared between the reference data and the formatted sequencing sample data; and remove one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data.

[0212] Item 139. The computer program product of any of Items 119 to 138, wherein the software instructions, when executed by the processor and when receiving the reference data, further cause the device to identify one or more reference datasets; and generate combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data.

[0213] Item 140. The computer program product of any of items 119 to 139, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

[0214] Item 141. A computer program product according to any one of items 119 to 140, wherein the software instructions, when executed by the processor and when receiving the formatted sequencing sample data, further cause the device to access raw sequencing sample data; format the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted sequencing sample data is obtained using the formatted raw sequencing sample data to create the sequencing variant sample data.

Claims

1. Obtaining sequencing variant sample data, wherein the sequencing variant sample data is based on the sequencing sample data and one or more reference genome maps; and training a neural network using the sequencing variant sample data, wherein training the neural network comprises: altering values ​​of one or more loci for one or more samples in the sequencing variant sample data to generate altered sequencing variant sample data; and using the altered sequencing variant sample data as input during training of the neural network, such that the neural network is trained to predict values ​​from unaltered sequencing variant sample data and the one or more reference genome maps. Training, including A method comprising:

2. Training the neural network includes training one or more layers of the neural network by performing one or more training iterations, each training iteration comprising: modifying one or more actual values ​​of the sequencing variant sample data from the one or more loci and the one or more samples corresponding to the sequencing variant sample data; generating one or more predictions for the one or more loci and the one or more samples using the neural network; and adjusting the neural network based on the one or more predicted values ​​and the one or more actual values. The method of claim 1 , comprising training comprising:

3. obtaining phenotypic data associated with said sequencing variant sample data; and training one or more layers of said neural network to predict said phenotypic data. The method of claim 1 further comprising:

4. 4. The method of claim 3, further comprising: (i) arranging the one or more layers into a hierarchy of attention encoder mechanisms; (ii) subdividing levels in the hierarchy into one or more groups; and (iii) for each group, using one or more parent tokens from another attention encoder mechanism associated with the group as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

5. The method of claim 4 , further comprising appending an input type token to the one or more groups.

6. 6. The method of claim 5, wherein the input type token corresponds to an input type, the input type corresponding to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

7. 4. The method of claim 3, wherein the sequencing variant sample data comprises variants obtained from whole exome sequencing.

8. 4. The method of claim 3, wherein the sequencing variant sample data comprises variants obtained from whole genome sequencing.

9. The method of claim 3 , wherein the phenotypic data comprises disease onset and control labels for one or more conditions.

10. 10. The method of claim 9, wherein the one or more conditions comprise breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

11. 2. The method of claim 1, wherein modifying the value further comprises masking a portion of the value of the sequencing variant sample data.

12. 2. The method of claim 1 , wherein modifying the value further comprises masking a first portion of the value of the sequencing variant sample data and replacing a second portion of the value of the sequencing variant sample data.

13. The method of claim 1 , wherein the sequencing variant sample data is one-hot encoded.

14. Obtaining the sequencing variant sample data comprises: receiving the reference data and the formatted sequencing sample data; identifying one or more values ​​shared between the reference data and the formatted sequencing sample data; and Removing one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data. The method of claim 1 further comprising:

15. receiving the reference data, Identifying one or more reference datasets; and generating combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data. The method of claim 14 further comprising:

16. 16. The method of claim 15, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

17. receiving the formatted sequencing sample data; Accessing the raw sequencing sample data; and 15. The method of claim 14, further comprising formatting the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted raw sequencing sample data is used to obtain the formatted sequencing sample data and create the sequencing variant sample data.

18. Obtaining subject sequencing variant sample data corresponding to sequenced sample data for a subject; and using the neural network to generate a predicted genome map for the subject based on the subject sequencing variant sample data. The method of claim 1 further comprising:

19. 20. The method of claim 18, further comprising determining one or more predicted states for the subject.

20. 20. The method of claim 19, wherein the one or more predicted conditions comprise breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

21. 21. The method of claim 20, wherein the one or more predicted conditions comprise a polygenic risk score determined for a particular condition of the subject, and wherein the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

22. Sequencing a sample from the subject to generate raw sequenced sample data; and formatting the raw sequenced sample data to generate the sequenced sample data for the subject; 20. The method of claim 18, further comprising:

23. Receive reference data; identifying one or more values ​​shared between the reference data and the sample data for the subject; and removing one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data to obtain the subject sequencing variant sample data for the subject.

23. The method of claim 22, further comprising:

24. 23. The method of claim 22, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

25. An apparatus comprising a processor and a memory storing software instructions which, when executed by said processor, cause said apparatus to perform the steps of any of claims 1 to 24.

26. A computer program product comprising at least one non-transitory computer readable storage medium having stored thereon software instructions which, when executed by an apparatus, cause said apparatus to perform the steps of any of claims 1 to 24.

27. obtaining subject sequencing variant sample data corresponding to the sequenced sample data for a subject; generating a predicted value of a genome map for the subject based on the subject sequencing variant sample data using a neural network trained to predict values ​​of a genome map generated from sample data and one or more reference genome maps; and determining one or more predicted states for the subject based on the predictive value. A method comprising:

28. Sequencing a sample from the subject to generate raw sequenced sample data; and formatting the raw sequenced sample data to generate the sequenced sample data for the subject; 28. The method of claim 27, further comprising:

29. Receive reference data; identifying one or more values ​​shared between the reference data and the sample data for the subject; and removing one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data to obtain the subject sequencing variant sample data for the subject.

30. The method of claim 28, further comprising:

30. 30. The method of claim 28, wherein the sample is sequenced using one or more of whole exome sequencing or whole genome sequencing.

31. 28. The method of claim 27, wherein the one or more predicted conditions comprise breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

32. 32. The method of claim 31 , wherein the one or more predicted conditions comprise a polygenic risk score determined for a particular condition of the subject, and wherein the one or more predicted risk scores for the particular condition are used to determine the polygenic risk score for the particular condition.

33. Obtaining sequencing variant sample data, wherein the sequencing variant sample data is based on the sequencing sample data and one or more reference genome maps; and training the neural network using the sequencing variant sample data, wherein training the neural network comprises: modifying values ​​of one or more loci for one or more samples in the sequencing variant sample data to create modified sequencing variant sample data; and using the altered sequencing variant sample data as input during training of the neural network, such that the neural network is trained to predict values ​​from unaltered sequencing variant sample data and the one or more reference genome maps. Training, including 28. The method of claim 27, further comprising:

34. Training the neural network includes training one or more layers of the neural network by performing one or more training iterations, each training iteration comprising: modifying one or more actual values ​​of the sequencing variant sample data from the one or more loci and the one or more samples corresponding to the sequencing variant sample data; generating one or more predictions for the one or more loci and the one or more samples using the neural network; and adjusting the neural network based on the one or more predicted values ​​and the one or more actual values.

34. The method of claim 33, comprising training comprising:

35. obtaining phenotypic data associated with said sequencing variant sample data; and training one or more layers of said neural network to predict said phenotypic data.

34. The method of claim 33, further comprising:

36. 36. The method of claim 35, further comprising: (i) arranging the one or more layers into a hierarchy of attention encoder mechanisms; (ii) subdividing levels in the hierarchy into one or more groups; and (iii) for each group, using one or more parent tokens from another attention encoder mechanism associated with the group as input tokens for an attention encoder mechanism at a subsequent level in the hierarchy.

37. 37. The method of claim 36, further comprising appending an input type token to the one or more groups.

38. 38. The method of claim 37, wherein the input type token corresponds to an input type, the input type corresponding to one or more of a cell type, a tissue type, an exome sequence, a whole genome sequence, or a species.

39. 37. The method of claim 36, wherein the sequencing variant sample data comprises variants obtained from whole exome sequencing.

40. 37. The method of claim 36, wherein the sequencing variant sample data comprises variants obtained from whole genome sequencing.

41. 37. The method of claim 36, wherein the phenotypic data comprises disease onset and control labels for one or more conditions.

42. 42. The method of claim 41, wherein the one or more conditions comprise breast cancer, cardiovascular disease, type 1 diabetes, type 2 diabetes, schizophrenia, or a combination thereof.

43. 34. The method of claim 33, wherein modifying the value further comprises masking a portion of the value of the sequencing variant sample data.

44. 34. The method of claim 33, wherein modifying the value further comprises masking a first portion of the value of the sequencing variant sample data and replacing a second portion of the value of the sequencing variant sample data.

45. 34. The method of claim 33, wherein the sequencing variant sample data is one-hot encoded.

46. Obtaining the sequencing variant sample data comprises: receiving the reference data and the formatted sequencing sample data; identifying one or more values ​​shared between the reference data and the formatted sequencing sample data; and Removing one or more of the one or more values ​​shared between the reference data and the formatted sequencing sample data.

34. The method of claim 33, further comprising:

47. receiving the reference data, Identifying one or more reference datasets; and generating combined reference data by combining the one or more reference datasets with one or more epigenetic maps, wherein the reference data comprises the combined reference data.

47. The method of claim 46, further comprising:

48. 48. The method of claim 47, wherein the one or more epigenetic maps comprise one or more deoxyribonucleic acid methylation maps.

49. receiving the formatted sequencing sample data; Accessing the raw sequencing sample data; and 47. The method of claim 46, further comprising formatting the raw sequencing sample data to create formatted raw sequencing sample data, wherein the formatted raw sequencing sample data is used to obtain the formatted sequencing sample data and create the sequencing variant sample data.

50. An apparatus comprising a processor and a memory storing software instructions which, when executed by said processor, cause said apparatus to perform the steps of any of claims 27 to 49.

51. A computer program product comprising at least one non-transitory computer readable storage medium having stored thereon software instructions which, when executed by an apparatus, cause said apparatus to perform the steps of any of claims 27 to 49.

Images