Generating genome sequence datasets

Abstract

In one example, a method includes receiving a trait indicator; obtaining a probability distribution of embedding vectors in a latent space based on the trait indicator, where the probability distribution is generated from input vectors representing variant segments associated with the trait indicator by a distribution generation sub-model of a trained generative machine learning model, the input vector being defined in a variant segment space having a greater number of dimensions than the latent space; obtaining a sample vector by sampling the probability distribution; and reconstructing an output vector from the sample vector by a sequence generation sub-model of the trained generative machine learning model and based on the trait indicator, where the output vector is defined in the variant segment space; and generating a simulated genome sequence based on the output vector.

Description

[Technical Field]

[0001] Cross-reference of related applications This application claims priority to U.S. Provisional Application No. 63 / 078,148, entitled “Genomic Sequence Dataset Generation,” filed on 14 September 2020, and is a PCT application whose entire contents are incorporated herein by reference for all purposes.

[0002] Statement regarding the rights to inventions made under federal government-sponsored research and development. This invention was made with government support under grant number HG009080, granted by the National Institutes of Health. The government has certain rights to this invention. [Background technology]

[0003] Most locations in the human deoxyribonucleic acid (DNA) sequence do not vary between individuals, but about 2 percent (5 million positions) do. These are called single nucleotide polymorphisms (SNPs). All human populations share a common ancient origin in Africa and a common set of variable regions, but modern human populations exhibit clear differences in the frequency of SNP variants at each site of the DNA sequence in the genomes of these populations. Because DNA is inherited as an intact sequence with only rare and random swaps in the ancestor (between two parent DNA sequences) in each generation, ancestral SNPs form contiguous segments. As a result, correlations between adjacent sites along the genome, which are typically inherited as a whole, differ among subpopulations worldwide.

[0004] Various pieces of information can be inferred from correlations between adjacent sites along the genome. For example, local ancestry inference uses patterns of variation observed at various sites along an individual's genome to estimate the ancestral origin of that individual's DNA. In addition, correlations along the genome can influence polygene risk scores (PRS), genome-wide association studies (GWAS), and many other aspects of precision medicine. Given that correlations between neighboring gene variants are ancestral-dependent, applying the results of these analyses to an individual's genome may require knowledge about the individual's ancestry at each site along the genome.

[0005] Unfortunately, many subpopulations of the world are not included in modern genetic research, and over 80% of these studies include only individuals of European ancestry. This severely limits our ability to make accurate predictions for the rest of the world's populations. Analyzing the ancestry of mixed individuals using local ancestry inference fills this gap and contributes to understanding the genetic architecture and relevance of non-European ancestry, thus enabling the benefits of medical genetics to extend to the majority of the world's populations.

[0006] Various methods exist for local ancestry inference, including Hidden Markov Model (HMM)-based analysis of polymorphism in mixed ancestors (HAPAA), HAPMIX, and SABE; Local Ancestry in Mixed Populations (LAMP) using probability maximization with a sliding window; RFMix using a random forest within the window; and Local Ancestry Inference Networks (LAI-Net) using neural networks. However, these algorithms require accessible training data from each ancestor to recognize the ancestral segment of each chromosome. The main challenge is that many datasets, including the Human Genome Reference, are protected by privacy restrictions and are either privately owned or otherwise not accessible to the public. The lack of training datasets can reduce the ability of these algorithms to perform accurate local ancestry inference.

[0007] Therefore, techniques for generating genome sequence datasets with a more diverse set of genetic variants for various ancestral origins are desirable. [Overview of the Initiative] [Means for solving the problem]

[0008] Embodiments of this disclosure provide methods, systems, and apparatus for generating simulated genome sequences having segments of gene variants (e.g., SNPs) of a given trait (e.g., ancestral origin) using a generative machine learning model. The generative machine learning model can receive data representing input variant (e.g., SNP) segments in haploid or diploid DNA sequences, and information indicating the trait of the segments. The DNA sequences can be obtained, for example, from a genome sequencing operation that provides the genome sequence of interest, or from a DNA microarray containing DNA segments. The data representing the input variant segments may include input vectors, where each dimension of the input vector represents a heterozygous site in the genome and is associated with a value indicating a variant. From the variant input segments and based on the trait, the generative machine learning model can randomly generate a set of output vectors representing simulated variant segments based on a multidimensional probability distribution. The output vectors may have different patterns of variants at different locations in the genome compared to the input variant segments. The simulated variant segments may be variants of the input variant segments and are statistically related to the input variant segments for a particular trait based on a multidimensional probability distribution.

[0009] In some embodiments, certain operations of a generative machine learning model can be performed in a reduced dimensional space (e.g., a latent space) that is reduced in dimension, i.e., by the number of variants in the segment. For example, an initial mapping can transform N variants into an embedding vector with M dimensions, where M (e.g., 40) is less than N (e.g., 500). For an input variant segment (e.g., having 500 SNPs or other variants), the generative machine learning model can determine a representation of a multidimensional probability distribution (e.g., one probability distribution for each dimension of the reduced space), and then take a sample of the embedding vector from the multidimensional probability distribution from one input variant segment. The sample is then reconstructed as a simulated variant segment. In one embodiment, the probability distribution can be modeled as a Gaussian distribution with a multidimensional mean and multidimensional variance. In some embodiments, the probability distribution can have different mean and variance values ​​for each dimension of the reduced space. In some embodiments, a zero-mean and unit-variance Gaussian distribution (e.g., an isotropic Gaussian distribution) can be achieved through training operations based on the Kullback-Leibler (KL) divergence. The determination of a particular probability distribution (or a multidimensional distribution) can be done based on a mapping of parameters learned in the training operations. Thus, the variant values ​​of the input variant segment can be mapped to a set of distributions (or a multidimensional distribution). The generative machine learning model can then take samples from the multidimensional Gaussian distribution, in which case the samples are reconstructed to generate an output vector.

[0010] In some embodiments, generative machine learning models include an encoder and decoder configured as a class-conditional variational autoencoder (CVAE). Both the encoder and decoder can be implemented as neural network models. The encoder can transform input vectors in the mutant segment space into a multidimensional probability distribution of embedding vectors in a latent space having a reduced number of dimensions, for example, by mapping them to the mean and width (variance) of the distribution for each of the reduced number of dimensions. For an isotropic distribution, the variance is the same for each dimension. The distribution in the reduced space can represent the variation of the input mutant segments. The decoder can take samples of the embedding vectors from the probability distribution, and then reconstruct the embedding vectors to form an output vector from the samples, the output vector having the same dimensions as the input vector and representing the simulated mutant segments.

[0011] Both the encoder and decoder of CVAE can be trained to fit different patterns of mutants to a target multidimensional probability distribution while reducing information loss in mapping from the mutant segment space to the latent space. This ensures that the simulated mutant segments generated by the decoder are statistically related to the input mutant segments according to the multidimensional probability distribution and can simulate the effects of random variation in the mutant segments. Training of the encoder and decoder can be based on minimizing a loss function that combines the reconstruction error (between the input and output vectors, respectively) and a penalty for divergence from the target probability distribution (e.g., based on the difference in parameters (e.g., mean and variance) between the multidimensional probability distribution and the target values ​​of the target probability distribution). Training operations can be performed to reduce or minimize the reconstruction error and the penalty for distribution divergence so that the distribution of mutant segments generated by the encoder can (to some extent) match a target probability distribution that may be a zero-mean unit variance Gaussian distribution. The center (mean) and variance of the distribution of mutant segments can be set based on the reduction / minimization of the reconstruction error and the penalty for distribution divergence.

[0012] To further reduce distribution errors so that simulated mutant segments more closely follow the target probability distribution, the CVAE can be trained using a class-conditional generative adversarial network (CGAN) including the decoder and discriminator in the training operation described above. The discriminator can also be implemented as a neural network model, which can classify whether the mutant segments output by the decoder are actual mutant segments or simulated mutant segments. The discriminator may not be able to distinguish actual mutant segments from simulated mutant segments when the simulated mutant segments follow the target probability distribution, at which point the discriminator's classification error rate may reach its maximum, meaning that the decoder reconstruction is optimal. An adversarial training operation can be performed in which the decoder parameters are adjusted to increase the classification error rate so that the probability distribution in the reduced dimension approaches the target probability distribution, while the discriminator parameters are adjusted to decrease the classification error rate. The training operation may be stopped when approximately 50% of the output vectors represent actual mutant segments and approximately 50% of the output vectors represent fake / simulated mutant segments.

[0013] Using the disclosed embodiments, a generative machine learning model can be used to generate a large number of random but statistically simulated mutant segments. For example, through training operations, encoder parameters can be obtained that map input mutant sequences to an embedding space for different ancestors, and decoder parameters can be obtained that map embedding vectors to reconstructed sequences for different ancestors. The generative machine learning model can receive a target ancestor as input. A specific probability distribution (e.g., a Gaussian distribution) for that target ancestor can then be selected, and multiple samples of embedding vectors can be obtained from that specific probability distribution. The embedding vectors and target ancestors can then be input into the decoder to generate simulated mutant segments. In another embodiment, an input mutant segment and the traits of this input mutant segment can also be input into the encoder to generate parameters for a probability distribution, from which embedding vectors can be sampled, and then the sampled embedding vectors and traits can be input into the decoder to generate simulated mutant segments.

[0014] Simulated mutant segments can be used for a variety of applications. In one example, simulated mutant segments can be used to train a local ancestral inference machine learning model. Since simulated mutant segments can contain a diverse set of statistically related mutant patterns for different traits, a local ancestral inference machine learning model trained on simulated mutant segments can learn from those mutant patterns and more accurately predict the traits of the mutant segments.

[0015] In another embodiment, simulated mutant segments can also be provided as additional data in genome-wide association studies (GWAS). For example, various statistical techniques can be used to detect fundamental relationships between genome sequences, traits, and specific target medical / biological traits. To improve the coverage of the training computation, additional mutant segments for the target medical / biological traits and simulated individuals that have (or do not have) those traits can be generated using a generative machine learning model, and these additional mutant segments can be provided to train the model. The additional mutant segments can be used to provide, for example, control data representing mutant segments of simulated individuals that do not have the target medical / biological traits, and control data representing mutant segments of simulated individuals that have the target medical / biological traits but have different traits, and so on.

[0016] In addition, generative machine learning models can provide a portable and publicly accessible mechanism for generating additional variant segment data (for training, GWAS, etc.). Specifically, datasets containing actual human genome references are private and protected by privacy restrictions. In contrast, the function / model parameters of generative machine learning models do not carry data that can identify any individual and can be made publicly available. As a result, generative machine learning models can be made publicly available to generate simulated variant segments to improve the training of local ancestral inference machine learning models and provide control data for GWAS, etc.

[0017] Some examples relate to systems and computer-readable media associated with the methods described herein.

[0018] A better understanding of the properties and advantages of the embodiments of this disclosure can be obtained by referring to the following detailed description and accompanying drawings. [Brief explanation of the drawing]

[0019] [Figure 1-1] Figures 1A and 1B illustrate examples of single nucleotide polymorphisms (SNPs) and the ancestral origin of SNPs in the genome. [Figure 1-2] Same as above.

[0020] [Figure 2-1] Figures 2A, 2B, and 2C illustrate an exemplary analysis of SNP sequences facilitated by embodiments of the present disclosure. [Figure 2-2] Same as above. [Figure 2-3] Same as above.

[0021] [Figure 3-1] Figures 3A, 3B, 3C, 3D, and 3E illustrate exemplary components of a generative machine learning model for generating simulated SNP sequences according to embodiments of the present disclosure. [Figure 3-2] Same as above. [Figure 3-3] Same as above. [Figure 3-4] Same as above. [Figure 3-5] Same as above.

[0022] [Figure 4] Figure 4 illustrates exemplary training operations of the generative machine learning model of Figures 3A-3E according to embodiments of the present disclosure.

[0023] [Figure 5-1] Figures 5A and 5B illustrate other exemplary training operations of the generative machine learning model of Figures 3A-3E according to embodiments of the present disclosure. [Figure 5-2] Same as above.

[0024] [Figure 6] Figure 6 illustrates another generative machine learning model according to some embodiments.

[0025] [Figure 7]Figure 7 shows a sample architecture of a machine learning model that provides relationships between various mutant segments according to an embodiment of the present disclosure.

[0026] [Figure 8] Figure 8 illustrates exemplary methods for generating simulated SNP sequences using several examples.

[0027] [Figure 9] Figure 9 illustrates a computer system in which an embodiment of the present disclosure may be implemented. [Modes for carrying out the invention]

[0028] Various pieces of information can be inferred from correlations between adjacent sites along the genome. For example, local ancestry inference uses patterns of variation observed at various sites along an individual's genome to estimate the ancestral origin of that individual's DNA. In addition, correlations along the genome influence polygenic risk scores (PRS), genome-wide association studies (GWAS), and many other aspects of precision medicine.

[0029] Each segment of the genome can be assigned a trait (e.g., ancestral origin, biomedical trait, demographic trait, or other phenotype). While examples are provided for ancestral origin, the techniques described herein are applicable to other traits as well. Synthetic sequences corresponding to a given trait can be generated based on input sequences, and these synthetic sequences can be obtained by sequencing the cellular DNA or cell-free DNA (e.g., from plasma) of a subject possessing the trait.

[0030] The aforementioned local ancestry inference operations, as well as genome-related medical research such as PRS and GWAS calculations, can be facilitated by large genome sequence datasets containing diverse sets of gene variants for various ancestral origins. For example, local ancestry inference machine learning models can be trained using diverse sets of statistically related SNP patterns for various ancestral origins, enabling the machine learning model to learn from those SNP patterns and more accurately predict the ancestral origin of SNP segments. Furthermore, SNP patterns of subjects with known traits can also be used as data for GWAS studies, for example, to provide data for statistical analysis to detect fundamental relationships between genome sequences, ancestral origins, and specific biological / medical traits and ancestral origins. However, the availability of datasets containing actual human genome references is typically limited because such data are privately owned and protected by privacy restrictions.

[0031] Embodiments of this disclosure provide methods, systems, and apparatus for generating simulated genome sequences having segments of a given ancestral origin gene variant (e.g., SNP) using a generative machine learning model. The generative machine learning model can receive data representing input SNP segments in haploid or diploid DNA sequences, and information indicating the ancestral origin of the segments. The DNA sequences can be obtained, for example, from a genome sequencing operation that provides the genome sequence of interest, or from a DNA microarray containing segments of DNA. The data representing the input SNP segments may include input vectors, where each dimension of the input vector represents a location in the genome and is associated with a value indicating an SNP variant. From the SNP input segments and based on their ancestral origin, the generative machine learning model can generate one or more output vectors representing simulated SNP segments. The output vectors may have different patterns of SNP variants in locations within the genome compared to the input SNP segments. The simulated SNP segments may be variants of input SNP segments that are statistically related to a particular ancestral origin input SNP segment.

[0032] According to several embodiments, a generative machine learning model can generate a representation of a multidimensional probability distribution (e.g., mean and variance) based on a transformation of variants of an input SNP segment into a reduced space (embedding / latent space), and then obtain a sample of embedding vectors from the probability distribution. The simulated SNP segment is then reconstructed as a simulated SNP segment (e.g., by a decoder) from the embedding vector sample. In one embodiment, the multidimensional probability distribution may be a Gaussian distribution with a calculated mean and variance determined from the mapping of the input SNP segment, and the parameters of the mapping function can be determined based on training comparing the accuracy of the reconstructions. The generative machine learning model can then obtain a sample from the Gaussian distribution to generate embedding vectors, which are then reconstructed to form an output vector.

[0033] In some embodiments, a generative machine learning model includes a first submodel and a second submodel, both of which can be implemented as neural network models. The first submodel may include an encoder configured to map a multidimensional probability distribution of embedding vectors in a latent space to an input vector. The latent space may have a number of dimensions reduced to the number of SNP sites represented in the input SNP segment. While reducing the number of dimensions, the mapping can still retain information indicating patterns of SNP variants in the input vector within the embedding vector. If the probability distribution includes a Gaussian distribution, the encoder can determine the mean and variance for the distribution of each dimension of the embedding vector based on the patterns of SNP variants in the input vector. For various SNP sequences, various probability distributions can be determined (e.g., various Gaussian distributions with different means and variances for various dimensions of the latent space). In some embodiments, an ancestral origin can be input to the encoder along with the input vector to generate parameters for the distribution of the embedding vector for that ancestral origin. For various ancestral origins, the encoder can generate multiple probability distributions.

[0034] In addition, a second submodel may include a decoder. The decoder can obtain samples of embedding vectors from a probability distribution. The probability distribution can be output from an encoder based on encoding the input SNP segment and ancestral origin, or from a probability distribution pre-generated from an encoder based on another input SNP segment and selected based on the ancestral origin and SNP site. The decoder can then reconstruct from the samples of embedding vectors an output vector having the same dimensions as the input vector representing the input SNP segment. As part of the sampling operation, a random function can be implemented based on parameters for generating random samples of embedding vectors. The random function may be part of or outside the decoder. As part of the reconstruction operation, the decoder can implement a reconstruction function for mapping samples of embedding vectors in latent space to output vectors in SNP segment space, based on the ancestral origin of the input SNP segment. The output vector can then represent a simulated SNP segment of ancestral origin.

[0035] Both encoders and decoders can be trained to maximize the representation of various patterns of SNP variants in the latent space. In some embodiments, the encoder and decoder may be part of a class-conditional variational autoencoder (CVAE), where various ancestral origins are represented as various classes. A CVAE can be trained using a training input vector representing actual SNP sequences for a given ancestral origin in the training operation. The training operation may include a forward and a backpropagation operation. As part of the forward operation, the encoder can determine the probability distribution of the embedding vectors of the input vector using a mapping function with an initial set of function parameters. The probability distribution can be represented, for example, by the mean and variance over each dimension of the latent space. Based on the probability distribution, the decoder can compute samples of the embedding vectors and compute an output vector using a reconstruction function (with an initial set of function parameters).

[0036] Backpropagation of the training operation can minimize a first loss function by adjusting the initial function parameters of the mapping and reconstruction functions. The first loss function can include reconstruction error components and distribution error components. The reconstruction error can be generated based on the difference between the input vector and the output vector, while the distribution error can be generated based on the difference between the probability distribution of the embedding vector and the target probability distribution. In some embodiments, the distribution error can be calculated based on the Kullback-Leibler divergence (KL divergence). Through a gradient descent scheme, the function parameters of the encoder and decoder can be adjusted based on the change in the first loss function with respect to the function parameters, with the aim of minimizing the first loss function. This training can be repeated for training input vectors for various ancestral origins to determine various function parameters of the mapping and reconstruction functions for various ancestral origins representing various classes.

[0037] Based on a combination of reconstruction error and distribution error, the training of the encoder and decoder allows the encoder to map input SNP segments to a probability distribution with target properties (e.g., isotropic) based on reducing the distribution error, while also allowing the center of the probability distribution to be determined based on the embedding vector of the input SNP segments, based on reducing the reconstruction error. In such a configuration, the simulated SNP segments (e.g., generated by a CVAE from input SNP segments given ancestral origins, or generated by a decoder based on an input probability distribution selected based on ancestral origins) may include a diverse set of SNP pattern variants, but the SNP pattern variants remain statistically related based on the target probability distribution.

[0038] To further reduce distribution errors so that simulated SNP segments more closely follow the target probability distribution, the CVAE can be trained using a class-conditional generative adversarial network (CGAN) that includes a decoder and a discriminator. This CGAN can be trained in the aforementioned training operation, a separate training operation from the encoder, or a separate loop of training (e.g., multiple training iterations for the VAE, then multiple training iterations for the CGAN, and back to the VAE). The discriminator can be a third submodel of the generative machine learning model and can also be implemented as a neural network model. During the training operation, as part of the feedforward operation, the decoder can compute a random sample of the embedding vector and reconstruct an output vector representing the simulated SNP segment. Furthermore, the discriminator can determine whether the output vector represents an actual SNP segment. The discriminator may not be able to distinguish an actual SNP segment from a simulated SNP segment when the simulated SNP segment follows the target probability distribution, at which point the classification error rate approaches 50%.

[0039] The goal of the training operation in CGAN is for the output vector to fit a target probability distribution (e.g., an isotropic Gaussian distribution). To achieve this goal, adversarial training operations are performed, adjusting the decoder parameters to increase the classification error (based on making the simulated SNP segments more similar to actual SNP segments) while adjusting the discriminator parameters to decrease the classification error. The decoder's reconstruction function parameters can be adjusted according to a second loss function that decreases as the classification error in the discriminator increases. Furthermore, the discriminator's model parameters can also be adjusted in the same training operation according to a third loss function that decreases as the classification error decreases. When the classification error rate approaches 50%, the adversarial training operation can be stopped.

[0040] Other variants besides single nucleotide polymorphisms (SNPs) can be used. The variants can be any genetic data at a site that may correspond to a gene location or range of locations. Examples of various types of variants include bases, deletions, amplifications (e.g., short tandem repeats), insertions, inversions, and methylation statuses. A site may contain two or more values, for example, specific alleles and specific methylation statuses of an SNP. These can be considered different variant values ​​occurring at the same variant site, or these sites can be considered different because they relate to different types of variants. In either case, the vector of variant values ​​will have the same total length. Therefore, a variant segment can contain any set (e.g., a continuous sequence) of variant sites, and a variant site may have different variant values ​​for one or more types of variants.

[0041] I. Examples of SNP sequences Single nucleotide polymorphisms (SNPs) can refer to DNA sequence variations that occur when a single nucleotide, adenine (A), thymine (T), cytosine (C), or guanine (G), differs among members of a species in the genome.

[0042] Figure 1A illustrates one example of a SNP. Figure 1A illustrates two sequenced DNA fragments 102 and 104 from different individuals. Sequenced DNA fragment 102 contains the sequence AT-AT-CG-CG-CG-TA-AT, while sequenced DNA fragment 104 contains the sequence AT-AT-CG-CG-TA-TA-AT. As shown in Figure 1A, DNA fragments 102 and 104 contain a difference of one nucleotide base pair (CG vs. TA, typically referred to as C and T). The difference can be counted as a single SNP. SNPs can be encoded to a value based on whether they are common or rare variants. Common variants may be more common in a population (e.g., 80%), while rare variants will occur in fewer individuals. In some examples, common variants can be encoded to a value of -1, while rare variants can be encoded to a value of +1.

[0043] Modern human populations originating from different continents and subcontinents exhibit discernible differences in the frequency of SNP variants at different sites of DNA sequences in the genomes of these human populations, and in the correlations between these variants at different neighboring sites, due to genetic drift and different demographic histories (bottlenecks, expansions, and mixing) over the past 50,000 years. Because DNA is inherited in each generation as an intact sequence with only rare and random swaps in ancestry (between two parental DNA sequences), ancestral SNPs form consecutive segments, enabling strong ancestry inference based on patterns of consecutive SNP variants.

[0044] Figure 1B illustrates an exemplary group of ancestral origins between SNP segments of a mixed chromosome pair in an individual, each originating from a different parent of the individual. Group 112 illustrates the true ancestral origin of genetic material at different SNP sites in the individual, as can be determined by analyzing the individual's genome. The individual's genome can be determined by sequencing DNA from the individual's tissues. In the example in Figure 1B, the ancestral origins of the SNP segments may include Africa, East Asia, and Europe.

[0045] Group 112 can be the first stage in classifying the ancestral origin of the SNP segment. As a second stage, smoothing can be performed. Group 114 illustrates the decoded ancestral origin of the SNP, which can be derived by performing a smoothing operation over the entirety of Group 112 to remove ancestral origin discontinuities of the segment, such as discontinuity 116 (Africa) of segment 118 (East Asia) and discontinuity 120 (East Asia) of segment 122 (Africa).

[0046] The ability to accurately infer ancestry along the genome with high resolution is crucial for understanding the roles of genetic phenomena and environment in complex traits such as predisposition to specific diseases and specific biomedical traits (e.g., blood pressure, cholesterol levels). This may stem from populations sharing a common ancestor that gives rise to complex physical and medical traits. For example, certain ethnic groups may have relatively high asthma mortality rates, while others may have relatively low asthma mortality rates. Elucidating the genetic associations within populations for predisposition to specific diseases and biomedical traits can inform the development of treatments and enable the construction of disease risk predictors known as polygenic risk scores. However, because correlations between neighboring gene variants (e.g., SNPs) are ancestral-dependent, applying these risk scores to an individual's genome requires knowledge of the individual's ancestry at each site along the genome.

[0047] This trait may be for any phenotype. For other types of traits, the genome of the subject can still be mixed. For example, segments containing cancer-associated variants (e.g., sequence variants, copy number variants, or structural variants) can be labeled with cancer-associated trait indicators, while other segments can be labeled with non-cancerous trait indicators. For yet other traits, the genome of the subject may not be mixed. For example, a subject with an autoimmune disorder may have all segments labeled with the disorder's trait indicator. Traits can be assigned to subjects in a variety of ways, for example, based on physician observation, pathological examination, genomic testing, or other types of tests.

[0048] A subject may possess multiple traits, such as ancestral origin, demographic traits (e.g., height), and biomedical traits (e.g., presence of disease such as diabetes). Subjects can be clustered based on the traits they possess. Subjects can be labeled with various traits in any number of ways. For example, one-hot encoding can be used to specify whether each trait is present in a segment. Several traits can be grouped together (e.g., presence or absence of disease, or different age ranges), and only one trait indicator from a group may be positive (e.g., 1).

[0049] The embodiments allow for the simulation of genome sequences associated with one or more of these traits without the need to use the genome of a specific person, thereby preserving privacy. For example, a hospital may have genome sequences for subjects who have type 2 diabetes, are members of a Native American tribe, and / or possess other traits, and people wish to keep the DNA of these subjects private. The embodiments can create synthetic genomes that have the same characteristics as these people, but are not their personal genomes. These synthetic genomes can then be used to train other models that predict traits in other subjects.

[0050] II. Exemplary Analysis of SNP Sequences Machine learning models can be used to perform ancestor-specific analyses of target genomic data. Various machine learning models exist for local ancestor inference, such as Hidden Markov Model (HMM)-based analysis of polymorphism in mixed ancestors (HAPAA), HAPMIX, and SABE, Local Ancestry in Mixed Populations (LAMP) which uses probability maximization with a sliding window, and RFMix which uses a random forest within the window.

[0051] Figure 2A illustrates a general topology of a machine learning model 200 for performing local ancestry inference in several embodiments. As shown in Figure 2A, the machine learning model 200 can receive data 202 representing the input genome sequence of a subject (e.g., a human). The input genome sequence may contain multiple segments, each containing multiple single nucleotide polymorphism (SNP) sites in the subject genome. Each segment may be represented in data 202 by a sequence of SNP values ​​at SNP sites, where each SNP value identifies a variant at the SNP site.

[0052] Data 202 may include SNP segments 204a, 204b, 204c, 204n, etc. For each segment, the machine learning model 200 can generate an ancestral origin prediction for each SNP segment (e.g., whether the SNP segment originates from Africa, Europe, or East Asia) based on the SNP values ​​within the segment and the patterns of the associated SNP sites. In Figure 2A, the machine learning model 200 can generate an ancestral origin prediction 206a for SNP segment 204a, an ancestral origin prediction 206b for SNP segment 204b, an ancestral origin prediction 206c for SNP segment 204c, and an ancestral origin prediction 206n for SNP segment 204n. The ancestral origin predictions can be concatenated to provide, for example, groups 112 and / or 114 in Figure 1B. Each segment may contain the same or different amounts of variants (e.g., SNPs). Exemplary numbers of variants within a segment include 50, 100, 150, 200, 250, 300, 400, 500, 1000, 5000, and 10000 sites.

[0053] The machine learning model 200 can be trained using genome data from individuals with known ancestral origins to learn various ancestral-specific patterns of SNPs, and by applying this learning, it can more accurately identify ancestral-specific patterns of SNPs from the input genome data.

[0054] Figure 2B illustrates an exemplary training operation. As shown in Figure 2B, the machine learning model 200 can receive training data 212 containing SNP segments 214a, 214b, 214c, and 214n, as well as known ancestral origins 216a, 216b, 216c, and 216n for each segment. The machine learning model 200 can apply an initial set of model parameters to generate ancestral origin predictions 218a for SNP segment 214a, 218b for SNP segment 214b, 218c for SNP segment 214c, and 218n for SNP segment 214n. The training module 230 can compare the ancestral origin predictions and known ancestral origins for each SNP segment and adjust the model parameters based on the comparison results. The adjustment may be based on maximizing the percentage of matching ancestral origin predictions among SNP segments in the training data 204.

[0055] Local ancestry inference can be useful in genome-wide association studies (GWAS). A GWAS is a genome-wide study of genetic variants in different individuals to determine whether any given variant is associated with a particular disease predisposition or a specific biomedical trait (e.g., blood pressure, cholesterol levels, etc.). Therefore, such studies can link specific genetic variations to specific diseases. Knowing the specific ancestral predisposition of a particular disease can help identify whether a particular variation is associated with that disease.

[0056] Figure 2C illustrates one example of GWAS240. In Figure 2C, population 242 has trait X, while population 244, which may be a control group, does not have trait X. The genome sequences of both populations are then analyzed to determine SNPs (if present) at each site. In Figure 2C, counting SNPs typically involves the appearance of a CG base pair at a DNA site with a TA base pair (or only C pair and T if only one strand is used, e.g., the Watson strand). In population 242, which has biological / medical trait X, 50% of individuals have a CG base pair at the first DNA site (labeled "SNP1"). In contrast, in population 244, which does not have trait X, only 5% of individuals have a CG base pair at the first DNA site. On the other hand, only 1% of both populations 242 and 244 have a CG base pair at the second DNA site (labeled "SNP2"). This study suggests that individuals possessing the CG base pair as SNP1 are excessively present within the population of 242, which may indicate a strong correlation between the appearance of the CG base pair as SNP1 and trait X. Furthermore, given that correlations between neighboring genetic variants are typically ancestral-dependent, it is desirable that the SNP patterns included in the study are associated with various ancestral origins, potentially revealing a relationship between traits and ancestral origins.

[0057] A large set of SNP sequence data containing diverse sets of SNP patterns for various ancestral origins may be useful for training the machine learning model 200 in Figures 2A and 2B and for providing the basis for the GWAS240 in Figure 2C. Specifically, to improve the performance of the machine learning model 200, the training data can include diverse sets of SNP patterns for each ancestral origin. Since the model parameters are tuned based on maximizing the percentage of matching ancestral origin predictions between input SNP segments, training the model parameters with a diverse set of SNP patterns can enable the machine learning model 200 to detect / distinguish a wider range of SNP patterns and improve the accuracy of ancestral origin predictions.

[0058] Furthermore, in GWAS240, both population 242 (possessing trait X) and population 244 (not possessing trait X) should include individuals with a wide variety of SNP patterns. This is to ensure that both populations are representative of the general population, so that conclusions from GWAS (e.g., the strong association between the appearance of the CG base pair as SNP1 and trait X) are applicable not only to populations 242 and 244 but also to the general population. Moreover, by including individuals with a wide variety of SNP patterns in both populations 242 and 244, it can be ensured that various less frequent SNP patterns are included in and considered in the analysis. This can further support the conclusion that the appearance of the CG base pair as SNP1, rather than other SNP variants, is predominantly associated with trait X. This can improve the specificity of GWAS240 for individuals of various ancestral origins. For example, by further subdividing individuals within populations 242 and 244 according to their ancestral origins, GWAS240 may demonstrate, for instance, that a strong association between the CG base pair at SNP1 and trait X is applicable only to a specific group of individuals with a particular ancestral origin, but not to other groups of individuals with diverse ancestral origins. In some embodiments, statistical analysis can be performed based on the SNP segments of individuals, the ancestral origin of individuals, and the biological / medical traits of these individuals to detect relationships between genome sequences, ancestral origins, and specific biological / medical traits.

[0059] While it is desirable to train local ancestry inference models and provide a foundation for GWAS using large SNP sequence datasets with diverse sets of SNP patterns for different ancestral origins, the availability of such datasets is typically limited. Specifically, SNP sequence datasets are typically obtained from actual DNA sequences collected from humans and containing human genome references. Such datasets are typically protected by privacy restrictions and are privately owned or otherwise not accessible to the public. The availability of SNP sequence datasets for certain populations, such as underserved or highly susceptible populations, can be particularly limited due to various reasons, including the lack of registration of these populations in GWAS. As a result, there may be a lack of SNP segment data for training machine learning models 200 in Figure 2A, and machine learning models for GWAS to improve the accuracy of those models.

[0060] III. Genome sequencing using machine learning To provide a larger and more diverse set of SNP patterns for various ancestral origins, simulated genome sequences are provided. Such simulated SNP patterns can be generated in a specific manner to create realistic SNP patterns, thereby making it possible to use these SNP patterns as training sets that will provide accurate local ancestry inference machine learning models.

[0061] For this purpose, a generative machine learning model can be used to generate a simulated genome sequence having segments of a given ancestral origin of genetic variants (e.g., SNPs). The generative machine learning model can receive data representing input SNP segments in haploid or diploid DNA sequences, and information indicating the ancestral origin of the segments. From the SNP input segments and based on their ancestral origin, the generative machine learning model can randomly generate a set of simulated SNP segments that may contain different patterns of SNP variants, based on a probability distribution. The simulated SNP segments may be variations of the input SNP segments and are statistically related to the input SNP segments for a particular ancestral origin, based on a probability distribution. The simulated SNP segments can be used, for example, to train a local ancestry inference machine learning model and provide control data in genome-wide association studies (GWAS).

[0062] Using generative machine learning models, a set of simulated SNP segments with random SNP patterns can be generated. Due to their random nature, simulated SNP segments can contain a diverse set of SNP patterns, but the SNP patterns are statistically related to those of actual SNP patterns from real DNA sequences, so that the simulated SNP segments can provide realistic variants of SNP patterns. Such simulated SNP segments can be used to improve local ancestry inference models (e.g., machine learning model 200) and to provide control data for GWAS (e.g., GWAS240). Specifically, using simulated SNP segments, machine learning model 200 can learn from a broader but more realistic range of SNP patterns to make ancestral origin predictions, thereby improving the likelihood that machine learning model 200 can generate accurate predictions of actual SNP patterns from real DNA sequences. Furthermore, simulated SNP segments can also improve GWAS. For example, simulated SNP patterns can be associated with specific traits.

[0063] A. General topology Figure 3A illustrates a general topology of a generative machine learning model 300 for generating a simulated genome sequence having segments of a given ancestral origin genetic variant (e.g., SNP). As shown in Figure 3A, the generative machine learning model 300 can receive an input genome sequence of a subject (e.g., a human) and data 302 representing a known group of ancestral origins for genomic variation within the sequence. The input genome sequence is divided into multiple non-overlapping segments, each containing multiple single nucleotide polymorphism (SNP) sites in the subject genome, including input SNP segments 303a, 303b, 303c, 303n, etc. Each segment may be represented in data 302 by a sequence of SNP values ​​at the SNP site, where each SNP value specifies a variant at the SNP site (e.g., A, C, T, or G). In addition, each segment is also associated with an ancestral origin indicator that shows the ancestral origin of the segment. For example, input SNP segment 303a is associated with ancestral origin indicator 304a, input SNP segment 303b is associated with ancestral origin indicator 304b, input SNP segment 303c is associated with ancestral origin indicator 304c, while input SNP segment 303n is associated with ancestral origin indicator 304n.

[0064] For each input SNP segment (e.g., input SNP segment 303b), and based on the ancestral origin indicator of each input SNP segment, the generative machine learning model 300 can generate multiple simulated SNP segments, including simulated SNP segments 305a, 305b, and 305m. Each simulated SNP segment can represent a variation of input SNP segment 303b and is statically associated with input SNP segment 303b. The simulated SNP segments for each input SNP segment can be concatenated to form multiple simulated genome sequences that can correspond to various hypothetical individuals.

[0065] Data 302 can be obtained from haploid or diploid DNA sequences. Data 302 can be obtained, for example, from a genome sequencing operation providing the target genome sequence, or from a DNA microarray containing DNA segments. Haplotype information can be encoded to include, for example, a first value (e.g., -1) indicating that a particular SNP is a large number of variants at the SNP site, a second value (e.g., +1) indicating that the SNP is a small number of variants at the SNP site, or a third value (e.g., 0) indicating missing genomic information at the SNP site. An input SNP segment, such as input SNP segment 303b, may include a multidimensional vector where each dimension corresponds to an SNP site and has one value among -1, +1, or 0. In addition, the ancestral origin indicator 304 can take various forms. In one embodiment, the ancestral origin indicator may include a set of codes indicating ancestral origin locales from a set of candidate ancestral origins (e.g., Africa, Europe, East Asia, etc.). In another embodiment, the ancestral origin indicator may include the geographical coordinates (e.g., longitude and latitude) of the ancestral origin locale. The SNP segments in data 302 may have the same number of SNP values ​​(e.g., 500) or different numbers of SNP values.

[0066] In some embodiments, the generative machine learning model 300 may include two submodels, including a distribution-generating submodel 306 and a sequence-generating submodel 308. The distribution-generating submodel 306 can accept input vectors representing an input SNP segment (e.g., input SNP segment 303b) and an associated ancestral origin indicator (e.g., ancestral origin indicator 304b) of the input SNP segment. Based on the input vectors and the ancestral origin indicator, the distribution-generating submodel 306 can determine a multidimensional probability distribution 310 in a reduced-dimensional space (latent space). The probability distribution 310 may correspond to the variation of the input SNP segment. Based on the probability distribution 310, the sequence-generating submodel 308 can generate a number of simulated SNP segments, including simulated SNP segments 305a, 305b, 305m, etc., each representing a random sample of SNPs statistically related to the input SNP segment according to the probability distribution 310.

[0067] Each simulated SNP segment can be viewed as a simulation of random variation in an input SNP segment, in contrast to an input SNP segment extracted from an actual DNA sample, for example, as the target input genome sequence. As will be described in detail below, the distribution generation submodel 306 can learn to introduce random variation into the SNP sequence during training and determine submodel parameters that reflect the relationship between SNP patterns and the probability distributions of variants of the SNP patterns. After training, the distribution generation submodel 306 can apply the submodel parameters to the SNP patterns in the input SNP sequence to determine the parameters of the probability distribution 310 of the SNP patterns, while the sequence generation submodel 308 can compute a random sample of variants of the SNP patterns based on the parameters of the probability distribution 310 as simulated SNP sequences.

[0068] In some embodiments, the distribution generation submodel 306 may also receive an ancestral origin indicator 304 and SNP site information as input without an input SNP segment 303, and output a probability distribution 310 based on the ancestral origin indicator 304. In such embodiments, the distribution generation submodel 306 stores multiple sets of probability distributions 310, each associated with an ancestral origin indicator and various SNP sites, and can retrieve a probability distribution 310 corresponding to an input ancestral origin indicator and input SNP site. Multiple sets of probability distributions 310 can be pre-generated by the distribution generation submodel 306 from other input SNP segments.

[0069] B. Exemplary components of a generative machine learning model In some embodiments, the distribution-generating submodel 306 of the generative machine learning model 300 can be configured as an encoder, while the array-generating submodel 308 of the generative machine learning model 300 can be configured as a decoder. The encoder and decoder can be combined to operate as a class-conditional variational autoencoder (CVAE).

[0070] Figure 3B illustrates exemplary operations of the distribution generation submodel 306 and the sequence generation submodel 308. Specifically, the distribution generation submodel 306 can implement a mapping function 324 that maps an input vector 320 representing an SNP segment to a multidimensional probability distribution 310 (represented as one-dimensional distributions 310a to 310c) of the embedding vector in the latent space. The mapping can represent a transformation from an input vector in an SNP segment space having a certain number of dimensions (defined based on the number of SNP sites represented in the input SNP segment) to an embedding vector in a latent space having a reduced number of dimensions.

[0071] In some embodiments (not shown in Figure 3B), the distribution generation submodel 306 may include multiple mapping functions, each associated with a class representing an ancestral origin. The distribution generation submodel 306 may select a mapping function 324 to transform an input vector into a probability distribution 310 of its ancestral origin based on an ancestral origin indicator associated with an SNP segment. In some embodiments, the distribution generation submodel 306 may also implement a mapping function 324 that receives the ancestral origin as part of an input vector 320 and generates a probability distribution 310 based on both the ancestral origin and the SNP segment represented in the input vector 320.

[0072] In addition, the sequence generation submodel 308 can implement a reconstruction function 325 for reconstructing the output vector 326 in the SNP segment space from the sample embedding vector 332. The sequence generation submodel 308 can obtain the sample embedding vector 332 from the probability distribution 310 output by the distribution generation submodel 306 based on the input vector 320, or from another set of probability distributions pre-generated by the distribution generation submodel 306 from other input SNP segments. Sampling can be performed by the sequence generation submodel 308 or by a sampling function separate from the sequence generation submodel 308. The output vector can represent simulated SNP segments 305a, 305b, 305m, etc., in Figure 3A, to model the effect of random variation in the SNP pattern in the input SNP segment.

[0073] In the embodiment shown in Figure 3B, the input vector 320 is a set of 500 SNP values ​​(si0, si1, ...si) corresponding to 500 dimensions in the SNP segment space. 499 While it can include ) the output vector 326 contains 500 SNP values ​​(so0, so1, ...so) corresponding to 500 dimensions in the SNP segment space. 499) can include. On the other hand, the latent space can have a reduced number of dimensions (e.g., three dimensions as shown in Figure 3B). For example, the sample embedding vector 332 can include three values ​​(is0, is1, and is2), each corresponding to a dimension in the latent space. Similar to the distribution generation submodel 306 which includes multiple mapping functions, the sequence generation submodel 308 may also include multiple reconstruction functions. The sequence generation submodel 308 may select a reconstruction function 325 for reconstructing an output vector from the sample vector 332 based on an ancestral origin indicator associated with an SNP segment. In some embodiments, the sequence generation submodel 308 may also implement a single reconstruction function 325 that takes the ancestral origin and sample vector 332 as input and generates an output vector 326 based on the ancestral origin and sample vector 332.

[0074] The transformation and reconstruction operations between the encoder and decoder, involving dimensionality reduction and restoration, can create a bottleneck in the input vector 320 that retains only the most relevant information representing the SNP pattern in the embedding vector, and this information can be recovered in the reconstruction of the output vector 326. On the other hand, noise information that is not necessary to represent the SNP pattern can be discarded during the transformation operation, and this noise information is not present in the reconstructed output vector.

[0075] Referring again to the distribution generation submodel 306, the probability distribution 310 is multidimensional and includes probability distributions for each dimension of the latent space, including probability distributions 310a, 310b, and 310c. In some embodiments, the probability distribution 310 can approach a multidimensional isotropic Gaussian distribution having the same variance for each dimension, and each dimension can be viewed as an independent one-dimensional Gaussian distribution centered on a mean that may differ between the dimensions of the latent space. An isotropic Gaussian distribution can include a covariance matrix such as: Σ=σ 2 I (Equation 1).

[0076] In Equation 1, Σ is the covariance matrix of the isotropic Gaussian distribution, and σ2 is the common variance between dimensions, while I is the identity matrix. If the probability distribution 310 does not exactly match an isotropic Gaussian distribution, then each of the probability distributions 310a, 310b, and 310c may have different variances and different means.

[0077] As described below, the parameters of the mapping function 324 can be adjusted so that the probability distribution 310 fits the target distribution. Such a configuration restricts the transformation from the SNP segment space to the latent space to fit the target probability distribution, ensuring that the latent space is continuous and that it provides a distribution of various SNP patterns, for example, having a center based on the input SNP segment. Both properties allow the decoder to obtain a random sample of the embedding vector that provides realistic SNP sequences, while allowing some variation defined according to the probability distribution 310. This makes it possible for the random sample to model the effect of random variation in SNP sequences in a real DNA sample. The distribution generation submodel 306 may include multiple distribution generation functions, each associated with a class representing ancestral origin. The distribution generation submodel 306 can select distribution generation functions to generate probability distributions based on ancestral origin indicators associated with SNP segments.

[0078] C. Neural Network Implementation of Generative Machine Learning Models Figures 3C, 3D, and 3E illustrate additional details of the distribution generation submodel 306 and the sequence generation submodel 308. Figure 3C illustrates one embodiment of a random sampling operation of the embedding vector between the distribution generation submodel 306 and the sequence generation submodel 308. As shown in Figure 3C, the distribution generation function 330 can generate representations 340 containing representations 340a, 340b, and 340c of the probability distribution 310. Representations 340a, 340b, and 340c can each contain the mean and variance of the probability distributions 310a, 310b, and 310c for each dimension of the latent space. For example, representation 340a may contain the mean μ0 and variance σ0 of the probability distribution 310a, representation 340b may contain the mean μ1 and variance σ1 of the probability distribution 310b, and representation 340c may contain the mean μ2 and variance σ2 of the probability distribution 310c.

[0079] In addition, the array generation submodel 308 can implement a random function 342 and a sampling function 344 for performing sampling of the probability distribution 310 to generate a sample embedding vector 332. In some embodiments, the random function 342 and the sampling function 344 may be outside the array generation submodel 308. The random function 342 can generate a random matrix R based on an isotropic Gaussian distribution with zero mean and unit variance. The sampling function 344 can generate a sample embedding vector 332, which is a sample of the embedding vector from the probability distribution 310, based on reparameterization of CVAE, by multiplying the output random matrix R from the random function 342 by a vector of variances from representation 340, and adding the result of the multiplication to a vector of means, also from representation 340. For example, a sample of the sample vector 332 can be generated using the sampling function 344 as follows:

number

[0080] In Equation 2, the value of the first dimension of the sample vector 332 is is0, which can be calculated by adding the product of the mean μ0, the variance σ0, and the random number r0 of the random matrix R. Furthermore, the value of the second dimension of the sample vector 332 is is1, which can be calculated by adding the product of the mean μ1, the variance σ1, and the random number r1 of the random matrix R. Furthermore, the value of the third dimension of the sample vector 332 is2, which can be calculated by adding the product of the mean μ2, the variance σ2, and the random number r2 of the random matrix R. The array generation submodel 308 can generate multiple random matrices R, combine these random matrices R with the mean and variance representations of the representation 340 to generate multiple random samples of the embedding vector, and then reconstruct the output vector based on the sample vectors.

[0081] The mapping function 324 and distribution generation function 330 of the distribution generation submodel 306, as well as the reconstruction function 325 of the sequence generation submodel 308, can be implemented using a neural network model.

[0082] Figure 3D illustrates an exemplary neural network model 350 of a distribution generation submodel 306 for implementing the mapping function 324 and the distribution generation function 330. The neural network 350 includes an input layer 352, a hidden layer 354, and an output layer 356. The input layer 352 includes multiple nodes, such as nodes 352a, 352b, and 352n, which may be a subset of the nodes in the input layer. The hidden layer 354 includes multiple nodes, such as nodes 354a, 354b, and 354m. The output layer 356 includes multiple nodes, such as nodes 356a, 356b, and 356c. Each node in the output layer 356 may correspond to a three-dimensional space in the latent space shown in Figure 3B.

[0083] The input layer 352 and the hidden layer 354 can implement a mapping function 324 for converting input vectors in the SNP segment space to embedding vectors in the latent space. Some of the nodes in the input layer 352 can receive encoded values ​​(e.g., 1, 1, -1) of SNP values ​​at specific SNP sites in the segment, received by the classifier. For example, input node 352a receives the encoded value si0, and input node 352b receives the encoded value si1, both of which are from input vector 320. In addition, some of the nodes in the input layer 352, such as node 352n, receive associated ancestral origin indicators (labeled c in Figure 3D) associated with input vector 320.

[0084] Each node in the input layer 352 is associated with a first set of encoder weights. For example, node 352a is associated with the set of encoder weights [WE1a], and node 352n is associated with the set of encoder weights [WE1n]. Each node can scale an input value (SNP value, ancestral origin indicator, etc.) with its associated set of weights to produce a set of scaled values ​​(scaled SNP values), and send these scaled values ​​to a node in the hidden layer 354. Larger encoder weights in the input layer 352 can indicate that a particular dimension in the SNP segment space contains important information about the SNP sequence, and therefore that this particular dimension is well represented in the latent space.

[0085] Each node in the hidden layer 354, which may contain one or more layers, receives scaled values ​​from each node in the input layer 352, and sums up the scaled values ​​to generate an intermediate value (also called an intermediate sum). Using this intermediate sum, the probability distribution 310 of the embedding vectors can be computed in the output layer 356. For example, node 354a can compute the intermediate sum, sum 354a, as follows:

number

[0086] In Equation 3, WE1j can represent the weight values ​​of each set of weights (e.g., [WE1a], [WE1n], etc.) used by each node in the input layer 352 to scale the input value inj, which can be either an SNP value (e.g., si0, si1, etc.) or an ancestral origin indicator c. The combination of ancestral origin indicators and SNP values ​​when calculating the intermediate sum may be equivalent to selecting different mapping functions for different ancestral origins.

[0087] Each node in the 354 hidden layers also implements a nonlinear activation function that defines the output of that node, to which an intermediate sum is given. The activation function can mimic the decision-making processes of biological neural networks. One example of an activation function is the normalized linear unit (ReLU) function, defined according to the following equation:

number

[0088] In addition to ReLU, other forms of activation functions can also be used, such as the softmax function, the softplus function (which can be a smooth approximation of the ReLU function), the hyperbolic tangent function (tanh), the arctangent function (arctan), the sigmoid function, and the Gaussian function. The activation function is part of the mapping function 324 and can provide a nonlinear transformation from the SNP segment space to the latent space, thereby improving the filtering of noise information.

[0089] In addition to summation and activation function processing, each node in the hidden layer 354 may also perform a batch normalization process to normalize the output of the hidden layer, thereby increasing, for example, the speed, performance, and stability of the neural network model 350. The normalization process may include, for example, subtracting the mean of the outputs from each output of the hidden layer node and dividing the result by the standard deviation of the outputs to generate a normalized output for each hidden layer node. In some embodiments, the normalization operation may be performed before applying the activation function. Based on the activation function processing and batch normalization processing, node 354a generates intermediate output ie0, node 354b generates intermediate output ie1, and node 354m generates intermediate output iem.

[0090] Each node in the hidden layer 354 is associated with a second set of encoder weights. For example, node 354a is associated with the set of encoder weights [WE2a], and node 354m is associated with the set of encoder weights [WE2m]. Each node is associated with the output values ​​of the activation function / batch normalization operation (e.g., ie0 for node 354a, ie1 for node 354b, ie1 for node 354m). m These can be scaled by the weighted set to generate a set of scaled values, and these scaled values ​​can be sent to the 356 nodes in the output layer.

[0091] Each node in the output layer 356 may correspond to a dimension in the latent space. Each node in the output layer 356 receives scaled values ​​from the hidden layer 354 and can calculate the mean and variance of the probability distribution 310 as part of the representation 340 of the corresponding dimension in the latent space. For example, node 356a can calculate representation 340a, node 356b can calculate representation 340b, and node 356c can calculate representation 340c. Each node can calculate the mean and variance by, for example, summing the scaled output values ​​received from each node in the hidden layer 354 based on equation 3 above.

[0092] In some embodiments, the ancestral origin indicator c is not provided as input to the input layer 352. Instead, the distribution generation submodel 306 may include multiple sets of encoder weights [WE1] and [WE2], each associated with an ancestral origin. The ancestral origin indicator c can be used to select a set of encoder weights for the neural network model 350.

[0093] Figure 3E illustrates an example of a neural network model 360 of the array generation submodel 308 for implementing the reconstruction function 325. Neural network model 360 may have an inverted architecture similar to neural network model 350 in Figure 3D. Neural network 360 includes an input layer 362, a hidden layer 364, and an output layer 366. The input layer 362 includes a plurality of nodes, including nodes 362a, 362b, 362c, and 362d, which may be a subset of the nodes in the input layer. Each of nodes 362a, 362b, and 362d corresponds to a dimension in the latent space and can receive elements of a sample vector (sample vector values) for the corresponding dimension (e.g., is0, is1, and is2 of sample vector 332), while node 362d receives an ancestral origin indicator c. Hidden layer 364 can contain the same number of nodes as hidden layer 354 and one or more layers of the neural network model 350 (of the distribution generation submodel 306), whereas output layer 366 contains multiple nodes such as nodes 364a, 364b, and 364n. Each node in output layer 366 corresponds to a dimension in the SNP segment space.

[0094] Each node in the input layer 362 is associated with a first set of decoder weights. For example, node 362a is associated with the set of decoder weights [WD1a], and node 362n is associated with the set of decoder weights [WD1n]. Each node can scale input values ​​(elements of embedding vectors, ancestral origin indicators, etc.) with its associated set of weights to generate a set of scaled values, and send these scaled values ​​to the output node of the output layer 364. The first set of decoder weights can be configured by the hidden layer 354 to invert the second stage of the mapping function 324. The combination of ancestral origin indicators and the embedding vector values ​​when calculating the intermediate sum (also called the intermediate value) may be equivalent to selecting different reconstruction functions for different ancestral origins.

[0095] Each node in the hidden layer 364 receives scaled values ​​from each node in the input layer 362 and sums the scaled values ​​based on Equation 3 to generate an intermediate sum. The intermediate sum can then be processed using a nonlinear activation function (e.g., ReLU) and batch normalization operations, such as in the hidden layer 354 in Figure 3D, to generate intermediate outputs. For example, node 364a generates intermediate output id0, node 364b generates intermediate output id1, node 364m generates intermediate output id m This generates the following. Each node in the hidden layer 364 is also associated with a second set of decoder weights. For example, node 354a is associated with the set of encoder weights [WD2a], and node 354m is associated with the set of encoder weights [WD2m]. Each node is associated with the output value of the activation function / batch normalization operation (e.g., id0 for node 364a, id1 for node 364b, id for node 364m). mThe data (etc.) can be scaled with a set of weighted values ​​to generate a set of scaled values ​​(also referred to as scaled sample vector values), and these scaled values ​​can be sent to the nodes of the output layer 366. A second set of decoder weights can be configured by the hidden layer 354 to invert the first stage of the mapping function 324.

[0096] Next, each node in the output layer 366 generates an output vector value corresponding to one dimension of the SNP segment space by summing the scaled values ​​from each node in the hidden layer 364. For example, node 366a can generate so0 of output vector 326, while node 366b can generate so1 of output vector 326.

[0097] In some embodiments, the ancestral origin indicator c is not provided as input to the input layer 362. Instead, the sequence generation submodel 308 may contain multiple sets of decoder weights [WD1] and [WD2], each associated with an ancestral origin. The ancestral origin indicator c can be used to select a set of decoder weights for the neural network model 360.

[0098] D. Training of a Class Conditional Variational Autoencoder Distribution generation submodel 306 and sequence generation submodel 308, configured as CVAEs, can be trained to maximize the representation of various patterns of SNP variants in latent space.

[0099] Figure 4 illustrates an exemplary training operation in which the training module 400 can train an encoder and decoder using a training input vector representing an actual SNP sequence for a given ancestral origin.

[0100] The training operation may include forward and backpropagation operations. As part of the forward operation, the distribution generation submodel 306 can receive the training input vector 420 and apply an initial set of parameters (e.g., encoder weights [WE1] and [WE2]) of the mapping function 324 to the training input vector 420 to generate an initial set of parameters (e.g., mean and variance) of the probability distribution 310 of the embedding vectors. The array generation submodel 308 can use the sampling function 344 to compute a set of sample embedding vectors 332 based on the probability distribution and apply an initial set of parameters (e.g., decoder weights WD1 and WD2) of the reconstruction function 325 to the sample embedding vectors to generate a set of training output vectors 426.

[0101] The backpropagation of the training operation can be adjusted to minimize the first loss function by adjusting the initial function parameters of the mapping function 324 and the distribution generation function 330. The first loss function may include a reconstruction error component calculated by the reconstruction error module 402 and a distribution error component calculated by the distribution error module 404, both of which are part of the training module 400. The reconstruction error can be generated by the reconstruction error module 402 based on the difference between each of the training input vectors 420 and the training output vectors 426, such as the mean squared error. The distribution error can be generated by the distribution error module 404 based on the difference between the probability distribution of the embedding vectors and the target probability distribution (represented by representation 340). In some embodiments, the distribution error can be calculated based on the Kullback-Leibler divergence (KL divergence). One embodiment of the first loss function may be as follows:

number

[0102] In Equation 5,

number

[0103] In addition, parameter adjustment module 406 can also adjust initial function parameters of reconstruction function 325 based on minimizing a second loss function. The second loss function includes the reconstruction error output by reconstruction error module 402 and represented by the expression [Num] As described below, the second loss function can include an adversarial loss component when array generation submodel 308 is also trained using a generative adversarial network (GAN).

[0104] Through a gradient descent scheme, the parameter tuning module 406 can tune the function parameters of the mapping function 324, the reconstruction function 325, and the distribution generation function 330 (e.g., [WE1], [WE2], [WD1], [WD2], etc.) based on the changes in the first and second loss functions with respect to the function parameters, with the aim of minimizing the first and second loss functions. For example, the parameter tuning module 406 can tune the function parameters to achieve reduction (and thus gradient descent) of the first and second loss functions. Training can be repeated to train input vectors for various ancestral origins, thereby determining various function parameters for various ancestral origins representing various classes.

[0105] Training the mapping function 324 and reconstruction function 325 that implement the encoder, based on a combination of reconstruction error and distribution error, enables the encoder to map the input SNP segment to a target stochastic distribution of SNP segment variants, based on reducing the distribution error. Furthermore, the probability distribution of the SNP pattern can be centered based on the embedding vector representing the input SNP segment, based on reducing the reconstruction error in the training operation. Using such a configuration, the simulated SNP segment generated by the generative machine learning model 300 from an input SNP segment with ancestral origins can include a diverse set of SNP pattern variants defined based on the target probability distribution. However, the SNP pattern variants remain closely related to the SNP pattern of the input SNP segment because the center of the target probability distribution is determined based on the input SNP segment.

[0106] E. Training using Class Conditional Adversarial Generative Networks To further reduce distribution errors so that the simulated SNP segments more closely follow the target probability distribution, a sequence generation submodel 308 (configured, for example, as a decoder for CVAEs) can be trained using a class-conditional generative adversarial network (CGAN) that includes a decoder and a discriminator. The discriminator attempts to determine the difference between the actual SNP segments and the simulated SNP segments.

[0107] In CGAN, decoders and classifiers can be trained using the same training operation, but for opposite purposes. Specifically, the classifier receives a vector representing an SNP segment as input and classifies the input as either a simulated SNP segment generated by the sequence generation submodel 308 (e.g., training output vector 426 in Figure 4) or an actual SNP segment from an actual DNA sequence (e.g., training input vector 420). The classifier can be trained to minimize the classification error rate (e.g., classifying an actual SNP segment as a simulated SNP segment, or vice versa).

[0108] If a simulated SNP segment is statistically related to an actual SNP segment according to a target probability distribution (e.g., isotropic Gaussian), and the simulated SNP segment has a very similar SNP pattern to the actual SNP segment (i.e., has a low reconstruction error), the discriminator will likely be unable to distinguish the simulated SNP segment from the actual SNP segment, resulting in an increased classification error rate. On the other hand, a decoder can be trained to generate simulated SNP segments that minimize the reconstruction error and fit the target probability distribution, thereby effectively maximizing the classification error rate of the discriminator. The fit of the simulated SNP segment to the target probability distribution can be further improved through iterative adversarial training operations, in which the decoder is forced to reconstruct the classification error by adjusting the decoding weights to include more simulated SNPs, and the discriminator reduces its classification error.

[0109] A CGAN can be trained as a VAE, including an encoder and decoder, in a separate or combined process. An example of a combined process is described below, but various training procedures can be used. For example, various input vectors can be used to train the VAE rather than the GAN. Furthermore, the distribution learned from training the VAE can be randomly sampled only when training the GAN.

[0110] Figure 5A illustrates additional components for performing adversarial training operations. As shown in Figure 5A, a discriminator 502, which may be part of or outside of the generative machine learning model 300, can form a CGAN with a sequence generation submodel 308, and the CGAN, in combination with a distribution generation submodel 306, forms a CVAE-CGAN model. During the training operation, as part of the feedforward operation, the distribution generation submodel 306 can receive the training input vector 420 and generate a probability distribution representation 340 in latent space, while the reconstruction function 325 can compute a sample of the output vector 426 based on the probability distribution representation 340 and the reconstruction function 325. The discriminator 502 can then perform a classification operation to classify a set of vectors, including the training input vector 420 and the training output vector 426, which represent actual SNP segments (e.g., SNP segments extracted from actual DNA sequences), whether each vector represents a simulated SNP segment or an actual SNP segment, and generate a classification output 504. SNP segments can be extracted from the actual DNA sequence, which is the target input genome sequence.

[0111] In some embodiments, the discriminator 502 can be implemented as a neural network. Figure 5B illustrates an embodiment of a neural network model 520 that may be part of the discriminator 502. The neural network 520 includes an input layer 522, a hidden layer 524, and an output layer 526. The input layer 522 includes multiple nodes, including nodes 522a, 522b, 522n, etc. The input layer 522 includes nodes for receiving input vectors representing SNP segments in SNP segment space (e.g., node 522a receives s0, node 522b receives s1, etc.), and nodes for receiving ancestral origin indicators (e.g., node 522n). The hidden layer 524 can provide a nonlinear mapping between input vectors and intermediate outputs and may include the same number of nodes as the hidden layer 354 in Figure 3D and the hidden layer 364 in Figure 3E. The output layer 526 includes a single node for calculating the probability of input vectors representing actual SNP segments based on the intermediate outputs from the hidden layer 524. The probabilities can be included in the classification output 504 to indicate that the input vector represents an actual SNP segment when the probability exceeds a threshold, and that the input vector represents a simulated segment when the probability falls below the threshold.

[0112] Each node in the input layer 522 is associated with a first set of discriminator weights. For example, node 522a is associated with the set of discriminator weights [WX1a], and node 362n is associated with the set of discriminator weights [WX1n]. Each node can scale input values ​​(input vector values, ancestral origin indicators, etc.) with its associated set of weights to generate a set of scaled values, which it can then send to the nodes in the hidden layer 364. The weights can, for example, represent the contribution of each SNP region in an SNP segment to the classification decision of whether the SNP segment is real or simulated. The combination of the ancestral origin indicator c and the input vector allows the discriminator 502 to perform classification operations based on various criteria for various ancestral origins.

[0113] Each node in the hidden layer 524 receives scaled values ​​from each node in the input layer 522 and sums the scaled values ​​based on Equation 3 to generate an intermediate sum. The intermediate sum can then be processed using a nonlinear activation function (e.g., ReLU) and batch normalization operations, as in hidden layer 354 in Figure 3D and hidden layer 364 in Figure 3E, to generate intermediate outputs. For example, node 524a generates intermediate output ix0, node 524b generates intermediate output ix1, and node 524m generates intermediate output id m The hidden layer 524 is also associated with a second set of discriminator weights [WX2], where each node is associated with a weight in the weight set. The weights associated with the nodes in the hidden layer 524 can indicate the node's contribution to the probabilistic output. Each node is associated with the output value of the activation function / batch normalization operation (e.g., ix0 for node 524a, ix1 for node 524b, ix for node 524m). m The values ​​(etc.) can be scaled by their associated weights and sent to a single node in the output layer 526, which can then generate a probability output (p) by summing the scaled values.

[0114] In some embodiments, the ancestral origin indicator c is not provided as an input to the input layer 522. Instead, the discriminator 502 may include multiple sets of discriminator weights [WX1] and [WX2], each associated with an ancestral origin. The ancestral origin indicator c can be used to select a set of discriminator weights for the neural network model 520.

[0115] Referring again to Figure 5A, the training module 400 may include a classification error module 506. During backpropagation, the classification error module 506 can determine whether the classification output 504 contains errors. The classification error module 506 can determine that the classification output 504 contains errors, for example, if the probability shown in the classification output 504 exceeds a threshold (indicating that the vector is an actual SNP segment) but the vector is generated by the array generation submodel 308, or if the probability is below a threshold (indicating that the vector is a simulated SNP segment) when the vector is a training input vector and contains actual SNP segments. The model parameters of the discriminator 502 can be adjusted to minimize the classification error in the classification output 504, while the function parameters of the reconstruction function 325 (e.g., decoder weights [WD1], [WD2], etc.) can be adjusted to maximize the classification error in the classification output 504.

[0116] Specifically, the parameter adjustment module 406 adjusts the reconstruction error component.

number

number

[0117] In Equation 6,

number

number

[0118] In addition, the parameter tuning module 406 can also tune the initial model parameters of the discriminator 502 based on minimizing a third loss function, which may take the form of a binary cross-entropy loss function, as follows:

number

[0119] In Equation 7,

number

number

[0120] The training operation in Figure 5A can be performed in multiple phases to minimize the first loss function in Equation 5 (for the distribution generation submodel 306), the second loss function in Equation 6 (for the reconstruction function 325), and the third loss function in Equation 7 (for the discriminator 502). Specifically, in the first phase, a full forward propagation operation can be performed on the training input vector 420 using an initial set of function / model parameters (e.g., encoder weights [WE1] and [WE2], decoder weights [WD1] and [WD2], and discriminator weights [WX1] and [WX2]). The training output vector 426, and the classification output 504 of the training output vector 426 and the training input vector 420 can be generated. Then, a full backpropagation can be performed, where the reconstruction error, distribution error, and classification error are determined by the training module 400 and backpropagated to adjust the parameters of the discriminator 502, the reconstruction function 325, the distribution generation function 330, and the mapping function 324. Based on minimizing the first loss function (reconstruction error and KL divergence) of the distribution generation submodel 306, a first set of tuned function / model parameters can be determined.

[0121] Next, a second phase of the training operation can be initiated, which includes an adversarial training operation between the reconstruction function 325 and the discriminator 502. During the adversarial training operation, the decoder weights [WD1] and [WD2], as well as the discriminator weights [WX1] and [WX2], can be adjusted (from a first set of tuned parameters) to minimize both a second loss function for the reconstruction function 325 and a third loss function for the discriminator 502, which leads to conflicting targets for classification error. The adversarial training operation can be performed in multiple iterations, each including a reduced forward propagation operation to compute a new training output vector (e.g., corresponding to output vector 326), and a reduced backpropagation operation to adjust only the parameters of the reconstruction function 325 and discriminator 502, and a classification output 504 for a new sample using the tuned parameters. The adversarial training operation can be stopped, for example, when approximately 50% of the classification outputs 504 are correct, which leads to an error rate of approximately 50%. This suggests that because the training output vector 426 is very close to the training input vector 420, the classifier 502 is unable to distinguish between the vectors, and the classification operation approaches a random coin flip operation, which leads to a 50% error rate.

[0122] The training input vector 420 is shown to be used to train the classifier 502, but other real SNP segments can be used for this purpose. Furthermore, the given output vector 426 can be used with multiple real SNP segments to determine the classification error. Alternatively, multiple output vectors can be generated using random sampling, and these output vectors can be used to determine the classification error for a set of real SNP segments.

[0123] When a 50% error rate is achieved in the discriminator 502, the second phase of the training operation can be stopped, and a second set of adjusted parameters for the reconstruction function 325 can be obtained. Then, the first phase of the training operation can be restarted, and the adjustments to the decoder weights of the reconstruction function 325 can be propagated back to the distribution generation submodel 306 to reduce the reconstruction error and distribution error. The training operation can be repeated for various training input vectors associated with various ancestral origins, for example, to obtain the relationship between the ancestral origin indicator and the probability distribution output, reconstruction output, and classification output, and to obtain various function / model parameters for various ancestral origins.

[0124] In Figures 4 and 5A-5B, the generative machine learning model 300 can be trained using a training input vector 420 containing haploid sequences to generate haploid sequences. To generate simulated diploid chromosomes, the generative machine learning model 300 can be separately trained using each of the pair of haploid sequences of the training diploid sequence to generate variants of each sequence in the pair of haploid sequences. The variant haploid sequences can then be paired to generate simulated diploid chromosomes.

[0125] In addition, in some embodiments, the simulated SNP sequences generated by the generative machine learning model 300 can be post-processed to further improve the diversity of SNP patterns in the sequences. For example, the generative machine learning model 300 can be run to generate simulated SNP sequences to represent several different individuals, generating N times the number of individuals. The correlation between pairs of the generated SNP sequences can be determined, and 1 / N of the set of simulated SNP sequences with the lowest mean correlation can be selected as the output. In some embodiments, N can be set to 2. [Examples]

[0126] IV. Experimental Results A. Experimental generative machine learning models An experimental generative machine learning model 600, illustrated in Figure 6, is developed and trained. The generative machine learning model 600 may include an encoder 602, a decoder 604, and a discriminator 606. The encoder 602, decoder 604, and discriminator 606 may correspond to the distribution generative submodel 306, sequence generative submodel 308, and discriminator 502 in Figures 5A and 5B, respectively. The generative machine learning model 600 is trained based on the training operations in Figures 5A-5B (trained as CVAE-CGAN). In Figure 6, "z" represents a sample vector 332 obtained from sampling a probability distribution 310 with a sampling function 344, as shown in Figure 3C. The generative machine learning model 600 is trained using two different datasets for two different experiments. In each experiment, the generative machine learning model 600 generates a set of simulated SNP sequences. Local ancestry inference models such as RFMix are trained using both simulated SNP sequences and actual SNP sequences (SNP sequences extracted from actual DNA sequences), and the performance of the local ancestry inference model is evaluated to examine the quality of the simulated SNP sequences compared to the actual SNP sequences.

[0127] B. Simulation datasets originating from Africa In the first experiment, a simulated dataset based on an African origin simulation is generated and used to train the generative machine learning model 600 and the RFMix local ancestry inference model. The African origin simulation models the origin and spread of humans as a single ancestral population that rapidly expanded across the African continent. This population has remained constant in size to the present day. At some point in the past, a small group of individuals migrated from Africa and later split in two directions, one to form the basis of the current European population and the other the basis of the current East Asian population. Both populations grew exponentially after the split.

[0128] Following the African origin model described above, three groups of 100 simulated diploid sequences were generated, each representing an individual with a single ancestor. One group was then generated for each of the African, European, and East Asian ancestors, resulting in 300 simulated individuals. These 300 simulated diploid sequences were divided into training, validation, and test sets, each containing 240, 30, and 30 diploid sequences, respectively. The validation and test diploid sequences were then used to generate mixed offspring using a Wright-Fisher sequential simulation over a series of generations. From the 30 diploid sequences of the single-ancestor individuals, a total of 100 diploid sequences representing 100 mixed individuals were generated using mixed events that occurred over the past eight generations to create both the validation and test sets.

[0129] RFMix was trained using 240 diploid sequences representing 240 single-ancestor individuals. The same diploid sequences were used to train the generative machine learning model 600 as a CVAE-CGAN model (provided as input sequence x and actual sequence xreal). Furthermore, the trained RFMix was evaluated using 100 diploid sequences representing 100 mixed individuals, generated using a Wright-Fisher sequential simulation. In this experiment, diploid sequences of chromosome 20 are simulated.

[0130] From the experiment, 80 simulated samples for each ancestor were generated using the generative machine learning model 600 and used to train RFMix. RFMix was then evaluated with 100 diploid sequences from mixed individuals. RFMix was also trained with 240 diploid sequences representing 240 single-ancestor individuals representing the African origin dataset, and then evaluated again with the same 100 diploid sequences from mixed individuals. The inference accuracy of local ancestry inference by RFMix trained on the two different datasets was then compared. Table 1 below illustrates the experimental results: [Table 1]

[0131] As shown in Table 1 above, RFMix achieves comparable accuracy when trained with African-source data and datasets generated by the generative machine learning model 600. The accuracy results also indicate that adding discriminators and adversarial losses helps the network learn to simulate human chromosome sequences that are more similar to African-source datasets, and is therefore more useful in training local ancestry inference models such as RFMix, thereby providing a significant increase in accuracy.

[0132] C. Global Datasets In the second experiment, RFMix and the generative machine learning model 600 are trained using SNP sequences from a total of 258 single-population individuals from East Asian (EAS), African (AFR), and European (EUR) ancestors. Specifically, 83 Han Chinese (CHB) SNP sequences from Beijing, China, 88 Yoruba (YRI) SNP sequences from Ibadan, Nigeria, and 87 Iberian (IBS) SNP sequences from Spain are used in the second experiment. In addition, 10 single individuals per ancestor are used to generate mixed offspring for testing and validation using Wright-Fisher sequential simulations over a series of generations. From the SNP sequences of 30 single-ancestor individuals, SNP sequences of a total of 100 mixed individuals were generated with mixed events that occurred over the past 12 generations to create both validation and test sets. RFMix and class-conditional VAE-GAN (CVAE-CGAN) are trained using SNP sequences from 258 single-ancestor individuals, while the trained RFMix is ​​evaluated using SNP sequences from 200 mixed individuals in validation and test sets. In this experiment, chromosome 20 is used from each individual.

[0133] A CVAE-CGAN is trained for each ancestor using SNP sequences from 258 single-ancestor individuals. After training, a total of 100 simulated SNP sequences are generated for each ancestor and used to train RFMix. RFMix is ​​then evaluated against SNP sequences from 100 mixed individuals in a validation set. Hyperparameters for the CVAE-CGAN, including W (number of SNPs per segment), H (size of the hidden layer), and J (number of dimensions in the latent space), as well as training parameters such as learning speed, batch size, and epochs, are selected to provide the best validation accuracy for RFMix. Specifically, W=4000, H=100, and J=10 are selected. In addition, two types of ancestral origin indicators are used: one-hot encoding to select one of three ancestral origins (C=3) and ancestral origin locale coordinates (C=2).

[0134] From the experiment, 100 simulated samples for each ancestor were generated using the generative machine learning model 600 and used to train RFMix. RFMix was then evaluated on 200 SNP sequences from mixed individuals. RFMix was also trained on 258 SNP sequences from single-ancestor individuals and then evaluated again on the same 200 SNP sequences from mixed individuals. The inference accuracy of local ancestor inference by RFMix trained on two different datasets was then compared. Table 2 below illustrates the experimental results: [Table 2]

[0135] As shown in Table 2 above, RFMix achieves comparable accuracy when trained with African-source data and datasets generated by the generative machine learning model 600. The accuracy results also indicate that adding a discriminator and adversarial loss helps the network learn to simulate human chromosome sequences that are more similar to African-source datasets, and is therefore more useful in training local ancestry inference models such as RFMix, thereby providing a significant increase in accuracy.

[0136] In addition, a similarity analysis is performed between simulated SNP sequences (generated by a generative machine learning model 600 from 258 single-ancestral individuals) and the actual SNP sequences of the 258 single-ancestral individuals. Extensive sampling of the simulated SNP sequences is performed to determine the frequency of simulated individuals matching one SNP sequence from the 258 single-ancestral individuals with thresholds of 99.9%, 99.99%, 99.999%, and 100%. Table 3 below shows the number of matches after generating 10,000 SNP sequences representing 10,000 individuals per ancestor: [Table 3]

[0137] V. Segment Merging In the embodiment shown in Figure 6, the encoder 602 and decoder 604 can be trained for a specific window of the genome. Then, by providing input sequences along with trait indicators, simulated sequences indistinguishable from actual sequences possessing the same traits can be generated. A separate model can be trained for each genome window. Therefore, each simulated genome window can be generated independently. However, it may be desirable to interconnect windows with simulated sequences of multiple windows (segments) that are collectively generated based on input sequences spanning the windows.

[0138] To provide interconnections, embodiments may add one or more layers of a model receiving input vectors, and / or embedding vectors for multiple windows. For example, there may be extra layers for neural networks interconnecting neural networks for various windows. In this way, the simulated sequences can more realistically simulate a combined genome affected by associating windows with various traits. Long-term relationships between distal regions can be captured for a given trait. For example, one window may have Spanish ancestry, another window may have Native American ancestry, and the interconnections can simulate a modern Latino person in the real world.

[0139] Figure 7 shows a sample architecture of a machine learning model 700 that provides relationships between various mutant segments according to embodiments of the present disclosure. The machine learning model 700 can be used when the input sequence is very long or when it is desired to model segments as subsequences (for example, to simulate individuals with mixed traits). The entire input sequence can be viewed as a single or multiple segments of a window, each segment corresponding to a different window. In the latter scenario, segments can form larger regions or supersegments.

[0140] The input sequence is the entire sequence of the window in which the simulated sequence is desired. For example, a mutant value of 0 or 1 may indicate whether a non-wild-type allele is present at the site (e.g., a different allele from the reference sequence). Different sites can be associated with different types of mutants. The windowed sequence shows mutant values ​​grouped by different mutant segments (windows), each corresponding to a different set of mutant sites.

[0141] Each set of variant values ​​for a given variant segment is provided as input to its respective encoder 702. As shown, there are four variant segments corresponding to encoders 1-4. In addition, each trait indicator vector 712 (P1-P4) provides its respective input to its respective encoder 702. The trait indicator vectors 712 can provide indications for a given window of input sequences whether one or more traits (e.g., phenotypes, ancestral indicators, ...) exist as a result of an object having one or more traits (e.g., a window sequence is obtained). These indicator / phenotype / trait descriptors can be provided by physicians, or by questionnaires, or by other techniques for biobank creation, or can be obtained by external algorithms (e.g., ancestral indicators can be obtained automatically by local ancestry inference methods).

[0142] Each trait indicator vector 712(P1, P2, ...) is input to each encoder 702 of the encoder system and to the decoding interconnect module 708 (an RNN2 module as shown) of the decoder system. Thus, each encoder 702(1, 2, ...) can receive the corresponding windowed array and its respective trait vector. The two inputs can be concatenated before being input. The decoding interconnect module 708 can receive the array of Gaussian embeddings concatenated with the trait indicator vectors as input.

[0143] Each encoder 702 outputs an encoder hidden layer for each window (e.g., a mutant segment). Each part of the encoder hidden layer (e.g., he1) may correspond to the outputs of the encoders described in a previous section for, for example, the distribution generation submodel 306. Thus, each part of the encoder hidden layer may exist in latent space.

[0144] The encoding interconnect module 706 receives the encoder output as an encoder hidden layer. In the illustrated embodiment, the encoding interconnect module 706 is a recurrent neural network (RNN). The encoding interconnect module 706 acts on all the values ​​in the latent space of each encoder (i.e., each window), and therefore acts collectively. The encoding interconnect module 706 provides an output that may or may not be the same as the size of the latent space for each segment (window) contained in the input array.

[0145] The embedding vector 732 can be determined in a manner similar to that described herein, for example, in Section III. As shown, the embedding vector 732 is determined using a Gaussian distribution. Sampling of the distribution can be performed after the encoding interconnect module 706 as part of the generation of the embedding vector 732, for example, as described herein for other sections. The decoding interconnect module 708 receives the embedding vector 732 and outputs a decoder hidden layer. The decoding interconnect module 708 acts on all possible values ​​of the embedding vector 732 in latent space and can also receive inputs from each trait indicator 712 of the window and thus act collectively on different window values. The encoding and decoding hidden layers may have the same or different amounts of data (e.g., the same number of dimensions), and the embedding vector 732 may be the same or different size as the hidden layer.

[0146] Each decoder 704 receives a portion of the decoder hidden layer (hd1-hd4) and outputs variant values ​​for each window in the reconstructed / simulated, windowed sequence, thereby yielding the final reconstructed / simulated sequence. Each decoder 704 may correspond to a decoder described in a previous section, for example, the sequence generation submodel 308.

[0147] The interconnection modules 706 and 708 can treat each he* and hd* as a single entry in the array. Thus, embodiments can include a network layer capable of modeling the array. Although the interconnection modules are named RNNs, they do not have to be recurrent neural networks (RNNs). Any neural architecture capable of modeling a 1d array can be applied, or any other differentiable function can be applied. Such examples include recurrent neural networks (RNNs) such as Long Short-Term Memory Networks (LSTMs) and Gated Recurrent Units (GRUs), 1d convolutional neural networks (CNNs) including ResNet-style architectures, transformer-based networks such as self-attention layers and networks with any fast variation of transformers, multilayer perceptron networks (MLPs)-mixers, and fully connected array modeling including networks such as graph MLPs (gMLPs).

[0148] As described above, interconnection modules 706 and 708 are optional. If they are not included, each subsequence (window) is processed independently, and any possible correlations between subsequences are not captured by the machine learning model 700. If they are removed, the machine learning model 700 can operate independently of any different subsequence, for example, in the same manner as described in Figures 3B to 6.

[0149] VI. Method Figure 8 illustrates a method 800 for generating a simulated genome sequence. The simulated genome sequence may include sequences of variant (e.g., SNP) values ​​for multiple variant (e.g., SNP) sites. Method 800 can be performed by a computer system implementing a generative machine learning model, such as generative machine learning model 300.

[0150] In step 802, the computer system receives a trait indicator as input. The trait indicator may include, for example, the ancestral origin indicator 304 in Figure 3A or other trait indicators. The computer system may receive other inputs. For example, the computer system may receive input mutant segments (e.g., SNP segments) for multiple mutant sites (e.g., SNP sites) of the genome of a subject having the trait associated with the trait indicator.

[0151] A mutant segment may be represented by a sequence of mutant values ​​at the mutant site (e.g., SNP values, other alleles, or methylation status). This sequence of mutant values ​​may also be referred to as an input vector. Each mutant value can specify a mutant at the mutant site. A mutant segment can be associated with a trait indicator and, for example, stored together with the trait indicator, based on the fact that the mutant segment originates from an object possessing the trait. In another embodiment, a computer system may receive information identifying multiple mutant sites from which sequences of mutant values ​​are generated.

[0152] As an example, the trait may be an ancestral trait, a biomedical trait, a demographic trait, or another phenotype as described herein. Furthermore, two or more trait indicators may be input. In such a situation, the mutant segment may be associated with one or more subjects having the multiple trait indicators provided. Thus, if a subject also has one or more additional traits, one or more additional trait indicators corresponding to one or more additional trait indicators may be received.

[0153] In step 804, the computer system obtains the probability distribution of the embedding vector in the latent space based on the trait indicator. The probability distribution can be generated by a distribution-generating submodel of a trained generative machine learning model from an input vector (e.g., an input mutant segment) that represents the sequence of mutant values ​​at multiple mutant sites in the genome of a subject possessing the trait. For example, the input vector and the trait indicator can be input into the distribution-generating submodel to generate the probability distribution.

[0154] Each variant value can specify a particular variant at the variant site (e.g., a specific base (A, C, G, T), a specific methylation status (methylated or unmethylated)). In some implementations, 0 can identify a reference value in the reference genome (e.g., an allele) or another common one in the population, while 1 can indicate the presence of a particular type of variant. The input vector can be defined in a variant segment space having a first number of dimensions, each corresponding to a variant site. The latent space can have a second number of dimensions smaller than the first number of dimensions. The probability distribution can be considered as a multidimensional entity with a second number of dimensions.

[0155] The type of variant may correspond to a class or characteristic of the variant value at a site. For example, one type of variant is a single nucleotide polymorphism (SNP) in which the variant value is a different nucleotide or, in some cases, a deleted nucleotide. Other examples of variant types, such as deletions, amplifications (e.g., short tandem repeats), insertions, inversions, and methylation statuses, are provided herein. Multiple variant sites may have multiple types of variants, for example, some sites may be SNPs and others may be methylation statuses.

[0156] In some embodiments, as part of step 804, the computer system may employ a distribution-generating submodel, such as distribution-generating submodel 306, to compute a probability distribution based on input vectors representing the input mutant segments. The distribution-generating submodel (e.g., acting as an encoder) can transform the input vectors in the mutant segment space into a multidimensional probability distribution of embedding vectors in a latent space having a reduced number of dimensions, for example, by mapping them to the mean and width (variance) of the distribution for each of the reduced number of dimensions. For an isotropic distribution, the variance is the same for each dimension. The distribution in the reduced space can represent the variation of the input mutant segments. The encoder may include a neural network model that takes input vectors and ancestor indicators as inputs and determines a multidimensional probability distribution based on the inputs.

[0157] In some embodiments, the computer system may also select a probability distribution of the embedding vector in the latent space from multiple probability distributions, each associated with a specific set of traits (e.g., ancestral origin) and variant (e.g., SNP) sites. The probability distribution can be computed by a distribution-generating submodel based on input variant segments of different traits (e.g., ancestral origin) at previous time points. Thus, each of the multiple probability distributions can be associated with a different trait indicator.

[0158] In step 806, the computer system obtains sample vectors by sampling the probability distribution in each of the second number dimensions in the latent space. Specifically, a random function and a sampling function can be implemented to perform sampling, as described with respect to Figures 3A to 3E. The random function can generate a random matrix based on an isotropic Gaussian distribution with zero mean and unit variance. The sampling function can generate sample vectors by reparameterizing the output random matrix (from the random function) by multiplying it by the variance vector of the probability distribution and adding the result of the multiplication to the mean vector of the probability distribution.

[0159] The probability distribution can include a Gaussian distribution (for example, as described in Section III.C), where the probability distribution is represented by the mean and variance of each dimension of the latent space. Obtaining the sample vector can involve the following steps for each of the dimensions of the second number: generating a random number, and combining the random numbers with their respective mean and variance to generate the values ​​of the dimension. The sample vector can then be formed based on the values ​​generated for the second number dimension of the latent space.

[0160] In step 808, the computer system uses a sequence-generating submodel of a trained generative machine learning model to reconstruct the output vector from the sample vector based on the trait indicator. In some embodiments, the sequence-generating submodel may or may be a decoder for implementing the reconstruction function. The reconstruction map can map samples of the embedding vector in the latent space to the output vector in the mutant segment space based on the trait of the input mutant segment. The output vector can then represent a simulated mutant segment of the trait. The decoder may also include a neural network model.

[0161] Method 800 can be repeated for multiple segments (for example, as shown in Figures 1B, 2A, and 2B). A computer system can receive multiple input mutant segments extracted from the input genome sequences of one or more subjects possessing traits. Each of the multiple input mutant segments can be a separate vector containing mutant values ​​at the mutant sites for that segment. The input mutant segments can contain input vectors so that the process is repeated for each segment. For each input mutant segment, a distribution-generating submodel can determine a probability distribution. By sampling the probability distribution, each sample vector can be obtained, thereby obtaining multiple respective sample vectors. A sequence-generating submodel can reconstruct each output vector from each sample vector based on each trait indicator of the segment, thereby determining multiple respective output vectors. A simulated genome sequence can then be generated based on each output vector (e.g., concatenated). The distribution-generating submodel and the sequence-generating submodel can form a class-conditional variational autoencoder (CVAE), where the traits of the input variational segments can represent various classes for the CVAE.

[0162] In step 810, the computer system generates a simulated genome sequence based on the output vectors. In some embodiments, the computer system may receive multiple input mutant segments and generate multiple output vectors representing the simulated mutant segments. In some embodiments, the computer system may also generate multiple output vectors for various mutant sites, each output vector being generated for a specific trait. In both cases, the output vectors can be concatenated to form a simulated genome sequence.

[0163] A. Implementation methods of neural networks As described in sections III.C and III.E of this specification and in other sections, a distribution-generating submodel may include a first neural network comprising a first input layer, a first hidden layer, and a first output layer. Each node in a first subset of nodes in the first input layer may correspond to a mutant site in an input mutant segment, receive a mutant value for the corresponding mutant site, and scale the mutant value by a first weight out of a plurality of first weights. Each node in the first hidden layer may generate a first median based on the sum of the scaled mutant values ​​from the first subset of nodes in the first input layer, and scale the first median based on a second weight out of a plurality of second weights to obtain a scaled first median. Each node in the first output layer may output the mean and variance of the dimensionality of the latent space based on the sum of the scaled first medians from each node in the first hidden layer. A plurality of first weights and a plurality of second weights may be selected based on the traits of the input mutant segment.

[0164] Each node in the second subset of nodes in the first input layer can receive a value representing the trait of the input mutant segment. Each node in the first hidden layer can generate a first intermediate value based on the sum of scaled mutant values ​​from the first subset of nodes in the first input layer and the sum of scaled values ​​representing traits from the second subset of nodes in the first input layer.

[0165] As further described in sections III.C and III.E of this specification and in other sections, the sequence generation submodel may include a second neural network comprising a second input layer, a second hidden layer, and a second output layer. Each node in a first subset of nodes in the second input layer corresponds to a dimension in the latent space, receives a sample vector value of the corresponding dimension, and can scale the sample vector value by a third weight. Each node in the second hidden layer can generate a second intermediate value based on the sum of the scaled sample vector values ​​from the first subset of nodes in the second input layer, and can scale the second intermediate value based on a fourth weight. Each node in the second output layer can output a vector value of its respective output vector representing a simulated mutant segment. The third and fourth weights can be selected based on the traits of the input mutant segment.

[0166] Each node in the second subset of nodes in the second input layer can receive a value representing the trait of the input mutant segment. Each node in the second hidden layer can generate a second intermediate value based on the sum of scaled mutant values ​​from the first subset of nodes in the second input layer and the sum of scaled values ​​representing traits from the second subset of nodes in the second input layer.

[0167] As further described in sections III.C and III.E of this specification and in other sections, the discriminator may include a third neural network comprising a third input layer, a third hidden layer, and a third output layer. Each node in a first subset of nodes in the third input layer may correspond to a mutant site, receive a mutant value for the corresponding mutant site in the output vector, and scale the mutant value by a fifth weight. Each node in the third hidden layer may generate a third median based on the sum of the scaled mutant values ​​from the first subset of nodes in the third input layer, and may scale the third median based on a sixth weight to obtain a scaled third median. The third output layer may include a node for calculating the probability that the output vector represents an actual mutant segment based on the scaled third median from the third hidden layer. The fifth and sixth weights may be selected based on the traits of the input mutant segment.

[0168] Each node in the second subset of nodes in the third input layer can receive a value representing the trait of the input mutant segment. Each node in the third hidden layer can generate a third intermediate value based on the sum of scaled mutant values ​​from the first subset of nodes in the third input layer and the sum of scaled values ​​representing traits from the second subset of nodes in the third input layer.

[0169] B. Training As described in Section III.E and other sections of this specification, encoders (e.g., distribution-generating submodels) and decoders (e.g., sequence-generating submodels) can be part of a CVAE and can be trained to fit various patterns of mutants to a target multidimensional probability distribution while reducing information loss in mapping from the mutant segment space to the latent space. This ensures that the simulated mutant segments generated by the decoder are statistically related to the input mutant segments according to the multidimensional probability distribution and can simulate the effects of random variation in the mutant segments. As shown in Figure 4, training of the encoder and decoder can be based on minimizing a loss function that combines the reconstruction error (between each of the input and output vectors) and a penalty for divergence from the target probability distribution (e.g., based on the difference in parameters (e.g., mean and variance) between the multidimensional probability distribution and, e.g., the target values ​​of the target probability distribution). Training operations can be performed to reduce or minimize the reconstruction error and the penalty for distribution divergence so that the distribution of mutant segments generated by the encoder can be forced to (to some extent) match a target probability distribution, which may be a zero-mean unit variance Gaussian distribution. The center (mean) and variance of the mutant segment distribution can be set based on reducing / minimizing the penalty for reconstruction error and distribution divergence.

[0170] To further reduce distribution errors so that simulated mutant segments more closely follow the target probability distribution, the CVAE can be trained using a class-conditional generative adversarial network (CGAN) including the decoder and discriminator in the aforementioned training operation, for example, as shown in Figures 5A and 5B. The discriminator can also be implemented as a neural network model, which can classify whether the mutant segment output by the decoder is an actual mutant segment or a simulated mutant segment. The discriminator may not be able to distinguish an actual mutant segment from a simulated mutant segment when the simulated mutant segment follows the target probability distribution, and at this point the discriminator's classification error rate may reach its maximum, meaning that the decoder reconstruction is optimal. An adversarial training operation can be performed in which the decoder parameters can be adjusted to increase the classification error rate so that the probability distribution in the reduced dimension approaches the target probability distribution, and the discriminator parameters can be adjusted to decrease the classification error rate. The training operation may be stopped when approximately half of the output vector represents actual mutant segments and approximately half of the output vector represents fake / simulated mutant segments.

[0171] As described in sections III.D and III.E of this specification and in other sections, a distribution-generating submodel can be trained on a first loss function that includes a reconstruction error component and a distribution error component. The reconstruction error component may be based on the difference between the output vector and the input vector. The distribution error component may be based on the difference between the probability distribution of the embedding vector and the target probability distribution. The parameters of the distribution-generating submodel can be adjusted to decrease the first loss function. The distribution error component may be based on the Kullback-Leibler divergence.

[0172] A sequence generation submodel can be trained based on a second loss function that includes a reconstruction error component. The sequence generation submodel can be trained in an adversarial training operation using a discriminator that classifies, based on the traits of the input mutant segment, whether the output vector produced by the sequence generation submodel represents an actual mutant sequence or a simulated mutant sequence. The second loss function may further include an adversarial loss component that decreases as the classification error rate in the discriminator increases. The discriminator can be trained based on a third loss function that decreases as the classification error rate decreases. The parameters of the sequence generation submodel and the discriminator can be adjusted to decrease the second and third loss functions, respectively.

[0173] C. Collective analysis of window sequences As described in Section V, each of the multiple output vectors can be collectively reconstructed from each of the multiple sample vectors. For example, the probability distribution can be collectively determined for multiple input variant segments.

[0174] For each of the multiple input mutant segments, each encoder of the sequence generation submodel can receive the mutant value of the input mutant segment and one or more respective trait indicators. Using one or more respective trait indicators, each encoder can act on the mutant value of the input mutant segment and output its respective encoder hidden vector (e.g., in a space of size between the mutant segment space and the latent space). Multiple encoder hidden vectors can be obtained. The encoding interconnect module can then receive these multiple encoder hidden vectors. The encoding interconnect module can generate embedding vectors that define probability distributions for each of the second number dimensions in the latent space for each of the multiple input mutant segments.

[0175] The reconstruction of multiple individual output vectors can be performed collectively using embedding vectors. A decoding interconnect module can receive the embedding vectors and one or more respective trait indicators. Using the trait indicators for multiple input mutant segments, the decoding interconnect module can act on the embedding vectors to output the respective decoder hidden vector for each of the multiple input mutant segments. For each of the multiple input mutant segments, each decoder of the sequence generation submodel can act on its respective decoder hidden vector to obtain the respective output vector for the input mutant segment.

[0176] VII. Computer Systems Any of the computer systems referred to herein may utilize any number of suitable subsystems. An example of such subsystems in computer system 10 is shown in Figure 9. In some embodiments, the computer system may include a single computer device, with subsystems being components of the computer device. In other embodiments, the computer system may include multiple computer devices, each of which is a subsystem having internal components. Examples of computer systems include desktop and laptop computers, tablets, mobile phones, and other portable devices. In some embodiments, the disclosed techniques can be implemented using cloud infrastructure (e.g., Amazon Web Services), graphics processing units (GPUs), etc.

[0177] The subsystems shown in Figure 9 are interconnected via a system bus 75. Additional subsystems are shown, such as a printer 74, a keyboard 78, a storage device 79, and a monitor 76 coupled to a display adapter 82. Peripherals and input / output (I / O) devices coupled to the I / O controller 71 can be connected to the computer system by any number of means known in the art, such as input / output (I / O) ports 77 (e.g., USB, FireWire®). For example, the computer system 10 can be connected to a wide area network such as the Internet, a mouse input device, or a scanner using the I / O ports 77 or an external interface 81 (e.g., Ethernet, Wi-Fi). The interconnection via the system bus 75 allows the central processor 73 to communicate with each subsystem and control the execution of multiple instructions from the system memory 72 or storage device 79 (e.g., a hard drive or a fixed disk such as an optical disk), and the exchange of information between subsystems. The system memory 72 and / or storage device 79 may embody computer-readable media. Another subsystem is a data acquisition device 85, such as a camera, microphone, and accelerometer. Any of the data referred to herein can be output from one component to another and to the user.

[0178] A computer system may include multiple identical components or subsystems connected to each other, for example, by an external interface 81 or by an internal interface. In some embodiments, computer systems, subsystems, or devices may communicate over a network. In such cases, one computer may be considered a client and another a server, each of which may be part of the same computer system. The client and the server may each include multiple systems, subsystems, or components.

[0179] Aspects of the embodiments may be implemented in the form of control logic using hardware (e.g., application-specific integrated circuits or field-programmable gate arrays) and / or computer software having a generally programmable processor in a modular or integrated manner. As used herein, the processor includes a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked together. Based on the disclosure and teachings provided herein, those skilled in the art will recognize and understand other ways and / or methods for implementing embodiments of the disclosure using hardware and combinations of hardware and software.

[0180] Any of the software components or functions described in this application may be implemented as software code executed by a processor, for example, using any suitable computer language such as Java, C, C++, C#, Objective-C, or Swift, or a scripting language such as Perl or Python using prior art or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-temporary computer-readable media include random access memory (RAM), read-only memory (ROM), magnetic media such as hard drives or floppy disks, optical media such as compact discs (CDs) or DVDs (digital multipurpose discs), and flash memory. The computer-readable medium may be any combination of such storage or transmission devices.

[0181] Such programs may be transmitted using carrier signals that are encoded and adapted for transmission over wired networks, optical networks, and / or wireless networks that conform to a variety of protocols, including the Internet. Thus, computer-readable media may be created using data signals encoded by such programs. Computer-readable media encoded with program code may be packaged with a compatible device or provided separately from other devices (e.g., via internet download). Any such computer-readable media may reside on or within a single computer product (e.g., a hard drive, CD, or an entire computer system), or on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing the user with any of the results referred to herein.

[0182] Any of the methods described herein can be performed in whole or in part using a computer system including one or more processors that can be configured to perform the steps. Therefore, embodiments may apply to computer systems that perform the steps of any of the methods described herein, and in some cases, different components configured to perform each step or each group of steps. Although presented as numbered steps, the steps of the methods herein can be performed simultaneously or in different orders. Additionally, parts of these steps may be used in combination with parts of other steps from other methods. Furthermore, all or part of the steps may be optional. Additionally, any of the steps of any of the methods can be performed by a module, unit, circuit, or other means for performing these steps.

[0183] Specific details of particular embodiments may be combined in any preferred manner without departing from the spirit and scope of the embodiments of this disclosure. However, other embodiments of this disclosure may focus on specific embodiments relating to each individual aspect or specific combinations thereof.

[0184] The above description of exemplary embodiments of the Disclosure is presented for illustrative and explanatory purposes only. It is not intended to be comprehensive or to limit the Disclosure to the forms described herein, and many modifications and changes are possible in light of the above teachings.

[0185] The use of “a,” “an,” or “the” is intended to mean “one or more” unless otherwise specifically indicated. The use of “or” is intended to mean “inclusive or” rather than “exclusive or” unless otherwise specifically indicated. A reference to a “first” component does not necessarily require the provision of a second component. Furthermore, a reference to a “first” or “second” component does not limit the component being referred to to a specific location unless explicitly stated.

[0186] All patents, patent applications, publications, and descriptions referenced herein are incorporated in their entirety by reference for all purposes. Nothing is considered prior art.

[0187] This specification includes an appendix containing additional information relating to specific embodiments. Other terms used in the appendix may not yet be commonly used in the industry. [Document Title] Claims [Section 1] A computer implementation method for generating a simulated genome sequence, Receiving trait indicators, Obtaining a probability distribution of an embedding vector in a latent space based on the trait indicator, wherein the probability distribution is generated by a distribution-generating submodel of a trained generative machine learning model from an input vector representing a sequence of mutant values ​​at multiple mutant sites in the genome of a subject having the trait associated with the trait indicator, each mutant value designates a specific mutant present at the mutant site, the input vector is defined in a mutant segment space having a first number of dimensions corresponding to the multiple mutant sites, the latent space has a second number of dimensions smaller than the first number of dimensions, and the probability distribution is multidimensional having the second number of dimensions. A sample vector is obtained by sampling the probability distribution in each of the second number dimensions of the latent space, Reconstructing an output vector from the sample vector using a sequence generation submodel of the trained generative machine learning model and based on the trait indicator, wherein the output vector is defined in the mutant segment space, and the reconstruction is performed accordingly. A computer implementation method comprising generating the simulated genome sequence based on the output vector. [Section 2] The method according to item 1, wherein the type of variant for at least one of the plurality of variant sites is a single nucleotide polymorphism (SNP). [Section 3] The method according to item 2, wherein the plurality of mutant sites have a plurality of types of mutants. [Section 4] The aforementioned trait is of ancestral origin, as described in item 1. [Section 5] The method according to item 1, wherein the trait is a biomedical trait or a demographic trait. [Section 6] The method of claim 1, further comprising receiving one or more additional trait indicators corresponding to one or more additional traits, wherein the subject also has the one or more additional traits. [Section 7] The method according to item 1, wherein obtaining the aforementioned probability distribution includes selecting the aforementioned probability distribution from a plurality of probability distributions, each associated with a different trait indicator. [Section 8] The method according to item 1, wherein obtaining the probability distribution includes inputting the input vector and the trait indicator into the distribution generation submodel in order to generate the probability distribution. [Section 9] Receiving a plurality of input mutant segments extracted from the target input genome sequence, wherein each of the plurality of input mutant segments includes a mutant value at the mutant site, and the plurality of input mutant segments includes the input vector. For each of the above-mentioned multiple input variant segments, The probability distribution is determined by the aforementioned distribution generation submodel. By sampling the aforementioned probability distribution, each sample vector is obtained, and thereby multiple sample vectors are obtained, and The process involves reconstructing each output vector from each sample vector using the aforementioned sequence generation submodel and based on each trait indicator, thereby determining multiple output vectors. The method of claim 8, further comprising generating the simulated genome sequence based on each of the output vectors. [Section 10] The method according to paragraph 9, wherein each of the plurality of output vectors is collectively reconstructed from each of the plurality of sample vectors. [Section 11] Determining the aforementioned probability distribution is performed collectively for the plurality of input variant segments, For each of the above-mentioned multiple input variant segments, Each encoder of the sequence generation submodel receives the variant value of the input variant segment and one or more respective trait indicators, and Each encoder acts on the mutant value of the input mutant segment by using one or more trait indicators, outputs a separate encoder hidden vector, and thereby obtains multiple encoder hidden vectors. The encoding interconnect module receives the multiple encoder hidden vectors, The method according to item 10, comprising: using the encoding interconnection module to generate an embedding vector for each of the plurality of input variant segments that defines the probability distribution for each of the second number dimensions in the latent space. [Section 12] Reconstructing each of the aforementioned multiple output vectors is performed collectively using the embedding vectors, In the decoding interconnection module, the embedding vector and the one or more trait indicators are received, The decoding interconnection module acts on the embedding vector by using the trait indicators of the multiple input mutant segments and outputs a corresponding decoder hidden vector for each of the multiple input mutant segments, For each of the above-mentioned multiple input variant segments, The method according to paragraph 11, comprising: using each decoder of the sequence generation submodel to act on each decoder hidden vector to obtain each output vector of the input variant segment. [Section 13] The probability distribution includes a Gaussian distribution, and the probability distribution is represented by the mean and variance for each dimension of the latent space, and obtaining the sample vector is, For each of the dimensions of the second number described above, Generating random numbers, and The process involves combining the aforementioned random numbers with their respective mean and variance to generate the value of the dimension, The method of paragraph 9, comprising forming the sample vector based on the values ​​generated for the second number dimension of the latent space. [Section 14] The distribution generation submodel includes a first neural network, the first neural network includes a first input layer, a first hidden layer, and a first output layer. Each node in the first subset of the nodes in the first input layer corresponds to a mutant site in the input mutant segment, receives the mutant value of the corresponding mutant site, and scales the mutant value by a first weight among a plurality of first weights. Each node in the first hidden layer generates a first intermediate value based on the sum of scaled variant values ​​from the first subset of nodes in the first input layer, and scales the first intermediate value based on a second weight among a plurality of second weights to obtain a scaled first intermediate value. The method according to paragraph 13, wherein each node of the first output layer outputs the mean and variance of the dimensions of the latent space based on the sum of the scaled first medians from each node of the first hidden layer. [Section 15] Each node in the second subset of the nodes in the first input layer receives a value representing the trait of the input mutant segment, The method according to item 14, wherein each node in the first hidden layer generates the first intermediate value based on the sum of scaled mutant values ​​from the first subset of nodes in the first input layer and the sum of scaled values ​​representing the trait from the second subset of nodes in the first input layer. [Section 16] The method of claim 14, further comprising selecting the plurality of first weights and the plurality of second weights based on the traits of the input mutant segment. [Section 17] The sequence generation submodel includes a second neural network, the second neural network includes a second input layer, a second hidden layer, and a second output layer. Each node in the first subset of the nodes in the second input layer corresponds to the dimension of the latent space, receives a sample vector value of the corresponding dimension, scales the sample vector value by a third weight, Each node in the second hidden layer generates a second median based on the sum of scaled sample vector values ​​from the first subset of nodes in the second input layer, and scales the second median based on a fourth weight. The method according to item 13, wherein each node in the second output layer outputs the vector value of the respective output vector representing the simulated mutant segment. [Section 18] Each node in the second subset of nodes in the second input layer receives a value representing the trait of the input mutant segment, The method according to paragraph 17, wherein each node in the second hidden layer generates a second intermediate value based on the sum of scaled variant values ​​from the first subset of nodes in the second input layer and the sum of scaled values ​​representing the trait from the second subset of nodes in the second input layer. [Section 19] The method of claim 17, further comprising selecting the third and fourth weights based on the traits of the input mutant segment. [Section 20] The method according to item 8, wherein the distribution generation submodel and the sequence generation submodel form a class-conditional variational autoencoder (CVAE), and multiple traits of multiple input variant segments represent different classes of the CVAE. [Section 21] The distribution generation submodel is trained based on a first loss function that includes a reconstruction error component and a distribution error component. The reconstruction error component is based on the difference between the output vector and the input vector. The aforementioned distribution error component is based on the difference between the probability distribution of the embedding vector and the target probability distribution. The method according to item 8, wherein the parameters of the distribution generating submodel are adjusted to reduce the first loss function. [Section 22] The distribution error component is determined by the method described in Section 21, based on the Kullback-Leibler divergence. [Section 23] The method according to item 21, wherein the sequence generation submodel is trained based on a second loss function that includes the reconstruction error component. [Section 24] The method according to paragraph 23, wherein the sequence generation submodel is trained in an adversarial training operation using a discriminant that classifies whether the output vector output by the sequence generation submodel represents an actual mutant sequence or a simulated mutant sequence, based on the traits of the input mutant segment. [Section 25] The second loss function further includes an adversarial loss component that decreases as the classification error rate in the discriminator increases, The classifier is trained based on a third loss function that decreases as the classification error rate decreases. The method according to item 24, wherein the parameters of the sequence generation submodel and the discriminator are adjusted to reduce the second loss function and the third loss function, respectively. [Section 26] The discriminator includes a third neural network, the third neural network includes a third input layer, a third hidden layer, and a third output layer. Each node in the first subset of the nodes in the third input layer corresponds to a mutant site, receives the mutant value of the corresponding mutant site in the output vector, and scales the mutant value by a fifth weight. Each node in the third hidden layer generates a third intermediate value based on the sum of scaled variant values ​​from the first subset of nodes in the third input layer, and scales the third intermediate value based on a sixth weight to obtain a scaled third intermediate value. The method according to item 25, wherein the third output layer includes a node for calculating the probability that the output vector represents an actual mutant segment based on the scaled third intermediate value from the third hidden layer. [Section 27] Each node in the second subset of the nodes of the third input layer receives a value representing the trait of the input mutant segment, The method according to paragraph 26, wherein each node in the third hidden layer generates the third intermediate value based on the sum of scaled mutant values ​​from the first subset of nodes in the third input layer and the sum of scaled values ​​representing the trait from the second subset of nodes in the third input layer. [Section 28] The method according to claim 26, further comprising selecting the fifth and sixth weights based on the traits of the input mutant segment. [Section 29] The method according to item 24, wherein the sequence generation submodel and the discriminant form a class-conditional generative adversarial network (CGAN). [Section 30] A computer product comprising a non-temporary computer-readable medium that stores a number of instructions, when executed, causing a computer system to perform the method described in any one of the preceding terms. [Section 31] It is a system, The computer products described in item 30, A system comprising one or more processors for executing instructions stored on the aforementioned non-temporary computer-readable medium. [Section 32] A system comprising means for performing the method described in any one of the preceding terms. [Section 33] A system comprising one or more processors configured to perform the method described in any one of the preceding terms. [Section 34] A system comprising modules that each perform a step of the method described in any one of the preceding items.

Claims

1. A computer implementation method for generating a simulated genome sequence, Receiving trait indicators, This includes determining the probability distribution of the embedding vector in the latent space using a distribution-generating submodel of a trained generative machine learning model based on the aforementioned trait indicator, Determining the probability distribution of the aforementioned embedding vector is: Receiving a plurality of input mutant segments extracted from the input genome sequence of a target having the trait associated with the trait indicator, wherein each of the plurality of input mutant segments includes a mutant value in a set of mutant sites among a plurality of mutant sites of the target genome. For each of the above-mentioned multiple input variant segments, Each encoder of the distribution generation submodel receives the mutant value of the input mutant segment and one or more respective trait indicators, and Each encoder acts on the mutant value of the input mutant segment by using one or more trait indicators, outputs a separate encoder hidden vector, and thereby obtains multiple encoder hidden vectors. The encoding interconnect module receives the multiple encoder hidden vectors, The encoding interconnection module generates, based on the plurality of encoder hidden vectors, an embedding vector that defines the probability distribution for each of the plurality of input variant segments for each of the second number dimensions in the latent space, Includes, The input mutant segment defines a mutant segment space having a first number dimension corresponding to the plurality of mutant sites, the latent space has a second number dimension smaller than the first number dimension, and the probability distribution is multidimensional having the second number dimension. The aforementioned computer implementation method is A sample vector is obtained by sampling the probability distribution in each of the second number dimensions of the latent space, Reconstructing an output vector from the sample vector using a sequence generation submodel of the trained generative machine learning model and based on the trait indicator, wherein the output vector is defined in the mutant segment space, and the reconstruction is performed accordingly. A computer implementation method comprising generating the simulated genome sequence based on the output vector.

2. The method according to claim 1, wherein the type of mutant for at least one of the plurality of mutant sites is a single nucleotide polymorphism (SNP).

3. The method according to claim 1, wherein the trait is of ancestral origin, a biomedical trait, or a demographic trait.

4. Determining the aforementioned probability distribution means To generate an input vector representing the sequence of mutant values ​​in the plurality of mutant sites, To generate the aforementioned probability distribution, the input vector and the trait indicator are input to the distribution generation submodel, The method according to claim 1, further comprising:

5. For each of the plurality of input mutant segments, By sampling the aforementioned probability distribution, each sample vector is obtained, and thereby multiple sample vectors are obtained, and The process involves reconstructing each output vector from each sample vector using the aforementioned sequence generation submodel and based on each trait indicator, thereby determining multiple output vectors. The method according to claim 4, further comprising generating the simulated genome sequence based on each of the output vectors.

6. The method according to claim 5, wherein each of the plurality of output vectors is collectively reconstructed from each of the plurality of sample vectors.

7. Reconstructing each of the aforementioned multiple output vectors is performed collectively using the embedding vectors, In the decoding interconnection module, the embedding vector and the one or more trait indicators are received, The decoding interconnection module acts on the embedding vector by using the trait indicators of the multiple input mutant segments and outputs a corresponding decoder hidden vector for each of the multiple input mutant segments, For each of the above-mentioned multiple input variant segments, The method according to claim 5, comprising: using each decoder of the sequence generation submodel to act on each decoder hidden vector to obtain each output vector of the input variant segment.

8. The probability distribution includes a Gaussian distribution, and the probability distribution is represented by the mean and variance for each dimension of the latent space, and obtaining the sample vector is, For each of the dimensions of the second number described above, Generating random numbers, and The process involves combining the aforementioned random numbers with their respective mean and variance to generate the value of the dimension, The method of claim 5, comprising forming the sample vector based on the values ​​generated for the second number dimension of the latent space.

9. The distribution generation submodel includes a first neural network, the first neural network includes a first input layer, a first hidden layer, and a first output layer. Each node in the first subset of the nodes in the first input layer corresponds to a mutant site in the input mutant segment, receives the mutant value of the corresponding mutant site, and scales the mutant value by a first weight among a plurality of first weights. Each node in the first hidden layer generates a first intermediate value based on the sum of scaled variant values ​​from the first subset of nodes in the first input layer, and scales the first intermediate value based on a second weight among a plurality of second weights to obtain a scaled first intermediate value. The method according to claim 8, wherein each node of the first output layer outputs the mean and variance of the dimensions of the latent space based on the sum of the scaled first intermediate values ​​from each node of the first hidden layer.

10. Each node in the second subset of the nodes in the first input layer receives a value representing the trait of the input mutant segment, The method according to claim 9, wherein each node in the first hidden layer generates a first intermediate value based on the sum of scaled mutant values ​​from a first subset of nodes in the first input layer and the sum of scaled values ​​representing the trait from a second subset of nodes in the first input layer.

11. The sequence generation submodel includes a second neural network, the second neural network includes a second input layer, a second hidden layer, and a second output layer. Each node in the first subset of the nodes in the second input layer corresponds to the dimension of the latent space, receives a sample vector value of the corresponding dimension, scales the sample vector value by a third weight, Each node in the second hidden layer generates a second intermediate value based on the sum of scaled sample vector values ​​from the first subset of nodes in the second input layer, and scales the second intermediate value based on a fourth weight. The method according to claim 8, wherein each node in the second output layer outputs the vector value of the respective output vector representing the simulated mutant segment.

12. Each node in the second subset of nodes in the second input layer receives a value representing the trait of the input mutant segment, The method according to claim 11, wherein each node in the second hidden layer generates the second intermediate value based on the sum of scaled variant values ​​from the first subset of nodes in the second input layer and the sum of scaled values ​​representing the trait from the second subset of nodes in the second input layer.

13. The method according to claim 4, wherein the distribution generation submodel and the sequence generation submodel form a class-conditional variational autoencoder (CVAE), and the multiple traits of the multiple input variant segments represent different classes of the CVAE.

14. A non-temporary computer-readable medium that stores a plurality of instructions that, when executed, cause a computer system to perform the method according to any one of claims 1 to 13.

15. One or more processors, A non-temporary computer-readable medium that stores a plurality of instructions which, when executed, cause one or more processors to perform the method according to any one of claims 1 to 13, A system that includes this.

Images