Downsampling of genomic sequence data

Abstract

In a method for automatically downsampling DNA sequence data using a variational autoencoder while preserving the genomic integrity of the original file, an embodiment includes an encoder that performs bootstrap on the genomic sequence data to generate resamples. The embodiment further includes an encoder that evaluates the non-representativeness and self-consistency of the resamples and selects a representative resample according to the evaluation. The modified encoder then constructs a vector representation from genotype likelihoods based on the selected representative resample. The embodiment also includes an analysis engine that integrates the mapping position information and the genotype likelihoods to identify an optimal vector representation of the resample. The modified decoder then decodes the identified optimal vector representation of the resample to obtain a downsampled read file that resembles the original file and maintains the genomic integrity of the original file.

Description

[Technical field]

[0001] The present invention relates generally to the field of genomic sequence data, and more particularly to downsampling deoxyribonucleic acid (DNA) sequence data. [Background technology]

[0002] Genomes are sequenced daily, where one human genome can occupy up to 100 gigabytes of storage space, so the storage needs for genome sequence data will grow from gigabytes to petabytes to exabytes. Furthermore, due to errors in sequencing, base calling, and genome alignment, a 30-fold increase in data (approximately 100 gigabytes) will need to be collected for every 3 billion bases of the human genome sequence. This means that the human genome alone will require 2 to 40 exabytes of storage capacity by 2025. Digital signal processing, downsampling, compression, and decimation are terms related to the process of resampling in multirate digital signal processing systems. Both downsampling and decimation can be synonymous with compression, or they can refer to the overall process of bandwidth reduction (filtering) and sample rate reduction. When a process is performed on an array of samples of a particular signal or other continuous function, downsampling produces an approximation of the array that would be obtained by sampling that signal at a lower rate (or density, as in the case of photography).

[0003] The advent of massively parallel high-throughput sequencing technology known as next-generation sequencing (NGS) has revolutionized the field of biological sciences. The advent of NGS has led to an increase in the amount and availability of DNA sequence data. The ultra-high scale and efficiency of NGS sequencing instruments can generate billions of short DNA reads, over several terabytes of data, in a single run. The short DNA reads are stored in FASTQ files (gigabytes of plain text files) that are used for a wide range of applications such as gene annotation, expression studies, personalized therapy, and precision medicine. The FASTQ format is a text-based format for storing both biological sequences (usually nucleotide sequences) and their corresponding quality scores. Both the sequence characters and the quality scores are each encoded with a single American Standard Code for Information Interchange (ASCII) character for brevity. Summary of the Invention

[0004] An embodiment of the present invention discloses a method, computer program product, and system for automatically downsampling DNA sequence data using a variational autoencoder and preserving genomic integrity of the original file, the method includes: an encoder performing bootstrap on the genomic sequence data to generate resamples; an encoder evaluating the resamples for non-representativeness and self-consistency and selecting a representative resample according to the evaluation; a modified encoder constructing a vector representation from genotype likelihoods based on the selected representative resample; an analysis engine integrating the mapping position information and the genotype likelihoods to identify an optimal vector representation of the resample; and a modified decoder decoding the identified optimal vector representation of the resample to obtain a downsampled read file that is similar to the original file and maintains the genomic integrity of the original file. [Brief description of the drawings]

[0005] [Figure 1]1 is a functional block diagram illustrating a distributed data processing environment, according to one embodiment of the present invention. [Figure 2A] FIG. 2 is a block diagram illustrating a downsampling component executing on a server computer in the distributed data processing environment of FIG. 1 for downsampling DNA sequence data using a variational autoencoder, according to one embodiment of the present invention. [Figure 2B] FIG. 2 illustrates an example overview of a bootstrap procedure, according to an embodiment of the present invention. [Figure 2C] FIG. 2 is a block diagram illustrating an overview of the generation of biological information within the distributed data processing environment of FIG. 1 for downsampling DNA sequence data using a variational autoencoder, according to one embodiment of the present invention. [Diagram 3] FIG. 2 illustrates operational steps of a downsampling component in a computing device in the distributed data processing environment of FIG. 1 for automatically downsampling DNA sequence data using a variational autoencoder, according to one embodiment of the present invention. [Figure 4] FIG. 2 is a block diagram of components of a server computer executing the downsampling component within the distributed data processing environment of FIG. 1 in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006]

[0005] Embodiments of the present invention recognize that there are problems with current methods and systems for genomic data analysis and integration and storage of DNA sequence data. For example, high-throughput genomic sequencing inherently produces files containing millions of reads per sample, files that require significant computational resources for storage and processing, and samples that need to be efficiently summarized (e.g., downsampled) to optimize computational efficiency. Embodiments of the present invention recognize that current methods of downsampling do not preserve the genomic integrity of the original samples.

[0007] It is known in the art that the explosion of genomic data presents significant challenges to the analysis, integration, sharing, and storage of DNA sequence data. Embodiments of the present invention address the above-mentioned issues by presenting systems and methods for downsampling billions of reads in FASTQ files to reduce file size and amount of DNA reads while preserving the integrity of the original information and avoiding loss of important genomic information (mainly due to loss of genomic variations, such as structural variations, that are encoded in a large number of reads), thereby enabling more effective and efficient downstream processing and analysis of genomic sequence data.

[0008] Additionally, embodiments of the present invention use composite representations to summarize genomic information. Embodiments of the present invention consider the entire genomic sequence data and transform the genomic information (e.g., the entire genomic sequence data) into a representation (e.g., a numerical vector) that preserves the original biological information, where the input data is derived from DNA sequencing of multiple reads (e.g., analyzing, identifying, and / or interpreting the transformations on multiple reads). In various embodiments of the present invention, the biological information primarily captured and encoded in the reads is various variations known in the art, such as insertion deletion mutations (indels), single nucleotide polymorphisms (SNPs), or coding number variations (cnvs), or combinations thereof, that can be used to explain different phenotypes. Additionally, embodiments of the present invention utilize shallow sampling using bootstrap techniques to calculate the minimum of representativeness and consistency. In some embodiments of the present invention, the present invention does not focus on single nucleotide polymorphism (SNP) calls, but rather on estimating genotype likelihoods at multiple positions within a read. Embodiments of the present invention may use a graph Poisson distribution to estimate genotype likelihoods. In various embodiments of the invention, the invention is a downsampling method, and in some instances does not involve alignment. In embodiments of the invention, alignment works when the user (i.e., the researcher) has a reference genome, and it is recognized that in many cases the reference genome is incomplete, and as a result the alignment is reference dependent (i.e., the alignment is only as good as the reference you have). In various embodiments of the invention, the invention focuses on estimating genotype likelihoods at multiple positions within a read for use as prior information, without relying on variant calling techniques.

[0009] The implementation of embodiments of the present invention may take various forms, and details of exemplary implementations are discussed below with reference to the figures (ie, FIGS. 1-4).

[0010] FIG. 1 is a functional block diagram illustrating a distributed data processing environment, generally designated 100, in accordance with one embodiment of the present invention. As used herein, the term "distributed" refers to a computer system that includes multiple physically separate devices that operate together as a single computer system. FIG. 1 provides only an illustration of one implementation and is not intended to suggest any limitation with respect to the environments in which various embodiments may be implemented. Many modifications to the illustrated environment may be made by one skilled in the art without departing from the scope of the invention as set forth in the claims. The distributed data processing environment 100 includes a computing device 110 and a server computer 120 interconnected through a network 130.

[0011] Network 130 may be, for example, a storage area network (SAN), a telecommunications network, a local area network (LAN), a wide area network (WAN) such as the Internet, wireless technologies for exchanging data over short distances (using short wavelength ultra-high frequency (UHF) radio waves in the Industrial, Scientific, and Medical (ISM) band of 2.4-2.485 GHz from fixed and mobile devices, a building personal area network (PAN), or a combination of the three), and may include wired, wireless, or fiber optic connections. Network 130 may include one or more wired networks, or wireless networks, or both, that may receive and transmit data, voice, or video signals including multimedia signals including voice, data, text, or video data, or combinations thereof. In general, network 130 may be any combination of connections and protocols that support communication between computing device 110 and server computer 120 and any other computing devices or storage devices (not shown in FIG. 1 ) in distributed data processing environment 100, or both.

[0012] In some embodiments of the present invention, computing device 110 may be, but is not limited to, a stand-alone device, a client, a server, a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a smartphone, a desktop computer, a smart television, a smart watch, a walkie-talkie, a stereo system, a cloud-based service (e.g., a cognitive cloud-based service), an AR pair of glasses, a virtual reality headset, any HUD known in the art, or any programmable electronic computing device capable of communicating with various components and devices in distributed data processing environment 100 via network 130, or any combination thereof. In general, computing device 110 may represent any programmable computing device or combination of programmable computing devices capable of executing machine-readable program instructions and communicating with users of other computing devices via network 130, or capable of executing machine-readable program instructions and communicating with server computer 120, or any combination thereof. In some embodiments, computing device 110 may represent multiple computing devices.

[0013] In some embodiments of the invention, computing device 110 may represent any programmable electronic computing device or combination of programmable electronic computing devices capable of executing machine-readable program instructions, manipulating executable machine-readable instructions, and communicating with server computer 120 and other computing devices (not shown) in distributed data processing environment 100 over a network, such as network 130. Computing device 110 may include an instance of user interface 106, and local storage 104. In various embodiments not shown in FIG. 1, computing device 110 may have multiple user interfaces. In other embodiments not shown in FIG. 1, distributed data processing environment 100 may include multiple computing devices, multiple server computers, or one or more networks, or combinations thereof. Computing device 110 may include internal and external hardware components as shown and as described in further detail with respect to FIG. 4.

[0014] The user interface (interface) 106 provides an interface with the prediction component 140. The computing device 110 may enable a user or client or both to interact with the component 140 or the server computer 120 or both in various ways, such as sending program instructions, receiving program instructions, sending or receiving or sending and receiving messages, updating data, sending data, inputting data, editing data, collecting data, or receiving data, or combinations thereof, through the user interface 106. In one embodiment, the interface 106 may be a graphical user interface (GUI) or a web user interface (WUI) and may display at least text, documents, web browser windows, user options, application interfaces, and instructions for operation. The interface 106 may include data presented to the user (such as pictorial, textual, and audio), as well as control sequences used by the user to control operations. In another embodiment, the interface 106 may be mobile application software that provides an interface between a user of the computing device 110 and the server computer 120. Mobile application software, or "apps," may be designed to run on smartphones, tablet computers, and other computing devices. In one embodiment, interface 106 may enable a user of computing device 110 to at least submit data, input data, edit (annotate) data, collect data, or receive data, or combinations thereof.

[0015] Server computer 120 may be a standalone computing device, an administrative server, a web server, a mobile computing device, one or more client servers, or any other electronic device or computing system capable of receiving, transmitting, and processing data. In other embodiments, server computer 120 may represent a server computing system utilizing multiple computers, such as, but not limited to, a server system, such as in a cloud computing environment. In another embodiment, server computer 120 may represent a computing system utilizing clustered computers and components (e.g., database server computers, application server computers, etc.) that function as a single pool of seamless resources when accessed within distributed data processing environment 100. Server computer 120 may include internal and external hardware components as illustrated and described in further detail with respect to FIG. 4. In some embodiments, server computer 120 may represent multiple server computers.

[0016] The shared storage 124 and the local storage 104 may each be a data / knowledge repository and / or database that can be written to and / or read by one or a combination of the components 140, the server computer 120 and the computing device 110. In the illustrated embodiment, the shared storage 124 is on the server computer 120 and the local storage 104 is on the computing device 110. In another embodiment, the shared storage 124 and / or the local storage 104 may be located elsewhere in the distributed data processing environment 100, each of which is accessible to and can be accessed by the computing device 110 and the server computer 120. The shared storage 124 and / or the local storage 104 may each be implemented using any type of storage device capable of storing data and configuration files that can be accessed and utilized by the server computer 120, such as, but not limited to, a database server, a hard disk drive, or flash memory.

[0017] In some embodiments of the present invention, the shared storage 124 and / or the local storage 104 may each be a hard drive, a memory card, computer output to a laser disk (cold storage), or any form of data storage known in the art, or a combination thereof. In some embodiments, the shared storage 124 and / or the local storage 104 may each be one or more cloud storage systems, or databases connected to a cloud network, or both. In various embodiments, the shared storage 124 and / or the local storage 104 may access, store, or contain, or a combination of, user data, physical room data, and meeting data, or data shared across the distributed data processing environment 100.

[0018] In the illustrated embodiment, the downsampling component (component) 140 executes on the server computer 120. In other embodiments, the downsampling component 140 may execute on the computing device 110. In various embodiments of the invention not illustrated in FIG. 1, the downsampling component 140 may execute on multiple server computers 120, or multiple computing devices 110, or combinations thereof. In some embodiments, the component 140 may be located and / or executed anywhere within the distributed data processing environment 100, so long as the downsampling component 140 is connected to and / or in communication with the computing devices 110 and / or the server computers 120 via the network 130. In the illustrated embodiment, the encoder 142, the decoder 144, and the analysis engine 146 are each located and executed in the downsampling component 140. In various embodiments not shown in FIG. 1, the encoder 142, the decoder 144, and the analysis engine 146 may execute anywhere within the distributed data processing environment 100, so long as the encoder 142, the decoder 144, or the analysis engine 146, or any combination thereof, is connected to and / or communicates with the computing device 110 and / or the server computer 120 via the network 130.

[0019] In various embodiments of the present invention, the encoder 142 may be one or more of a modified encoder, an audio encoder, a simple encoder, a priori encoder, a compressor, a rotary encoder, a linear encoder, an absolute encoder, an incremental encoder, or any other type of encoder known in the art, or a combination thereof. In various embodiments of the present invention, the decoder 144 may be one or more of an audio decoder, a binary decoder, a decompressor decoder, an instruction decoder, an orthogonal decoder, a video decoder, or any other decoder known in the art, or a combination thereof. The analysis engine 146 may be one or more analysis engines as understood in the art.

[0020] In various embodiments, the downsampling component 140, via the encoder 142, utilizes a stochastic resampling framework to select a subset of read fragments from the entire file. The encoder (e.g., the encoder 142) performs a bootstrap optimization to generate a set of resamples for a given size of N (e.g., N=1 million reads with a total of 100 resamples). In various embodiments of the invention, the downsampling component 140, via the encoder 142, selects a subset of encoded read fragments from one or more resamples after meeting one or more predefined criteria. In various embodiments of the invention, the decoder 144 may be used to generate a final set of read fragments that represent the original FASTQ file. In various embodiments of the invention, the decoder 144 is used to generate a final read fragment that may be put into a final resample. In various embodiments of the invention, the decoder 144 may be used to generate x′ (i.e., a final representative resample). The decoder 144 may reconstruct one or more of the vector representations that encode the reads as they move through the pipeline to generate the actual reads / DNA fragments.

[0021] In various embodiments of the invention, the downsampling component 140 consists of two steps: First, an encoder (e.g., encoder 142) retrieves and / or receives genomic sequence data and bootstraps to generate resamples, where the generated resamples are evaluated for non-representativeness and self-consistency, the variations are minimized, and the most representative resample is selected. Second, a decoder (e.g., decoder 144) receives the most representative sample (e.g., the selected most representative resample) as input and generates a reconstruction of the original input reads. In various embodiments of the invention, the downsampling component 140 removes duplicate reads from the FASTQ file to reduce redundancy of the genomic information. In various embodiments of the invention, the downsampling component 140 may process FASTQ files containing millions and / or billions of reads by tagging one or more reads (e.g., all reads) with a unique identifier and then mapping them to a sample reference sequence. In various embodiments of the invention, the downsampling component 140 may encode position information as well as mapping and quality scores for each read fragment via the encoder 142. In various embodiments of the invention, a statistical model is trained to estimate the genotype likelihood at each position and encode this information for each read fragment via encoder 142.

[0022] In various embodiments of the present invention, the downsampling component 140 may resample DNA fragments generated from massively parallel sequencing without compromising (i.e., maintaining) the genomic integrity of the original file. For example, genomic integrity refers to genomic variations that result in different phenotypes and are encoded in different reads (i.e., multiple or multiple reads). In this particular example, to maintain genomic integrity, the downsampling component 140 performs downsampling on one or more of the multiple reads, where the downsampling component 140 retains a majority or a predetermined amount of reads.

[0023] In various embodiments of the invention, the downsampling component 140 may retrieve and / or receive genomic sequence data (e.g., DNA sequence data) via the encoder 142 from one or more users, one or more databases, or one or more search engines, or a combination thereof, and may bootstrap the retrieved and / or received genomic sequence data to generate resamples. In various embodiments of the invention, the downsampling component 140 may perform stochastic resampling via the encoder 142 to obtain a set of resamples including multiple reads. In various embodiments of the invention, the component 140 may evaluate the non-representativeness and self-consistency of the resamples and select the best / "most representative" resample.

[0024] In various embodiments of the invention, the downsampling component 140 may construct a representation from the genotype likelihoods (e.g., converting genomic reads in FASTQ format to a numerical vector representation) via the encoder 142. The downsampling component 140 may integrate the mapping position information and the genotype likelihoods via the analysis engine 146 to identify an optimal vector representation of the resample. In various embodiments of the invention, the downsampling component 140 may take and / or receive the most representative sample as input via the decoder 144, resulting in a reconstruction of the original input reads. In various embodiments of the invention, the downsampling component 140 may decode the optimal resample vector representation via the decoder 144 to obtain a downsampled read file that resembles the original file and maintains the genomic integrity of the original file.

[0025] In various embodiments of the invention, given several reads overlapping one genomic location, the algorithm estimates the probability of genotype likelihood per read by maximizing the joint likelihood function parameters. These parameters are based on the Poisson Graphical Distribution (PGD). The parameters θ0, ..., θ D-1 The likelihood of a is the probability that the read was sampled from a distribution with these particular parameters. This can be simplified as Equation 1 as follows: Formula 1:

[0026]

number

[0027] The test statistic can remain the same if the likelihood value is calculated by pooling together genotype counts across multiple reads: for each read, the probability of each possible genotype likelihood is estimated.

[0028] In various embodiments of the present invention, the resamples are generated using a bootstrap method. For each bootstrap iteration, FASTQ X N where X is a FASTQ file with a total number of reads, N, and a portion of the reads is randomly selected to compute χ 1n and sampled with replacement (Figure 2B). This process is repeated I times. In the next iteration of the resampling procedure, χ n are selected by increasing the number of reads.

[0029] The best Z i (i.e., Z, where Z is the best set of resamples obtained from the bootstrap), where Self-contradiction is defined using Equation 2. Formula 2:

[0030]

number

[0031] The self-consistency average is then taken using Equation 3. Formula 3:

[0032]

number

[0033] Non-representativeness is defined using Equation 4. Formula 4:

[0034]

number

[0035] The average of the non-representatives is taken using Equation 5. Formula 5:

[0036]

number

[0037] It should be noted that in Equation 1, N represents the sample size (the number of reads in the original file), n represents the resample size (the number of reads in the resample), and θ represents the Poisson ratio. In Equations 2-5, Nr represents the number of resamples, N indicates the number of sequences / reads in each resample, K represents the size of the k-mer, M represents the total number of samples, and D ri stands for non-representativeness, and D s It should be noted that represents a self-consistency. In various embodiments of the invention, a set of resamples is selected for further processing by the analysis engine 146. In various embodiments of the invention, the input to the analysis engine 146 is a set of resamples (e.g., 100 resamples each containing 1 million reads (i.e., a selected set of 100 resamples, each containing 1 million reads)). The analysis engine 146 then processes the 100 resamples to obtain the best resample that truly represents the original file. This is done by minimizing the steps in the formula as described below.

[0038] To obtain the best resample or set of resamples (i.e., the optimal resamples) (the best resample or set of resamples are those resamples with the lowest values ​​of non-representativeness and self-consistency), the location information and genotype likelihood are combined and analyzed as follows:

[0039] Q θ (Z|X) where Z is the best set of resamples obtained from the bootstrap and z i …z n is each resample with m reads, and X is the entire FASTQ file), and P θ Given (X|[z,y]), where y is the constraint obtained from the alignment, including the position information of the reads and their corresponding genotype information, we consider one or more constraints and find the q θD means [z,y]~X such that ([z,y]|X) gives the best [z,y]. KL [q θ ([z,y]|X)P θ In various embodiments of the invention, the best [z,y] are passed through a decoder 144, which produces a downsampled FASTQ file with a reduced number of reads, but with the retained reads maintaining the genomic integrity of the input file.

[0040] In other embodiments of the present invention, the downsampling component 140 may further use one or more constraints to identify an optimal set of resamples (i.e., best Z). One or more constraints that may be utilized by the downsampling component 140 to identify an optimal set of resamples may include: (i) selecting the largest interval based on the position information encoded from the mapping, (ii) calculating the distribution of intervals within each resample such that position coverage is maximized, (iii) finding the least overlapping resamples with the largest coverage distribution, (iv) calculating an average read weight based on the genotype likelihood occurring for each read, or calculating a total weight for the resamples taking into account the weight of each read, or a combination thereof.

[0041] FIG. 2A illustrates a block diagram showing a downsampling component 140 in communication with a computing device 110 or a server computer 120, or both, in the distributed data processing environment 100 of FIG. 1 for downsampling DNA sequence data using a variational autoencoder. FIG. 2A provides one example of an overview of steps followed to downsample FASTQ files using a modified variational autoencoder. FIG. 2A provides an illustration of one implementation and is not intended to suggest any limitation with respect to the environments in which various embodiments may be implemented. Many modifications to the illustrated environment may be made by one skilled in the art without departing from the scope of the invention as set forth in the claims.

[0042] In step 201, downsampling component 140 receives and / or retrieves DNA sequence data. In various embodiments of the invention, downsampling component 140 may receive and / or retrieve DNA sequence data from a database or as input from a user or another program.

[0043] In step 202, the downsampling component 140 bootstraps the DNA sequence data. In various embodiments of the invention, the downsampling component 140 bootstraps the DNA sequence data received and / or retrieved via the encoder 142. In various embodiments of the invention, the downsampling component 140 performs stochastic resampling using a modified variational encoder (e.g., the encoder 142) to obtain a set of resamples including multiple reads, as shown in FIG. 2B. The selected resamples may be self-consistent and representative of the original file, where self-consistency refers to the minimal variation occurring between resamples of the same size for a given number of resampling iterations, and representativeness refers to the minimal variation of the resamples relative to the original file. In various embodiments of the invention, the downsampling component 140 uses Equations 2-5 (described above) to ensure that the integrity of the genome is maintained. To obtain a set of self-consistent resamples, the downsampling component 140 compares the similarity of the distributions of the samples, using the encoder's hash table to perform a distribution operation.

[0044] For example, FIG. 2B provides an example of a schematic of the bootstrap procedure. In FIG. 2B, θ kSample 1, denoted as (X1), is subjected to random sampling 250 to produce bootstrap samples of quantity “n”, where “n” is a predetermined quantity. In this particular example, the encoder 142 calculates θ k (X 11 ) bootstrap sample 251, θ k (X 12 ) and θ k (X 1NR ), outputting 253 bootstrap samples.

[0045] In step 204, the downsampling component 140 generates biological information. In various embodiments of the invention, the downsampling component 140 generates biological information as further described in FIG. The biological information may include location information, genotype likelihood, or any other biological information known in the art, or a combination thereof.

[0046] In step 206, the downsampling component 140 generates a file containing the location information and the genotype information. In various embodiments of the invention, the downsampling component 140 generates a file containing the location information and the genotype information, as further described in FIG. 2C.

[0047] In step 208, the downsampling component 140 integrates the mapping position information and the genotype likelihood. In various embodiments, the downsampling component 140, via the analysis engine 146, may integrate the mapping position information and the genotype likelihood to identify an optimal vector representation of the resample. The analysis engine 146 calculates an interval distribution for the reads in each resample such that nucleotide position coverage is maximized, and finds the reads in the least overlapping resample with the greatest nucleotide coverage distribution for a given sample genome. The analysis engine 146 may also calculate an average read weight based on the genotype likelihoods occurring in each read and the total weight of the resample based on the weights of the reads.

[0048] In step 210, downsampling component 140 can identify a best Z. In various embodiments of the invention, downsampling component 140, via analysis engine 146, can identify the best Z. In various embodiments, downsampling component 140, via analysis engine 146, can calculate the best Z and display a list of the best Zs to a user.

[0049] At step 212, the downsampling component 140 decodes the vector representation of the optimal resample. In various embodiments of the invention, the downsampling component 140, via the decoder 144, may decode the vector representation of the optimal resample to obtain a downsampled reads file that resembles the original file and maintains the genomic integrity of the original file.

[0050] FIG. 2C illustrates a block diagram showing a downsampling component 140 in communication with a computing device 110 or a server computer 120, or both, in the distributed data processing environment of FIG. 1 for downsampling DNA sequence data using a variational autoencoder. FIG. 2C provides an overview of the generation of biological information, in which read fragments are mapped and the positional information and genotype likelihoods of the read fragments are encoded into a numerical representation. FIG. 2C provides an illustration of one implementation and is not intended to suggest limitations with respect to the environment in which various embodiments may be implemented. Many modifications to the illustrated environment may be made by one skilled in the art without departing from the scope of the invention as set forth by the claims.

[0051] In step 204, the downsampling component 140 generates the biological information. In various embodiments of the invention, the downsampling component 140 enables resampling of DNA fragments generated from massively parallel sequencing without compromising the genomic integrity of the original file.

[0052] In step 204A, the downsampling component 140 aligns the nucleotides. In various embodiments of the invention, the downsampling component 140 utilizes a modified variational encoder (e.g., encoder 142) that aligns multiple reads containing nucleotides to different locations on the sample genome.

[0053] In step 204B, the downsampling component 140 generates position information of the aligned reads. In various embodiments of the present invention, the downsampling component 140 may generate and / or identify position information of nucleotides in the aligned reads via the encoder 142, where the position information includes the mapping positions, mapping scores, and nucleotide quality scores of the mapping reads. In various embodiments of the present invention, the downsampling component 140 may encode the mapping positions, mapping scores, and nucleotide quality scores of the mapping reads to a particular position in the sample genome into a vector representation via the encoder 142. For example, a FASTQ file containing millions of reads is mapped to a reference genome using the Smith-Waterman algorithm known in the art. The Smith-Waterman algorithm performs local sequence alignment to determine similar regions between two nucleic acid sequences. The mapping positions of each read are then encoded into a matrix (matrix 220) as start and end positions. In matrix 220, the matrix structure comprises start and end positions of reads spaced apart by a data structure [n1, and n2] successively until biological data or multiple reads or both are represented, where n may be any positional, integer, or numeric representation known in the art, or a combination thereof.

[0054] In step 204C, the downsampling component 140 obtains genotype likelihoods for one or more genomic locations from the multiple reads, where the genotype likelihoods are for a plurality of alleles having a specified positional change on the multiple reads. In various embodiments of the invention, the encoder 142 is modified to obtain the genotype likelihoods for each genomic location from the multiple reads using a Poisson graph distribution model (Equation 1). In various embodiments of the invention, the downsampling component 140, via the encoder 142, determines that the genotype likelihoods are consistent across the multiple reads (where the genotype likelihoods give the probability that an allele occurs at a nucleotide position) and discards low genotype likelihood probabilities that are not supported by multiple sequence reads.

[0055] In various embodiments of the invention, a modified encoder (e.g., encoder 142) calculates and determines read weights using multiple genotype likelihoods on a read, downweighting reads with fewer genotype likelihoods, and discarding sequences that do not have an estimated genotype likelihood at any nucleotide position and sequences that do not map to any position on the sample genome to minimize redundancy. In various embodiments of the invention, a modified encoder (e.g., encoder 142) may be used to build a representation from the genotype likelihoods. The representation captures subtle differences in the genome in the original file such that variations in the genome in the original file are maintained in the downsampled representation.

[0056] To calculate the genotype likelihood for each read, the reference genome positions are not considered independently, but together to ensure that genotype accumulation is taken into account at the read level. This allows positional read anomalies to be viewed as dependent, instead of assuming independence. If we assume that the occurrence of overlapping reads at positions where genotype accumulation exists is random, the genotype likelihood can be modeled using PGD, which aggregates the counts of overlapping reads at a position, taking into account the read dependency.

[0057] In addition, PGD treats reads overlapping one genomic location with respect to the reference as an exponential family of multinomial joint distributions. This makes it possible to model the dependency between the counts of reads overlapping one location, assuming an upper bound that is reasonable to implement, especially considering the depth of the reads. The joint likelihood function may estimate the genotype likelihood call as the probability of a change from the reference to an alternative allele. The reads overlapping one genotype location are encoded in a matrix (e.g., matrix 222) that is used to train a model to estimate the probability of the genotype likelihood. The estimated genotype likelihood probability is also used to weight the reads. Thus, many of the reads that ideally accumulated more genotype likelihood can be found within the same resample, ensuring that the genomic integrity of the original file is maintained. Resamples that accumulated a higher genotype likelihood than the average are weighted upwards to increase their chances of being selected.

[0058] Figure 3 illustrates operational steps of a downsampling component 140 in communication with a computing device 110 in the distributed data processing environment 100 of Figure 1 for automatically downsampling DNA sequence data using a variational autoencoder, generally designated 300, in accordance with one embodiment of the present invention. Figure 3 provides an illustration of one implementation and is not intended to suggest any limitation with regard to the environments in which various embodiments may be implemented. Many modifications to the illustrated environment may be made by one skilled in the art without departing from the scope of the invention as set forth in the claims.

[0059] In step 302, the downsampling component 140 retrieves genomic sequence data from a file. In various embodiments of the invention, the downsampling component 140 may retrieve and / or receive genomic sequence data from one or more files, one or more databases, one or more search engines, or manually entered data, or a combination thereof.

[0060] In step 304, the downsampling component 140 performs bootstrapping to generate resamples. In various embodiments of the invention, the downsampling component 140 may perform bootstrapping on the genomic sequence data received and / or retrieved via the encoder 142 to generate resamples. In various embodiments of the invention, the downsampling component 140 may perform stochastic resampling via a modified variational encoder (e.g., the encoder 142) to obtain a set of resamples including multiple reads.

[0061] In step 306, the downsampling component 140 resamples the DNA fragments. In various embodiments of the invention, the downsampling component 140 may resample the DNA fragments generated from the parallel sequencing without compromising the genomic integrity of the original file.

[0062] In step 308, the downsampling component 140 generates biological information. In various embodiments of the present invention, the downsampling component 140 may generate biological information via an encoder 142, where the read fragments are mapped and the positional information and genotype likelihoods of the read fragments are encoded into a numerical representation, as shown in FIG.

[0063] In step 310, the downsampling component 140 generates a file containing the location information and the genotype information. In various embodiments of the invention, the downsampling component 140, via the encoder 142, may generate one or more files containing the location information and the genotype information.

[0064] In step 312, the downsampling component 140 accesses the non-representativeness and self-consistency of the resulting resamples. In various embodiments of the present invention, the downsampling component 140 may access one or more non-representativeness and self-consistency of the generated resamples via the encoder 142 and select one or more representative resamples.

[0065] At step 314, downsampling component 140 constructs representations from the genotype likelihoods. In various embodiments of the invention, downsampling component 140 may construct one or more vector representations from the genotype likelihoods via encoder 142 (e.g., a modified encoder).

[0066] In step 316, the downsampling component 140 combines the mapping position information and the genotype likelihoods. In various embodiments of the invention, the downsampling component 140, via the analysis engine 146, may combine the mapping position information and the genotype likelihoods to identify one or more optimal vector representations of one or more resamples.

[0067] In step 318, the downsampling component 140 decodes the vector representation of the resamples. In various embodiments of the invention, the downsampling component 140 may receive the representative samples as input via a decoder 144 and produce a reconstruction of the original input reads. In various embodiments of the invention, the downsampling component 140 may decode the vector representation of the optimal resamples via a decoder 144 (e.g., a modified decoder) to obtain a downsampled reads file that resembles the original file and maintains the genomic integrity of the original file.

[0068] In step 320, the downsampling component 140 outputs downsampled read files. In various embodiments of the invention, the downsampling component 140, via the decoder 144, may output one or more downsampled read files to one or more users, which resemble the original files and maintain the genomic integrity of the original files, by displaying the data in the UI 106 and / or by displaying a response prompt informing the user of the file data. In various embodiments of the invention, the downsampling component 140, via the decoder 144, may output one or more downsampled read files to one or more users, which resemble the original files and maintain the genomic integrity of the original files, by displaying the data in the UI 106 and / or by displaying a response prompt informing the user of the file data.

[0069] Figure 4 illustrates a block diagram of components of a server computer 120 in the distributed data processing environment 100 of Figure 1 in accordance with one embodiment of the present invention. It should be appreciated that Figure 4 is only intended to provide an illustration of one implementation and is not intended to suggest any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

[0070] FIG. 4 illustrates a computer system 400, where the server computer 120 represents an example of a computer system 400 including the downsampling component 140. The computer system includes a processor 401, a cache 403, a memory 402, a persistent storage 405, a communication unit 407, an input / output (I / O) interface 406, a display 409, an external device 408, and a communication fabric 404. The communication fabric 404 provides communication between the cache 403, the memory 402, the persistent storage 405, the communication unit 407, and the input / output (I / O) interface 406. The communication fabric 404 may be implemented using any architecture designed to communicate data and / or control information between the processor (such as a microprocessor, communication, and network processor), system memory, peripheral devices, and any other hardware components in the system. For example, the communication fabric 404 may be implemented using one or more buses or crossbar switches.

[0071] Memory 402 and persistent storage 405 are computer-readable storage media. In this embodiment, memory 402 includes random access memory (RAM). In general, memory 402 may include any suitable volatile or non-volatile computer-readable storage medium. Cache 403 is a high-speed memory that improves the performance of processor 401 by retaining recently and nearly recently accessed data from memory 402.

[0072] Program instructions and data used to practice embodiments of the present invention may be stored in persistent storage 405 and memory 402 for execution by one or more of the respective processors 401 via cache 403. In one embodiment, persistent storage 405 includes a magnetic hard disk drive. As an alternative to, or in addition to, a magnetic hard disk drive, persistent storage 405 may include a solid-state hard drive, a semiconductor storage device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0073] The media used by persistent storage 405 may be removable. For example, a removable hard drive may be used as persistent storage 405. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive to transfer data to another computer-readable storage medium that is also part of persistent storage 405.

[0074] Communications unit 407, in these examples, enables communication with other data processing systems or devices. In these examples, communications unit 407 includes one or more network interface cards. Communications unit 407 may provide communications through the use of either or both physical and wireless communications links. Program instructions and data used to practice embodiments of the present invention may be downloaded to persistent storage 405 via communications unit 407.

[0075] The I / O interface 406 allows for the input and output of data to and from other devices that may be connected to each computer system. For example, the I / O interface 406 may provide a connection to an external device 408, such as a keyboard, a keypad, a touch screen, or any other suitable input device, or a combination thereof. The external device 408 may also include portable computer-readable storage media, such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention may be stored on such portable computer-readable storage media and loaded into the persistent storage 405 via the I / O interface 406. The I / O interface 406 also connects to a display 409.

[0076] Display 409 provides a mechanism for displaying data to a user and may be, for example, a computer monitor.

[0077] The programs described herein are identified based on the applications for which they are implemented in particular embodiments of the invention, but it should be understood that any specific program designations herein are used merely for convenience, and thus the invention should not be limited to use with only any particular application identified and / or implied by such designations.

[0078] The present invention may be a system, a method, and / or a computer program product, which may include a computer readable storage medium (or media) having computer readable program instructions for causing a processor to implement aspects of the present invention.

[0079] A computer readable storage medium may be any tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disk read-only memories (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised structures in grooves in which the instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through a fiber optic cable), or electrical signals transmitted over electrical wires.

[0080] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface within each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage within a computer readable storage medium within the respective computing / processing device.

[0081] The computer readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server, as a stand-alone software package. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry to perform aspects of the invention.

[0082] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0083] These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may be stored in a computer-readable storage medium capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner such that the computer-readable storage medium on which the instructions are stored constitutes an article of manufacture including instructions implementing aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0084] Computer readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device, causing the computer, other programmable apparatus, or other device to perform a series of operational steps to generate a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0085] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of instructions, including one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may possibly be executed in reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special-purpose hardware-based system that performs the specified functions or that operates or implements a combination of special-purpose hardware and computer instructions.

[0086] The description of various embodiments of the present invention is presented for the purpose of explanation, and is not intended to be exhaustive or limited to the disclosed embodiments. Many changes and modifications will be apparent to those skilled in the art without departing from the scope and spirit of the present invention. The terms used in this specification have been selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology found in the market, and to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for automatically downsampling DNA sequence data using a variational autoencoder by computer information processing, while preserving the genomic integrity of the original file, comprising: performing bootstrap on the genome sequence data to generate a resample using an encoder; evaluating, by the encoder, the non-representativeness and self-consistency of the resamples and selecting a representative resample in accordance with said evaluation; constructing, with a modified encoder, a vector representation from the genotype likelihoods based on the selected representative resamples; and combining the mapping position information and the genotype likelihoods with an analysis engine to identify an optimal vector representation for resamples; decoding, with a modified decoder, the identified optimal vector representation of the resample to obtain a downsampled read file that resembles the original file and maintains the genomic integrity of the original file; The method includes:

2. 2. The method of claim 1, further comprising resampling DNA fragments generated from massively parallel sequencing by the encoder without compromising the genomic integrity of the original file.

3. The method of claim 1 , further comprising performing probabilistic resampling with a modified variational encoder to obtain a set of resamples containing a plurality of reads.

4. receiving, by the decoder, the representative resamples as input; generating, by the decoder, a reconstruction of a plurality of original input leads based on the received representative resamples; The method of claim 1 further comprising:

5. 2. The method of claim 1, further comprising outputting the downsampled read file to a user, the downsampled read file being similar to the original file and maintaining the genomic integrity of the original file.

6. The method of claim 1 , further comprising generating, by the encoder, a file containing location information and genotype information.

7. The method further includes generating biological information by the encoder, the generating biological information comprising: aligning a plurality of reads containing nucleotides to different locations on the sample genome using the modified variational encoder; generating, by the encoder, position information of the nucleotides within the aligned reads, the position information including a mapping position of the aligned reads, a mapping score, and a nucleotide quality score; encoding, by the encoder, the mapping positions, the mapping scores, and the nucleotide quality scores of a plurality of reads mapping to a particular position in a sample genome into a vector representation; obtaining, by the encoder, genotype likelihoods for one or more genomic locations from the plurality of reads, the genotype likelihoods being a plurality of alleles having a specified position variation on the plurality of reads; The method of claim 1 , comprising:

8. A computer program for causing a computer to execute the computer-implemented method described in any one of claims 1 to 7.

9. A computer-readable recording medium having the computer program according to claim 8 recorded thereon.

10. 1. A computer system for automatically downsampling DNA sequence data using a variational autoencoder and preserving genomic integrity of the original file, comprising: one or more computer processors; one or more computer readable storage devices; program instructions stored in said one or more computer readable storage devices for execution by at least one of said one or more computer processors; wherein the stored program instructions include: program instructions for performing bootstrap on the genomic sequence data to generate a resample by the encoder; program instructions for evaluating, by the encoder, the resamples for non-representativeness and self-consistency and for selecting a representative resample in accordance with the evaluation; program instructions for constructing, with a modified encoder, a vector representation from the genotype likelihoods based on the selected representative resamples; and program instructions for integrating, with an analysis engine, mapping position information and the genotype likelihoods to identify an optimal vector representation for resamples. and program instructions for decoding, with a modified decoder, the identified optimal vector representation of the resample to obtain a downsampled reads file that resembles the original file and maintains the genomic integrity of the original file.

1. A computer system comprising:

11. 11. The computer system of claim 10, further comprising program instructions for resampling DNA fragments generated by the encoder from massively parallel sequencing without compromising the genomic integrity of the original file.

12. and program instructions for performing stochastic resampling with the modified variational encoder to obtain a set of resamples including a plurality of reads. The computer system of claim 10 further comprising:

13. program instructions for receiving, by the decoder, the representative resamples as input; program instructions for generating, by the decoder, a reconstruction of a plurality of original input leads based on the received representative resamples; The computer system of claim 10 further comprising:

14. program instructions for outputting to a user the downsampled reads file, which is similar to the original file and maintains the genomic integrity of the original file; program instructions for generating a file containing location information and genotype information by the encoder; The computer system of claim 10 further comprising:

15. The method further includes program instructions for generating biological information by the encoder, the generating the biological information comprising: program instructions for aligning a plurality of reads, each read comprising a nucleotide, to different locations on a sample genome using a modified variational encoder; program instructions for generating, by the encoder, position information of the nucleotides within the aligned reads, the position information comprising a mapping position of a mapping read, a mapping score, and a nucleotide quality score; and program instructions for encoding, by the encoder, the mapping positions, the mapping scores, and the nucleotide quality scores of a plurality of reads mapping to a particular position in a sample genome into a vector representation. program instructions for obtaining, by the encoder, a genotype likelihood for one or more genomic locations from the plurality of reads, the genotype likelihood being a plurality of alleles having a specified position variation on the plurality of reads; 11. The computer system of claim 10, comprising:

Images