Quality score compression

Abstract

This invention provides a method, system, and computer program for compressing nucleic acid sequence data. [Solution] The method may include: obtaining nucleic acid sequence data representing (i) a read sequence and (ii) a plurality of quality scores; determining whether the read sequence contains at least one N base; and generating a first coded dataset by encoding each of the quality scores of the read sequence using a (x minus 1) number based on the determination that the read sequence does not contain at least one N base, using a first coding process, wherein x is an integer representing the number of different quality scores used by the nucleic acid sequencing device; and encoding the first coded dataset using a second coding process, thereby compressing the data to be compressed.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application claims the interests of U.S. Patent Application No. 63 / 110,308, filed on November 5, 2020, the entire contents of which are incorporated herein by reference. (Technical field) This invention relates to quality score compression. [Background technology]

[0002] In some cases, genome sequencing describes a method for identifying nucleotides or other component parts of genomic data. Using computers, one or more sets of genomic data can be analyzed to correlate a collection of component parts, such as nucleotides, with their respective locations within a given reference genome. In this way, computers can "map" collections of molecular markers onto the reference genome. [Overview of the project] [Means for solving the problem]

[0003] Generally, this disclosure relates to methods, systems, and computer programs for compressing quality scores generated by a sequencing engine based on genomic data. In one implementation, quality scores generated by a sequencing engine based on genomic data can first be compressed by grouping one or more quality scores into a single data item within a sequence of data items representing a set of quality scores. The sequences of data items can then be further compressed or encoded into a final compressed form.

[0004] According to one innovative aspect of the present disclosure, a method for compressing nucleic acid sequence data is disclosed. In one aspect, the method may include: obtaining nucleic acid sequence data by one or more computers, which includes (i) a read sequence containing data corresponding to a plurality of base calls generated by a nucleic acid sequencing device, and (ii) a plurality of quality scores, each of which quality score indicates the likelihood that a particular base call of the read sequence was correctly generated by the nucleic acid sequencing device; determining by one or more computers whether the read sequence contains at least one "N" base; generating a first coded dataset by one or more computers, based on the determination that the read sequence does not contain at least one "N" base, by using a first coding process to code each of the quality scores of the read sequence using a number in base (x minus 1), where x is an integer representing the number of different quality scores used by the nucleic acid sequencing device; and coding the first coded dataset by one or more computers using a second coding process to compress the data to be compressed.

[0005] Other versions include corresponding systems, devices, and computer programs for performing actions in a manner defined by instructions encoded on a computer-readable storage device.

[0006] These and other versions may optionally include one or more of the following features. For example, in some implementations, x is equal to 3.

[0007] In some implementations, the first coding process may include one or more computers coding each set of five quality scores from a set of multiple quality scores in a read array into a single byte, by representing each quality score in the set of five quality scores as a ternary number.

[0008] In some implementations, the method may further include generating a second coded dataset by encoding each of four quality scores of the read sequence into a single byte in memory using a third coding process by one or more computers, based on the determination that the read sequence contains at least one "N" base, and coding the second coded data using a fourth coding process by one or more computers.

[0009] In some implementations, the second and fourth coding processes are the same.

[0010] In some implementations, the acquired data may include a FASTQ file.

[0011] In some implementations, the first coded dataset is a compressed version of multiple quality scores.

[0012] In some implementations, the second coding process is a compression process.

[0013] In some implementations, the compression process includes an implementation of prediction by partial matching of a range encoder (PPMD).

[0014] In some implementations, for a given value in the first coded dataset, the given value is compressed according to a 4-bit context with respect to the position of the given value in the first coded dataset.

[0015] Another innovative aspect of this disclosure discloses another method for compressing nucleic acid sequence data. In one aspect, the method involves one or more computers obtaining (i) a read sequence containing data corresponding to a plurality of base calls generated by a nucleic acid sequencing device, and (ii) a plurality of quality scores, each of which quality score represents the likelihood that a particular base call of the read sequence was correctly generated by the nucleic acid sequencing device; determining, by one or more computers, the frequency of occurrence of each quality score group in the plurality of quality scores, where each quality score group contains a subset of quality scores from the plurality of quality scores; and for each particular quality score in a first subset of the plurality of quality scores, by one or more computers, the quality This may include determining that a core is a member of a specific quality score group having an occurrence frequency that satisfies a predetermined threshold, generating first data to be used as a single entry in a reduced sequence using a predetermined group mapping, based on the determination that a quality score is a member of a specific quality score group having an occurrence frequency that satisfies a predetermined threshold, by one or more computers, and using a predetermined group mapping, wherein the first data to be used as a single entry in the reduced sequence represents a specific quality score group, and generating a reduced sequence by aggregating the first data generated for each of the specific quality score groups by one or more computers.

[0016] Other versions include corresponding systems, devices, and computer programs for performing actions in a manner defined by instructions encoded on a computer-readable storage device.

[0017] These and other versions may optionally include one or more of the following features. For example, in some implementations, the retrieved data includes a FASTQ file.

[0018] In some implementation forms, each quality score among a plurality of quality scores is data representing the ASCII value of the quality score.

[0019] In some implementation forms, the method includes, for each specific quality score in a second subset of a plurality of quality scores, determining by one or more computers that a specific quality score in the second subset of the array of quality scores is not a member of a specific quality score group having an occurrence frequency satisfying a predetermined threshold, and generating by one or more computers and using a predetermined single mapping, second data to be used as a single entry in a reduced array, wherein the second data to be used as a single entry in the reduced array represents a quality score that is not a member of a specific quality score group having an occurrence frequency satisfying a predetermined threshold, and the predetermined single mapping defines a one-to-one mapping between each of the plurality of single quality scores and a corresponding single entry.

[0020] In some implementation forms, generating a reduced array by one or more computers can include aggregating first data generated for each of the specific quality score groups by one or more computers and aggregating second data generated for each quality score that is not a member of a specific quality score group having an occurrence frequency satisfying a predetermined threshold by one or more computers.

[0021] In some implementation forms, the method can further include identifying, by one or more computers, a plurality of quality score groups among the plurality of quality scores.

[0022] In some implementation forms, a predetermined group mapping defines a one-to-one mapping between each of a plurality of different quality score groups and a corresponding single entry.

[0023] According to another innovative aspect of the present disclosure, another method for compressing nucleic acid sequence data is disclosed. In one aspect, the method comprises, by one or more computers, (i) a read sequence comprising data corresponding to a plurality of base calls generated by a nucleic acid sequencing device, and (ii) a plurality of quality scores, wherein each quality score of the plurality of quality scores indicates the likelihood that a particular base call of the read sequence was correctly identified by the nucleic acid sequencing device, obtaining nucleic acid sequence data representing the same; determining, by one or more computers, the occurrence frequency of each quality score group in the plurality of quality scores, wherein each quality score group comprises a subset of the quality scores of the plurality of quality scores; for each particular quality score within a first subset of the plurality of quality scores, determining, by one or more computers, that the particular quality score in the first subset of the plurality of quality scores is not a member of a particular quality score group having an occurrence frequency that meets a predetermined threshold, and generating, by one or more computers and using a predetermined single mapping, first data to be used as a single entry in a reduced sequence, wherein the first data to be used as a single entry in the reduced sequence represents quality scores that are not members of a particular quality score group having an occurrence frequency that meets a predetermined threshold; and generating, by one or more computers, a reduced sequence by aggregating the first data generated for each of the quality scores that are not members of a particular quality score group having an occurrence frequency that meets a predetermined threshold.

[0024] Other versions include corresponding systems, devices, and computer programs for performing the actions of the methods defined by instructions encoded on a computer-readable storage device.

[0025] These and other versions may optionally include one or more of the following features. For example, in some implementations, the obtained data includes a FASTQ file.

[0026] In some implementations, each quality score in a set of multiple quality scores is data representing the ASCII value of the quality score.

[0027] In some implementations, the method may further include: for each particular quality score in a second subset of multiple quality scores, one or more computers determining that the quality score is a member of a particular quality score group having an occurrence frequency that satisfies a predetermined threshold; and generating second data to be used as a single entry in a reduced array, using a predetermined group mapping, based on the determination that the quality score is a member of a particular quality score group having an occurrence frequency that satisfies a predetermined threshold, using one or more computers. The second data to be used as a single entry in a reduced array represents a particular quality score group, and the predetermined group mapping defines a one-to-one mapping between each of a plurality of different quality score groups and the corresponding single entry.

[0028] In some implementations, generating a reduced array by one or more computers may include aggregating first data generated by one or more computers for each quality score that is not a member of a particular quality score group having an occurrence frequency that satisfies a predetermined threshold, and aggregating second data generated by one or more computers for each of the particular quality score groups.

[0029] In some implementations, the method may further include one or more computers identifying multiple quality score groups across multiple quality scores.

[0030] In some implementations, a given single mapping defines a one-to-one mapping between each of several single quality scores and the corresponding single entry.

[0031] These and other innovative aspects of the present disclosure are described below herein with reference to the modes for carrying out the invention, drawings, and appended claims. [Brief explanation of the drawing]

[0032] [Figure 1] This figure shows one embodiment of a system for encoding, for example, compressing, an array of quality scores having a first data format. [Figure 2] This flowchart illustrates one embodiment of a process for encoding, for example, a compression process for an array of quality scores having a first data format. [Figure 3] This figure shows one embodiment of a system for encoding, for example, compressing, an array of quality scores having a second data format. [Figure 4] This flowchart illustrates one example of a process for encoding, for example, a compression process for an array of quality scores having a second data format. [Figure 5] This flowchart illustrates one embodiment of the process for reconstructing an array of quality scores having a first data format. [Figure 6] This flowchart illustrates one embodiment of the process for reconstructing an array of quality scores having a second data format. [Figure 7] This flowchart illustrates one example of a process for determining a method for compressing quality scores. [Figure 8] This is a graphical representation of experimental results for a process to encode an array of quality scores having a first data format. [Figure 9] This is a graphical representation of experimental results for a process to encode an array of quality scores having a second data format. [Figure 10] This is a diagram of a computer system component that can be used to implement a process for encoding an array of quality scores having a first data format. [Modes for carrying out the invention]

[0033] Similar reference numbers and names in various drawings refer to the same elements. This disclosure relates to a method, system, and computer program for compressing data representing a sequence of quality scores of a read sequence generated by a nucleic acid sequencing device. Each quality score in the sequence of quality scores provides an indicator of the likelihood that the corresponding base in the read sequence was correctly sequenced by the nucleic acid sequencing device. The methods and systems disclosed herein enable faster compression speeds and lower compression ratios compared to conventional methods that do not utilize the techniques described herein. The faster compression speeds and lower compression ratios are achieved by performing a pre-compression coding step to reduce the size of the input data representing the sequence of quality scores that is processed by the compression engine. Since the compression engine receives and processes the reduced-size input representing the sequence of quality scores, the compression engine can compress the input data faster and achieve a smaller compressed file size compared to conventional methods. Thus, the compression method of this disclosure can achieve a higher compression ratio than conventional systems, where the compression ratio is equal to the uncompressed file size divided by the compressed file size.

[0034] Generally, this disclosure describes systems and methods for performing an initial coding step on data representing a quality score array before inputting the data representing the quality score array into a compression engine. The advantages of this approach can be described with respect to specific examples. In one or more first implementations, each quality score in the quality score array may be represented as an 8-bit (or one-byte) ASCII value. In such first implementations described herein, this disclosure can perform initial coding on such 8-bit representations of quality scores in the quality score array, reducing the 8-bit representations to 2-bit or 1.6-bit representations of quality scores, and thus enabling the coding of four or five quality scores, respectively, into a single byte. Thus, such an initial coding step can significantly and predictably reduce the input data size to the compression engine by one-quarter or one-fifth of the size of the initial representation of the quality scores.

[0035] However, this disclosure is not limited to reducing an 8-bit representation of a quality score to either a 2-bit representation or a 1.6-bit representation of a quality score for input to a compression engine. Rather, similar ratio reductions can be achieved with other sizes of representations of the quality score. These embodiments are provided, in part, to highlight the technical improvements achieved by this disclosure.

[0036] Furthermore, other second implementations of the present disclosure describe other initial-stage coding engines that perform operations on data representing an array of quality scores to generate a reduced array set for input to a compression engine. Such second implementations offer similar technical benefits to the implementations described above (e.g., faster compression speeds and higher compression ratios compared to conventional methods). However, these second implementations, which still result in faster compression speeds, lower compressed file sizes, and higher compression ratios than conventional methods due to the initial size reduction of the input data before input to the compression engine, which is variable and involves spontaneous grouping or degrouping of quality scores, may ultimately have faster speeds and compression ratios that are less predictable than the first implementations of the present disclosure described above, which may have defined data sizes at each stage.

[0037] In general, as used herein, the term “encoding” refers to a process performed by one or more software engines, one or more hardware engines (e.g., processors), or a combination thereof, which includes receiving a first set of data, processing the first set of data, and generating a second set of data that represents a different form of the first set of data. In some embodiments, the second set of data may be stored in less memory than the first set of data received. For example, one form of encoded data may include compressing the data to a size smaller than the size of the data before compression, for example, using a compression engine.

[0038] Figure 1 shows an embodiment of system 100 for compressing a sequence of quality scores having a first data format. The first data format can assign "X" different quality scores to corresponding bases in a read sequence, where "X" is an arbitrary positive integer less than a given threshold. The given threshold can be determined based on the number of unique quality scores produced using system 100 shown in Figure 1, which is more effective or practical than other systems, such as system 300 shown in Figure 3. For example, the given threshold could be 8. If "X" is less than 8, the corresponding quality score can be processed by system 100. In a particular embodiment, in some implementations, "X" may be equal to 4, indicating that the first data format can use any one of four different quality scores to show a correct likelihood for one or more base calls in a read sequence generated by a nucleic acid sequencing device. In some implementations, the likelihood can include the probability that a sequencing error occurred in one or more base calls corresponding to a quality score. In some embodiments, the sequencing error can include base calls produced by the nucleic acid sequencing device for a specific location in the read sequence that is incorrect. For example, a sequencing device might determine the base call for adenine, represented by the letter A, at a particular location in the read sequence, even though the correct base call should actually be cytosine, represented by the letter C. A low quality score for a given base call may indicate that such an error is more likely, while a high quality score may indicate that such an error is less likely. Base calls can include data generated by the nucleic acid sequencing device representing a specific nucleotide in the read sequence.

[0039] System 100 is configured to receive input data 102 from one or more data sources. In some implementations, one or more data sources may include nucleic acid sequencing devices. These nucleic acid sequencing devices may be next-generation sequencing devices such as Novaseq® 6000, Nextseq® 2000, etc. In other implementations, one or more data sources may include one or more processors running on computing devices such as tablet computers, desktop computers, one or more server computers, or a combination thereof. In some implementations, input data 102 may be received from one or more data sources via one or more networks. One or more networks may include wired Ethernet networks, wired optical networks, wireless networks, LANs, WANs, Wi-Fi networks, cellular networks, the Internet, or any combination thereof. In some implementations, input data 102 may be received from one or more data sources via direct connections such as USB cable connections or USB-C cable connections. In yet another implementation, the entire system 100 may be hosted within one or more data sources. For example, in some implementations, the entire system 100 may be hosted by a nucleic acid sequencing device.

[0040] System 100 may include an input engine that receives input data 102. Input data 102 may include multiple records, each containing data corresponding to a series of base calls, and data describing a read sequence, each containing data describing a quality score of a series of nucleotides or bases. Base calls may include data generated by a nucleic acid sequencer device that corresponds to or represents nucleotides of a DNA fragment sequenced by the nucleic acid sequencer device. However, for the purposes of this disclosure, the terms base call and nucleotide may be used interchangeably throughout to refer to data generated by a nucleic acid sequencer that corresponds to letters such as A, C, T, or G in the read sequence. The meanings of such letters are described in more detail below.

[0041] In each record, each quality score in the quality score sequence may correspond to a specific nucleotide or base in the read sequence. For example, in the embodiment of Figure 1, the first quality score "F" in the quality score sequence "F#FFFF...F;FF" corresponds to the first nucleotide or base in the read sequence "CNTGTA...ATAAG". In some implementations, the input data 102 may include one or more FASTQ files, and each record of multiple records may include a portion of a FASTQ file, referred to herein as a FASTQ record. Each portion of a FASTQ file may include one read sequence and the corresponding sequence of quality scores for the read sequence.

[0042] Each read sequence may be pre-generated by one or more nucleic acid sequencing devices from sequencing of a biological sample by one or more nucleic acid sequencers. The biological sample may include nucleic acid samples from any organism, such as humans, animals, or plants. Each read sequence contains a string of letters from a defined vocabulary. For example, the smallest vocabulary can be represented by a set of five symbols: {A, C, G, T, N}. The letters A, C, G, and T represent the four types of nucleotides present in deoxyribonucleic acid (DNA), namely adenine, cytosine, guanine, and thymine. In ribonucleic acid (RNA), thymine is replaced by uracil (U). The letter "N" may be used by a nucleic acid sequencing device to indicate that the sequencing device was unable to determine any base at a particular location in the read sequence occupied by "N," and as a result, the actual and precise nature of the position in the read sequence is not determined. The use of the letters A, C, G, and T or U is common because these letters represent the first letter of each nucleotide. However, this disclosure is not limited to the use of the letter "N" to represent undetermined locations within the generated read sequence. Alternatively, a nucleic acid sequencing device may use any letter or symbol to represent locations within the read sequence that the nucleic acid sequencing device cannot accurately determine as the correct base. In the implementations described herein, it is understood that any letter or symbol used to represent an undetermined base is equivalent to the use of the letter "N".

[0043] Storing quality scores of bases in a read sequence can have many useful applications. However, given the sequenced genomes of organisms such as humans, this can include more than 3 billion bases and corresponding more than 3 billion corresponding quality scores. The data corresponding to the resulting set of quality scores can be very large (e.g., several gigabytes to several terabytes, depending on the sequencing depth), and compression may be required to efficiently store, transmit, or archive the quality score information. An aspect of the present disclosure, illustrated with reference to Figure 1, provides a multi-stage compression method that relies on an initial coding engine for preparing an input dataset for a compression engine. By initially coding the data based on the inherent characteristics of the input data 102 before use in later stages of the compression engine, the system 100 can achieve a high compression ratio while maintaining a fast compression rate. The resulting compressed quality scores can reduce the memory footprint, which can reduce the operational cost of associated memory storage, as well as increase the processing speed when accessing or analyzing the resulting compressed quality scores. For the purposes of the present disclosure, the “engine” may include one or more software modules, one or more hardware modules, or any combination thereof.

[0044] In stage A, the classifier engine 104 can acquire input data 102 and transfer a target or specific portion of the input data 102 to different initial quality score coding engines based on the attributes of that specific portion of the input data 102. In the context of this disclosure, the process of acquiring refers to the process of receiving, retrieving, or otherwise obtaining. In some implementations, the classifier engine 104 can function as a decision engine that determines whether each specific portion of the input data 102 should be provided to an initial quality score coding engine V1 106 or an initial quality score coding engine v2 118. After performing their respective initial coding operations on the portions of the input data 102 they receive, each of the initial coding engines V1 and V2 can provide their respective coded outputs 114 and 126 as input to the compression engine 116. The compression engine can process the coded outputs 114 and 126, which it receives as input and can produce a final output 128, which is a compressed version of the input data 102.

[0045] More specifically in stage A of Figure 1, the input data 102 acquired by the classifier engine 104 may include a first record 102a, a second record 102b, and an i-th record 102c. In the embodiment of Figure 1, the first record 102a, the second record 102b, and the i-th record 102c may be parts of a FASTQ file, also referred to herein as FASTQ records. A FASTQ record may include a record header (e.g., "@A0:90:H46:1...") that identifies the FASTQ file from which the FASTQ record originates and distinguishes the FASTQ record from other FASTQ records. A FASTQ record may include data representing a read sequence generated by a nucleic acid sequencer. A FASTQ record may also include a sequence of quality scores corresponding to the sequence of the FASTQ record. A FASTQ record may further include one or more delimiters to separate one or more data components, e.g., a record header, a quality score sequence, a sequence, etc.

[0046] In the embodiment shown in Figure 1, the input data 102 is represented by three data records, but the input data 102 may contain any number of records. These three data records are shown in the later processes in Figure 1, but any number of records can be processed in a similar manner. In step B, the classifier engine 104 can determine the classification of the first record 102a, the second record 102b, and the i-th record 102c of the input data 102, based on one or more classification rules. In some implementations, the classification may be determined based on the determination of the read sequence within each FASTQ record.

[0047] In some implementations, for example, the classifier engine 104 may classify or transfer each FASTQ record to the initial quality score coding engines V1 and V2 based on whether the read sequence in the FASTQ record contains at least one "N" base. If it is determined that the read sequence of a FASTQ record contains at least one "N" base, the FASTQ record may be transferred to the initial quality score coding engine V1 106. Alternatively, if it is determined that the read sequence of a FASTQ record does not contain at least one "N" base, the FASTQ record may be transferred to the initial quality score coding engine V2 118. However, this is merely one embodiment of the classifier rules, and it is conceivable that other types of classification rules can be used in accordance with this disclosure to classify and transfer FASTQ records between the initial quality score coding engine V1 and the initial quality score coding engine V2. In some implementations, the classifier engine 104 classifies the input data using other elements of the input data. For example, instead of classifying based on the presence or absence of an "N" base, the classifier engine 104 can classify based on the percentage or determined portion of an "N" base or other bases. Furthermore, the classifier engine 104 can classify or transfer one or more FAST Q records or related data using other base calls or corresponding data, such as quality scores.

[0048] In the embodiment shown in Figure 1, the classifier engine 104 can determine in step B, based on the base calls of the respective read sequences of the first FASTQ record 102a and the i-th FASTQ record 102c, that both the first FASTQ record 102a and the i-th FASTQ record 102c contain data representing a read sequence having at least one "N" base. Based on the determination that the first FASTQ record 102a and the i-th FASTQ record 102c each have at least one "N" base, the classifier engine 104 can transfer the first FASTQ record 102a and the i-th FASTQ record 102c to the initial quality score coding engine v1 106. In some implementations, the classifier engine 104 transfers data corresponding to the first FASTQ record 102a and the i-th FASTQ record 102c without, for example, transmitting one or more complete FASTQ records corresponding to the first FASTQ record 102a or the i-th FASTQ record 102c. After transferring the data corresponding to the first FASTQ record 102a and the i-th FASTQ record 102c to the initial quality score coding engine v1 106, the execution of the system 100 can continue to stage C.

[0049] In stage C, the initial quality score coding engine v1 106 can obtain the respective quality score sequences for the first FASTQ record 102a and the i-th FASTQ record 102c, corresponding to the read sequences of the first FASTQ record 102a and the i-th FASTQ record 102c, respectively. In this embodiment, the quality score sequence received by the initial quality score coding engine v1 106 may contain four quality scores, each of which is represented by an 8-bit ASCII value "F", ":", "," and "#", where "#" represents an "N" base quality score. However, in other implementations, there may often be fewer than four quality scores, using other characters or symbols that can be used to represent similar information.

[0050] The initial quality score coding engine v1 106 can perform initial coding on the array of quality scores in the first record 102a at stage C. This initial coding performed by the initial quality score coding engine v1 106 can code each quality score from the array of quality scores in the first FASTQ record 102a. This initial coding may include coding each 8-bit ASCII representation of the quality score "F", ":", "," or "#" into a 2-bit representation of each respective quality score. In some implementations, coding each 8-bit ASCII representation of each quality score in the quality score array of the first FASTQ record 102a may result in the initial quality score coding engine 106 generating output data 114 containing 4 quality scores per byte. This coding ratio generated by the initial quality score coding engine v1 106 reduces the size of the input data record to the compression engine 116 by a quarter. In other implementations, other compression ratios can be achieved by compressing a larger or smaller quality score, or a quality score of a different data size, into one or more bits of information.

[0051] The initial quality score coding engine v1 106 can continue to perform the initial coding process for each FASTQ record transferred to the initial quality score packing engine v1 106. For example, upon receiving the i-th FASTQ record 102c, the initial quality score coding engine v1 106 can code the quality scores from the array of quality scores from 8-bit ASCII values ​​to a 2-bit representation of the quality score. This process can continue until each of the arrays of quality scores received by the initial quality score packing engine v1 has been processed and an initial coding of the 2-bit quality score has been generated.

[0052] More specifically, the initial quality score coding engine v1 106 can perform the coding of 8-bit quality scores into an output binary stream 114 of 2-bit quality scores by performing the processes shown in items 108, 110, and 112. The quality score coding engine v1 106 can obtain four first quality scores from a first record 102a. In some implementations, this may include the initial quality score coding engine v1 106 obtaining the ASCII value for each of the quality scores. In other implementations, this may include obtaining other representations of the quality scores and mapping each quality score to the corresponding ASCII value, as shown in 110. For example, the four first quality scores "F", "#", "F", and "F" may be mapped by the initial quality score coding engine v1 106 to their corresponding ASCII values ​​70, 35, 70, and 70. Table 110 shows the mapping of other quality scores using a first data format having "N". Next, the ASCII representation of the quality score can be mapped to each quality score category, where there is one category corresponding to each quality score category. In this embodiment, the quality score categories are represented by 0, 1, 2, and 3, since there are four possible quality score ranges. However, depending on the quality scoring system in use, there may be more (e.g., 0, 1, 2, 3, 4, 5, etc.) or fewer (e.g., 0, 1) categories. In this embodiment, 0 corresponds to an unidentified base and 3 corresponds to a high base quality score, but the disclosure is not limited to this implementation. Rather, instead of integers, quality scores can be represented using, for example, percentages relative to 100%, language-based score values, e.g., low, medium, and high, or other indicator values ​​known in the art.

[0053] In the embodiment shown in Figure 1, the initial quality score coding engine v1 106 can perform calculations using quality score values ​​3, 0, 3, and 3, corresponding to four ASCII-based quality scores "F", "#", "F", and "F". The initial quality score coding engine v1 106 uses q1+4 * q2+42* q3+4 3* The quality coded score can be calculated using formulas such as q4, where q1, q2, q3, and q4 each represent a quality score value. For quality score values ​​of 3, 0, 3, and 3, the formula is 3+4, which is equal to 243 as shown in item 112. * 0+4 2* 3+4 3* It can be evaluated as 3. The initial quality score coding engine v1 106 can generate corresponding binary representations of these four quality scores by generating a binary representation of the number 243. This binary representation is 11110011, and this binary representation can then be appended to the output binary stream 114. This process can be repeated until each of the quality scores in the array of quality scores of the first FASTQ record 102a has been coded into the output binary stream 114 for the first time.

[0054] After the initial coding of the quality score sequence for the first FASTQ record 102a, the initial quality score coding engine v1 106 can continue the initial coding process for each subsequent FASTQ record received. For example, the initial quality score coding engine v1 106 can continue the initial coding of the i-th FASTQ record 102c in the same manner as described above, referring to the first FASTQ record 102a. The initial quality score coding engine v1 106 can take subsequent quality scores and continue the initial coding in the same manner as the four initial quality scores shown in item 108.

[0055] In stage D, the initial quality score coding engine v1 106 can send the output binary stream 114 to the compression engine 116. The compression engine 116 can then perform further compression on the output binary stream 114 corresponding to a predetermined compression method. In general, any compression process can be used by the compression engine 116 to further compress or reduce the size of the output binary stream 114. For example, in some implementations, the compression engine 116 can perform compression using level 11 of the Zstandard (ZSTD) library. However, this disclosure is not limited thereto. Alternatively, in some implementations, other compression methods may be used, including the ZSTD library and other levels of other compression libraries. In general, any compression method or combination of compression methods known in the art can be used.

[0056] In another example, the classifier engine 104 can determine, by applying one or more classification rules, that an incoming FASTQ record contains a read sequence that does not contain at least one "N". Based on the determination that the incoming FASTQ record does not contain at least one "N", the classifier engine 104 can transfer the data corresponding to the second FASTQ record 102b to the initial quality score coding engine v2 118.

[0057] Since the second FASTQ record 102b does not contain any N, the range of quality score candidates can be the number of quality score candidates minus 1. That is, if the classifier engine 104 determines that the second FASTQ record contains a quality score candidate of "X", then the quality score candidates that can be processed by the initial quality score coding engine v2 118 are "X" - 1. In this implementation, the quality score array has only three distinct quality scores. Therefore, in this embodiment, in order to achieve a higher compression ratio, the three distinct quality scores are partially compressed separately.

[0058] In stage E, the initial quality score coding engine v2 118 can obtain the quality score array from the second record 102b. Item 120 shows a subset of the quality scores from the quality score array of the second record 102b. More specifically, the initial quality score coding engine v2 118 can perform the coding of 8-bit quality scores into an output binary stream 126 of 1.6-bit quality scores by performing the processes shown in items 120, 122, and 124. The initial quality score coding engine v2 118 can obtain a number of quality scores determined from the second FASTQ record 102b, for example, 5 quality scores. In some implementations, this may involve the initial quality score coding engine v2 118 simply obtaining the ASCII value for each quality score. In other implementations, this may involve obtaining a different representation of the quality scores and mapping each respective quality score to the corresponding ASCII value as shown in 122. For example, the first five quality scores ":,", "F,", ":,", ",," and "F," can be mapped by the initial quality score coding engine v2 118 to their corresponding ASCII values ​​58, 70, 58, 44, and 70.

[0059] Next, the ASCII representation of the quality score can be mapped to each quality score category, where there is one category corresponding to each category of the quality score. In this embodiment, the quality score categories are represented as 0, 1, and 2, since there are three possible quality scores (i.e., X possible quality scores - 1, because there are no "N" bases in the FASTQ record transferred to the initial quality score coding engine v2 118). In this embodiment, the initial quality score coding engine v2 118 can encode the 8-bit ASCII quality score into a 1.6-bit quality score by representing the 8-bit ASCII quality score as a ternary number. Here, ternary is used because there are three specific categories of quality scores.

[0060] However, the present disclosure is not limited to the above-described embodiments. Instead, in implementations with more quality score categories, such as a total of eight unique quality score categories, the initial quality score encoding engine v2 118 can obtain an array having seven unique quality score categories where the original eight unique quality score categories include the quality scores of "N" bases. In such implementations, a septenary number can be used to represent the initial encoding of the initial quality score encoding engine v2 118.

[0061] Similar to item 112, item 124 shows the calculations performed by the initial encoding engine 118 associated with the first set of quality scores. In this case, the initial quality score packing engine v2 118 uses an expression such as q1 + 3 * q2 + 3 2* q3 + 3 3* q4 + 3 4* q5, etc. to calculate the quality packing score. For quality score values of 1, 2, 1, 0, and 2, the expression can be evaluated as 1 + 3 * 2 + 3 2* 1 + 3 3* 0 + 3 4* 2, which is equal to 178 as shown in item 124. Then, the binary representation of 178, for example, 10110010, can be added to the output binary stream 126.

[0062] In the embodiment of FIG. 1, the initial quality score packing engine v2 118 can encode five 8-bit quality scores into a single ASCII character of 1 byte, thereby compressing each quality score of the second record 102b to one-fifth. In other implementations, other compression ratios can be achieved by compressing larger or smaller quality scores, or quality scores of different data sizes, into information of 1 bit or more.

[0063] Similar to the initial quality score coding engine v1 106, the initial quality score coding engine v2 118 can acquire subsequent quality scores and continue coding them in a similar manner to the initial five quality scores shown in item 120. In stage F, the initial quality score coding engine v2 118 can provide the output binary stream 126 as input to the compression engine 116. The compression engine 116 can then perform subsequent compression on the output binary stream 126 corresponding to a predetermined compression method. For example, in some cases, the compression engine 116 performs compression based on an implementation of Partial Matching Prediction (PPMD) of a range encoder for compressing a byte string. In some implementations, other compression methods known in the art can be used instead of, or in addition to, the PPMD ​​implementation. In general, the compression engine 116 can use any compression or any combination of compression methods known in the art.

[0064] In some implementations, each symbol in the output binary stream 126 supplied to the range encoder may be calculated according to a 4-bit context, representing the number of highest quality score values ​​in the previous score. For example, the 4-bit context may be calculated as the total number of highest quality scores in 30 previous quality scores divided by 2. This particular 4-bit context expression can yield a value in the range of 0 to 15 that fits the 4-bit context. By using a 4-bit context or other context-based coding approach, system 100 can take into account multiple adjacent instances of a particular quality score, e.g., the highest score "F", which can be a good predictor of the subsequent score. In some implementations, a larger or smaller context is used depending on memory, computation, or other requirements. For example, a larger context may result in a higher compression ratio but may require more memory and computation time.

[0065] In some implementations, other coding engines can be used within system 100. For example, in addition to the initial quality score coding engine v1 106 and the initial quality score coding engine v2 118, the implementation may include a third coding engine based on specific criteria defined by the classifier engine 104. In some implementations, more than three engines may be used. In some implementations, the classifier engine 104 may determine a third set of records in the input data 102 that use only two different quality scores. In this way, system 100 can achieve an even higher compression ratio for groups that have only two different quality scores.

[0066] For the sake of clarity, we will use stages A through G. The processes performed by system 100 may occur in the order shown in stages A through G, although in other implementations, the order of certain stages may differ. In some implementations, two or more stages may occur simultaneously.

[0067] Figure 2 is a flowchart illustrating one embodiment of process 200 for quality score compression based on a first input data format. Process 200 can be carried out by one or more electronic systems, for example, system 100 in Figure 1.

[0068] Process 200 includes obtaining a genetic read sequence generated by a genetic sequencing device (202). For example, as shown in Figure 1, input data 102 is obtained and provided to system 100. The input data 102 may include one or more records, each containing data representing a read sequence with data corresponding to multiple base calls generated by the nucleic acid sequencing device, and data describing the quality scores of the multiple base calls. Each quality score corresponds to a specific base call in the read sequence.

[0069] In some implementations, characters are used to represent quality scores within the input data 102. For example, the first quality score "F" in the quality score sequence "F#FFFF...F;FF" of the first FASTQ record 102a corresponds to the first nucleotide or base (hereinafter referred to as "base") in the read sequence "CNTGTA...ATAAG" of the first FASTQ record 102a. In some implementations, the input data 102 may contain one or more FASTQ files, and each of the multiple records may contain a portion of a FASTQ record, which is referred to herein as a FASTQ record. Each portion of a FASTQ file may contain one read sequence and a corresponding sequence of quality scores for the read sequence.

[0070] Process 200 may include obtaining multiple quality scores corresponding to the genetic read sequences (204). For example, as shown in Figure 1, the input data 102 includes a first record 102a, a second record 102b, and an i-th record 102c. Each of the first record 102a, the second record 102b, and the i-th record 102c includes both the genetic sequence and the quality score sequence corresponding to the genetic sequence. For example, the first record 102a includes the corresponding quality score sequences for the genetic sequences "CNTGTA...ATAAG" and "F#FFFF...F:,FF", where each value in the quality score sequence indicates the likelihood of a sequencing error at a particular location in the corresponding genetic sequence.

[0071] Process 200 includes determining that a genetic reading sequence contains at least one "N" base (206). For example, as shown in Figure 1, the classifier engine 104 obtains input data 102 containing one or more genetic reading sequences. The input data 102 contains a first record 102a. The first record 102a contains the genetic sequence "CNTGTA...ATAAG". The genetic sequence "CNTGTA...ATAAG" contains the base "N". The classifier engine 104 can determine that the genetic sequence "CNTGTA...ATAAG" of the first record 102a contains the base "N", and can transfer the data corresponding to the first record 102a to the initial quality score coding engine v1 106. Similarly, the classifier engine 104 can obtain a second record 102b. The second record 102b contains the genetic sequence "GTCTAG...CACTT" which does not contain the base "N". The classifier engine 104 can determine that the genetic sequence "GTCTAG...CACTT" of the second record 102b does not contain the base "N", and can transfer the data corresponding to the second record 102b to the initial quality score coding engine v2 118.

[0072] Process 200 includes generating a first coded dataset by coding each quality score using a number in base x, where x is an integer number representing the number of different quality scores used by the genetic sequencing device (208). For example, the initial quality score coding engine v1 106 obtains the quality score sequence of the first record 102a. The quality score sequence of the first record 102a contains four unique quality scores "F", ":", "," and "#". Other preferred symbols or values ​​can be used in other implementation forms. The initial quality score coding engine v1 106 can then compute integers based on a number in base 4. For example, as shown in item 108 of Figure 1, the initial quality score coding engine v1 106 generates a value 3033 corresponding to the genetic quality score "F#FF" based on the mapping shown in item 110. The initial quality score coding engine v1 106 then generates an integer based on the value 3033, as if the value 3033 were written in base 4 notation. As shown in item 112, the resulting integer is equal to 243, which can be written as "11110011" with 8 binary bits. The expression used to generate the binary form of a group of quality scores, such as quality score "F#FF", can be constructed so that the integer value can be represented using 8 or fewer bits. For example, the expression used by the initial quality score coding engine v1 106 can be constructed so that the integer value is less than 255.

[0073] In another embodiment, the initial quality score coding engine v2 118 obtains the quality score array of the second record 102b. In contrast to the quality score array of the first record 102a, the quality score array of the second record 102b contains three unique quality scores: "F", ":", and ",". Other suitable symbols or values ​​can be used in other implementations. The initial quality score coding engine v2 118 can then compute an integer based on the ternary number. Because there are few unique quality scores, the initial quality score coding engine v2 118 can encode additional quality scores (e.g., 5 instead of 4) into 8-bit binary. For example, as shown in item 124 of Figure 1, the initial quality score coding engine v2 118 generates a value 12102 corresponding to the genetic quality score ":F:,F" based on the mapping shown in item 122. Next, the initial quality score coding engine v2 118 generates an integer based on the value 12102, as if the value 12102 were written in ternary notation. As shown in item 124, the resulting integer is equal to 178, which can be written as "10110010" with 8 binary bits. The expression used to generate the binary form of a group of quality scores, such as quality score ":F:,F", can be constructed such that the integer value can be represented using 8 or fewer bits. For example, the expression used by the initial quality score coding engine v2 118 can be constructed such that the integer value is less than 255.

[0074] Process 200 includes generating a second coded dataset by coding the first coded dataset using a second coding algorithm (210). For example, as shown in Figure 1, the compression engine 116 generates output 128 based on the input provided by the initial quality score coding engine v1 106 or the initial quality score coding engine v2 118. In some implementations, the compression engine 116 generates output 128 by combining multiple outputs from different compression processes. For example, both the initial quality score coding engine v1 106 and the initial quality score coding engine v2 118 can generate data about the compression engine 116, as shown in coded outputs 114 and 126.

[0075] In some implementations, the compression engine 116 performs one or more types of compression based on the acquired data. For example, the compression process used to compress the output of the initial quality score coding engine v1 106 may differ from the compression process used to compress the output of the initial quality score coding engine v2 118. As discussed herein, the compression of the coded output 114 of the initial quality score coding engine v1 106 may include compression using level 11 of the Z standard (ZSTD) library, or other forms of compression. The compression of the coded output 126 of the initial quality score coding engine v2 118 may include compression using an implementation of prediction by partial matching of a range encoder (PPMD), or other forms of compression. In some implementations, a given array can be compressed using context around a given value in the array. For example, a 4-bit context representing the number of highest quality score values ​​in the previous score may be used by the compression engine 116. In some cases, multiple compression processes can be combined to produce a compressed output.

[0076] Figure 3 shows an embodiment of a system for compressing a sequence of quality scores having a second data format. System 300 includes a quality score sequence 302 formatted based on the Q40 data format. The quality score sequence 302 is processed by a group identification engine 304, a frequency counter engine 310, a reduced sequence generation engine 316, a single mapping engine 320, a group mapping engine 326, and a compression engine 334 to produce a reduced sequence 332 and subsequent output 336 representing a compressed version of the quality score sequence 302.

[0077] In step A of Figure 3, the quality score sequence 302 is generated and transmitted to the group identification engine 304. In the embodiment of Figure 3, the quality score sequence 302 is generated by a sequencer that uses the Q40 data format to encode the quality scores associated with the bases as described above. In general, some sequencers that use more than a threshold number of unique quality scores can compress the resulting output data using the process performed by the system 300 shown in Figure 3 or a similar process. For example, if a sequencer uses more than 8 unique quality scores to encode the quality scores corresponding to the genetic sequence, the sequencer can use the encoding and compression process considered with reference to Figure 3.

[0078] In stage B, the group identification engine 304 obtains the quality score array 302 and generates one or more groups based on the quality score array 302 as shown in item 306. In this implementation, the group identification engine 304 groups adjacent quality scores of three groups. In other implementations, other numbers of quality scores may be included in one or more groups. As shown in item 316, the first group of three quality scores are the characters "@", "C", and "@", which correspond to the characters in the quality score array 302. In the embodiment of Figure 3, each character in the quality score array 302 represents a quality score indicating the likelihood of an array determination error.

[0079] The group identification engine 304 generates quality score groups 308 and sends the quality score groups 308 to the frequency counter engine 310. In stage C, the frequency counter engine 310 retrieves the quality score groups 308 and determines the number of occurrences for each group within the quality score groups 308, as shown in item 312.

[0080] In some implementations, the group identification engine 304 and the frequency counter engine 310 operate in at least partially parallel. For example, the group identification engine 304 can identify a single group based on the quality score array 302. The group identification engine 304 can then send the identified single group to the frequency counter engine 310. The frequency counter engine 310 can then determine the number of occurrences of the quality score for the identified single group. In some cases, generating a group from the quality score array 302 may include identifying one or more quality scores in the quality score array 302. In general, any process described herein may be threaded or run concurrently with another process, and two or more processes may run on one or more instances of devices or software.

[0081] The frequency counter engine 310 generates a quality score group frequency 314 and sends it to the reduced sequence generation engine 316. The quality score group frequency 314 may include the number of occurrences of one or more quality scores in the quality score sequence 302. In step D, the reduced sequence generation engine 316 obtains the quality score group frequency 314 and can communicate with both the single mapping engine 320 and the group mapping engine 326. That is, if the quality score of the quality score sequence 320 is not part of a group of quality scores that exceed a threshold number of quality scores, the reduced sequence generation engine 316 can use the single mapping engine 320 to generate entries in the reduced sequence 332 in step E. Alternatively, if the quality score of the quality score sequence 320 is part of a group of quality scores that exceed a threshold number of quality scores, the reduced sequence generation engine 316 can use the group mapping engine 326 in step F. For the purposes of this disclosure, an “entry” or “single entry” in the reduced sequence 332 may include a single value in the reduced sequence 332, such as “72,” which was used to replace a single quality score or a group of quality scores.

[0082] For the purposes of this specification, it is possible to determine whether a group of quality scores exceeds a threshold number of quality scores by using positive or negative expressions for the threshold number of quality scores and the number of quality scores in a group. Therefore, it is consistent with this specification to determine whether the number of quality scores in a group of quality scores "meets" the threshold, rather than simply whether the number of quality scores is greater than the threshold. This is because such a relationship can be described as a group of quality scores having four quality scores that exceeds the threshold of three quality scores, or a group of quality scores having four negative quality scores that are less than or equal to the negative threshold of three quality scores. In either case, a group of quality scores has more than three quality scores, regardless of how the threshold is implemented.

[0083] For the sake of clarity, this specification describes the process of the single mapping engine 320 before describing the process of the group mapping engine 326. However, by using concurrent processing and other similar methods, the single mapping engine 320 may not require a complete single mapping process before the group mapping engine 326 completes the group mapping process. Alternatively, whether the reduced sequence generation engine 316 calls the single mapping engine 320 or the group mapping engine 326 is determined based on the specific quality score of the sequence of quality score 302 processed by the reduced sequence generation engine 326.

[0084] In stage E, the single mapping engine 320 obtains the quality score array 302. The single mapping engine 320 uses the quality score array 302 and the single mapping character list 322 to generate a given single mapping 324, as shown in item 321. In the embodiment of Figure 3, the single mapping character list 322 contains integer values ​​from 0 to 63. The ASCII values ​​of the quality score array 302 contain values ​​from 33 to 96. In this way, each value in the quality score array 302 can be mapped to a specific value from 0 to 63. For example, the "@" character in the quality score array 302 corresponding to the ASCII value 64 can be mapped to the integer value 31. Similarly, the "A" character in a given quality score array corresponding to the ASCII value 65 can be mapped to the integer value 33, and so on.

[0085] As shown in a given single mapping 324, the ASCII character "!" corresponding to the value 33 is mapped to the value 0, which is between 0 and 63. The ASCII character "" corresponding to the value 34 is mapped to the value 1. Similarly, the ASCII character "_" corresponding to the value 95 is mapped to the value 62, and the ASCII character "`" corresponding to the value 96 is mapped to the value 63. Other mappings not shown in a given single mapping 324 may also be generated by a single mapping engine 320.

[0086] In other embodiments, other mappings may be used. For example, instead of 0 to 63, a smaller or larger range may be used in which the values ​​of the quality score array 302 are mapped to values ​​from 33 to 96, such as ASCII values ​​from 33 to 96. Other mappings, such as those generated by the group mapping engine 326, may occupy value ranges other than 33 to 96. In some implementations, the number of unique quality scores in the quality score array 302 is used to determine the range to which the quality scores of the quality score array 302 are mapped. For example, if the quality score array 302 contains 63 unique quality scores, the range to which the quality scores of the quality score array 302 are mapped may contain 63 values. In some implementations, other ranges are used. For example, if the quality score array 302 contains a first number of unique quality scores, the range may be calculated, for example, by a single mapping engine 320 to include the number of unique quality scores of the first number divided by 2, or other calculation results based on the number of unique quality scores of the first number. In some implementations, subsequent actions are applied to the processing result used to determine the mapping range. For example, if the intrinsic quality score of the first number is odd, and the first operation applied to the intrinsic quality score of the first number is division by the integer 2, then the second operation may, depending on the implementation, include rounding up or rounding down the corresponding result.

[0087] In stage F, the group mapping engine 326 obtains the quality score group frequencies 314. The group mapping engine 326 uses the quality score group frequencies 314 or other data related to the quality score array 302 together with the group mapping character list 328 to generate a given group mapping 330, as shown in item 327. In the embodiment of Figure 3, the group mapping character list 328 includes integer values ​​64 to 245 corresponding to the 190 most frequently occurring groups. In some implementations, the group mapping engine 326 may map more or fewer groups. For example, instead of generating a given group mapping 330 that includes the 190 most frequently occurring groups, the group mapping engine 326 may generate a mapping that includes 200, 230, 185, or any other number of most frequently occurring groups. The ASCII values ​​corresponding to the quality score group frequencies 314 include values ​​33 to 96. Based on the quality score group frequencies 314, the group mapping engine 326 may determine a portion of the groups for the mapping. For example, the group mapping engine 326 can determine a specific number of most frequently occurring groups (e.g., 190 frequently occurring groups) and assign a value (e.g., an integer value between 64 and 254) to each of the most frequently occurring groups.

[0088] As shown in the predetermined group mapping 330, in the embodiment of Figure 3, the quality score group represented by the letters "ACD" is mapped to the value 64. The quality score group represented by the letters "FFF" is mapped to the value 72. The quality score group represented by the letters "HIJ" is mapped to the value 73. Other mappings not shown in the predetermined group mapping 330 may also be generated by the group mapping engine 326.

[0089] In other embodiments, other mappings can be used. For example, instead of 64-254, a smaller or larger range can be used to which groups within the quality score group frequency 314 are mapped. For example, groups within the quality score group frequency 314 may be mapped to values ​​from 0 to 255. Other mappings, such as those generated by a single mapping engine 320, may occupy values ​​other than those used for the 0-255 group mapping.

[0090] The reduced sequence generation engine 316 processes the quality score sequence 302 using the group mapping engine 326 and the single mapping engine 320 to generate the reduced sequence 332. The data derived from the quality score sequence 302 may include the quality score sequence 332 itself. Alternatively, the data derived from the quality score sequence 302 may include the data output by the frequency counter engine 310. The data output by the frequency counter engine 310 may include the quality score group frequencies 314. The reduced sequence 332 is a combination of values ​​from a predetermined single mapping 324 and a predetermined group mapping 330.

[0091] The reduced sequence generation engine 316 can process the quality score sequence 302, the quality score group frequencies, or both, and determine whether to generate entries in the reduced sequence 332 using a single mapping engine 320 or group mapping engine 326. Occurrences of groups in the quality score sequence 302 that belong to a group in a given group mapping 330 are replaced with values ​​from the given group mapping 330. For example, occurrences of an "A" quality score followed by a "C" quality score followed by a "D" quality score are replaced with values ​​64 in the reduced sequence 332.

[0092] For quality scores that are not members of any of the groups in a given group mapping 330, a given single mapping 324 is used. For example, the first value in the quality score array 302 is the character "@" corresponding to the ASCII value 64. When the character "@" appears in the quality score array 302, it is not a member of any of the groups in the given group mapping 330. The character "@" is then mapped based on the given single mapping 324. The mapping of "@" based on the given single mapping 324 is the value 31. The reduced array generation engine 316 adds the value 31 corresponding to the position of "@" in the reduced array 332.

[0093] In stage G, the reduced sequence generation engine 316 generates a reduced sequence 332 using a predetermined group mapping 330, a predetermined single mapping 324, and data derived from the quality score sequence 302, as described above and as shown in item 331. The reduced sequence generation engine 316 sends the reduced sequence 332 to the compression engine 334.

[0094] In stage H, the compression engine 334 performs subsequent compression on the reduced array 332, corresponding to a predetermined compression method. For example, in some cases, the compression engine 334 performs compression based on an implementation of Partial Matching Prediction (PPMD) of a range encoder for compressing the reduced array 332. In some implementations, other compression methods known in the art can be used instead of, or in addition to, the PPMD ​​implementation. Generally, the compression engine 334 can use any compression or any combination of compression methods known in the art. The compression engine 334 generates an output 336 based on the reduced array 332 as input to the compression method used by the compression engine 334. The output 336 represents a compressed version of the quality score array 302.

[0095] In some implementations, each value in the reduced array 332 supplied to the compression engine 334 is compressed according to a one-byte context. For example, for a given value in the reduced array 332, the previous value in the reduced array 332 may be used as context to compress the given value in the reduced array 332. Using a one-byte context, the reduced array 332 can be compressed according to a range encoder or another PPMD ​​implementation of compression or coding method. Larger or smaller contexts may be used to balance speed, the resulting compressed output, or other parameters. In some cases, the compression ratio resulting from output 336 may be two bits or more per quality score. For example, four or more quality scores can be compressed into one byte or eight bits in memory space. In general, output 336, similar to output 128 in Figure 1, may be used in any further process or stored depending on the implementation.

[0096] For the sake of clarity, we will use stages A through H. The processes performed by system 300 may occur in the order shown in stages A through H, although in other implementations, the order of certain stages may differ. In some implementations, two or more stages may occur simultaneously.

[0097] Figure 4 is a flowchart showing one embodiment of a process for compressing an array of quality scores having a second data format. Process 400 can be carried out by one or more electronic systems, for example, system 300 in Figure 3.

[0098] Process 400 includes obtaining a genetic read sequence generated by a genetic sequencing device (402) and obtaining a plurality of quality scores corresponding to the genetic read sequence (404). For example, as shown in Figure 3, a quality score sequence 302 is generated by a sequencer based on a genetic data sequence and sent to a group identification engine 304. In the embodiment of Figure 3, the quality score sequence 302 is generated by a sequencer that uses the Q40 data format to encode the quality scores associated with the bases as described above. In general, any sequencer that uses more than a threshold number of unique quality scores can compress the resulting output data using the process performed by the system 300 shown in Figure 3 or a similar process. For example, if a sequencer uses more than eight unique quality scores to encode the quality scores corresponding to the genetic sequence, the sequencer can use the encoding and compression process considered with reference to Figure 3.

[0099] Process 400 includes determining the frequency of occurrence for each quality score group of multiple quality scores (406). For example, as shown in Figure 3, the group identification engine 304 obtains a quality score array 302 and generates several quality score groups based on multiple quality scores in the quality score array 302. In some implementations, a group of quality scores includes quality scores that are adjacent to each other in the quality score array 302. For example, if the quality score array 302 contains the quality score "@C@DFFFF...", the group determined by the group identification engine 304 may include the quality score "@C@" in the implementation where 3 quality scores are used to generate the group.

[0100] In some implementations, the group identification engine 304 sends one or more groups to the frequency counter engine 310, which determines the frequency of occurrence of each quality score group. For example, for each group determined by the group identification engine 304, the frequency counter engine 310 can determine how often the group occurs within the quality score array 302. In the embodiment of Figure 3, the quality scores in the quality score array 302 are grouped into groups of 3. However, other implementations can use other numbers of quality scores. For example, the group identification engine 304 can determine groups of 4 quality scores, 2 quality scores, or any other number of quality scores to generate quality score groups.

[0101] Process 400 includes determining for each of a plurality of quality scores whether the quality score is a member of a particular group of quality scores that has an occurrence frequency that satisfies a predetermined threshold (408). For example, as shown in Figure 3, a reduced sequence generation engine 316, a single mapping engine 320, and a group mapping engine 326 generate a reduced sequence 332 based on a predetermined single mapping 324 and a predetermined group mapping 330. For each quality score in the reduced sequence 332, the reduced sequence generation engine 316 determines whether the quality score is a member of a group included in the predetermined group mapping 330. If the quality score is a member of a group included in the predetermined group mapping 330, the quality score is coded as a single entry in the reduced sequence 332 together with the other quality scores in the group.

[0102] In some implementations, each group in a given group mapping 330 satisfies a predetermined threshold. For example, each group in a given group mapping 330 may occur within a predetermined number of quality score arrays 302 included in the group mapping. Each group in a given group mapping 330 may belong to a predetermined number of groups that occur more frequently than other groups in the quality score arrays 302. In the embodiment of Figure 3, the given group mapping 330 includes 190 most frequently occurring quality score groups. The threshold can then be defined, in particular depending on the implementation, as the number of occurrences corresponding to the 191st most frequently occurring quality score group, the 190th most frequently occurring quality score group, and so on. Then, all 190 most frequently occurring quality score groups of the given group mapping 330 satisfy this threshold.

[0103] In some implementations, a given group mapping 330 can be generated using different quantities of quality score groups. For example, the number of groups in a given group mapping 330 may be determined based on the number of quality scores used by the sequencer. The mapping can then be generated to satisfy an optimization process. For example, the optimization process may include maximizing the number of groups coded as a single value and minimizing the number of quality scores that do not belong to any quality score group in the given group mapping 330 and are coded as a single value. Parameters to optimize may include the number of groups included in a group mapping, such as the given group mapping 330 in Figure 3, and the number of quality scores used to generate one or more groups within the given group mapping 330. The number of quality scores in a single mapping, such as the given single mapping 324 in Figure 3, may correspond to the number of unique quality scores used by the sequencer to represent the quality scores corresponding to the genetic sequence reads.

[0104] In some implementations, quality score groups are based on adjacent quality scores that appear together in a quality score array. For example, a quality score array 302 represented as "@C@DFFFF..." contains a group of adjacent quality scores "FFF". The group of adjacent quality scores "FFF" is included in a predetermined group mapping 330. According to the predetermined group mapping 330, the group of adjacent quality scores "FFF" should be coded by the reduced array generation engine 316 as a single value 72. Of course, in other implementations, other values ​​can be used for mapping or coding purposes.

[0105] In some implementations, scores that are not members of a quality score group are mapped as a single entry in a reduced array. For example, in an implementation that generates a group of quality scores using three quality scores, the first value "@" in the quality score array 302, represented as "@C@DFFFF...", is not a member of a particular quality score group. Therefore, the first value "@" is coded as a single value 31 in the reduced array 332 according to a predetermined single mapping 324. In the embodiment of Figure 3, the predetermined single mapping 324 is a one-to-one match of the values ​​used to represent the quality score array 302 and a new range of quality scores of the same length, and as a result, any quality score that is not part of a group in a predetermined group mapping 330 is coded in the reduced array 332 as a shifted value of the original value plus or minus used to generate the predetermined single mapping 324. For example, in Figure 3, the single value corresponding to a quality score that is not a member of a quality score group is shifted by a value of 33. Shifts can be used to generate a dedicated continuous range of values ​​for either the group mapping or the single mapping.

[0106] Process 400 includes generating a single entry (410) to be included in a reduced array as a representation of a particular quality score group, based on the determination that a quality score is a member of a particular quality score group. For example, the reduced array generation engine 316 may determine that a quality score array 302 represented as "@C@DFFFF..." contains the quality score group "FFF". The quality score group "FFF" is included in a given group mapping 330. According to the given group mapping 330, adjacent quality score groups "FFF" should be coded by the reduced array generation engine 316 as a single value 72. However, this disclosure is not limited to replacing a group of quality scores with a single entry or value. In other implementations, a single quality score may be replaced with a single entry or value using a given single mapping. Such replacement of a single quality score may be beneficial because the single entry or value may be chosen to represent a single quality score having a smaller bit size than the single quality score.

[0107] Process 400 includes generating a reduced sequence (412) by aggregating each of the generated entries. For example, the reduced sequence generation engine 316 can aggregate the first coded value 31 corresponding to the quality score "@" in the quality score sequence 302, represented as "@C@FFFF...", the second coded value 34 corresponding to the quality score "C", the third coded value 31 corresponding to the quality score "@", the further coded value 35 corresponding to the quality score "D", and the fifth coded value 72 corresponding to the quality score group "FFF". The reduced sequence generation engine 316 can generate subsequent coded values ​​based on the subsequent values ​​of the quality score sequence 302. The reduced sequence generation engine 316 can continue until all the values ​​of the quality score sequence 302 are represented in the reduced sequence 332.

[0108] In some implementations, further compression steps are performed based on the reduced array 332. For example, as shown in Figure 3, the reduced array 332 may be sent to the compression engine 334. The compression engine 334 can then perform one or more compression processes on the reduced array 332 to produce the output 336. Similar to the process shown in Figure 1, the initial coding represented by the reduced array 332 generated before the subsequent compression steps performed by the compression engine 334 may be advantageous in improving the compression resulting in the output 336. The form of the reduced array 332 may be such that the compression engine 334 can compress the data more quickly or effectively than the original quality score array 302. For example, the reduced array 332 may be a compressed version of the quality score array 302, and the duration or quality of the compression performed by the compression engine 334 may depend on the size of the input. In this way, the initial coding step for generating the reduced array 332 can reduce the time required to compress and increase the quality of compression achieved by the compression engine 334.

[0109] Figure 5 is a flowchart showing one embodiment of a process 500 for reconstructing an array of quality scores having a first data format. Process 500 can be carried out by one or more electronic systems, for example, system 100 in Figure 1.

[0110] Process 500 involves obtaining a first coded dataset generated by coding each of several quality scores using a number in base x, where x is an integer number representing the number of different quality scores used by the genetic sequencing device (502). For example, the decoding engine may obtain binary output 114 or binary output 126.

[0111] Process 500 includes generating a first decoded dataset using a number in base x (504). For example, as in the processes shown in items 108 and 120, the first coded dataset may be decoded based on a number in base x used for coding, where x is an integer corresponding to the number of unique quality scores present in a set of quality scores. In some implementations, the integer value of the binary representation of the first decoded data may be repeatedly divided by the number in base x to generate a number in base x.

[0112] Process 500 includes ordering the first decoded dataset within one or more other decoded datasets (506). In some implementations, the ordering engine may take the decoded dataset and one or more other decoded datasets and order the first decoded data based on the first coded dataset. For example, the ordering engine may determine a portion of the first coded dataset which is then decoded to generate the first decoded dataset. The ordering engine may also determine a portion of the first coded dataset which is decoded to generate one or more other decoded datasets. Based on the original location of the first decoded dataset and one or more other decoded datasets within the first coded dataset, the ordering engine may order the first decoded dataset within one or more other decoded datasets.

[0113] Process 500 includes generating an aggregated decoded dataset based on a first decoded dataset and one or more other decoded datasets (508). For example, based on the ordering of the first decoded dataset in one or more other decoded datasets, the aggregated decoded dataset engine can generate an aggregated decoded dataset containing the first decoded dataset and one or more other decoded datasets. The aggregated decoded dataset may contain data similar to the data used to generate the first coded dataset. The aggregated decoded dataset may be used in other processes or operations depending on the implementation.

[0114] Figure 6 is a flowchart showing one embodiment of process 600 for reconstructing an array of quality scores having a second data format. Process 600 can be carried out by one or more electronic systems, for example, system 300 in Figure 3.

[0115] Process 600 includes obtaining reduced sequences (602) generated based on a single mapping database and a group mapping database. For example, the decoding engine may obtain reduced sequences 332 or other related data from system 300 or other systems. Then, the process shown corresponding to the reduced sequence generation engine 316 can be reversed to generate the quality score sequence 302.

[0116] Process 600 includes generating a first decoded dataset (604) based on the reduced sequences and a single mapping database and a group mapping database. As described above, the decoding engine is similar to the reduced sequence generation engine 316, but can perform the operations in reverse order to generate the quality score sequences 302. For example, the decoding engine may take the reduced sequences 332, decode the reduced sequences 332 using a given group mapping 330 and a given single mapping 324, and generate a first decoded dataset corresponding to at least a portion of the quality score sequences 302.

[0117] Process 600 includes ordering the first decoded dataset within one or more other decoded datasets (606). For example, a decoding engine may decode a first portion of the reduced sequence 332 and determine a first order corresponding to the first portion of the reduced sequence 332. In some implementations, the order of the reduced sequence 332 can be used to determine the order of the first decoded dataset and one or more other decoded datasets. For example, the first decoded dataset may correspond to a first portion of the reduced sequence 332. As a result, the first decoded dataset may be ordered at the beginning of the finally aggregated decoded dataset. Then, one or more subsequent decoded datasets may be ordered based on the order of data corresponding to the reduced sequence 332 used to generate the one or more other decoded datasets.

[0118] Process 600 includes generating an aggregated decoded dataset based on a first decoded dataset and one or more other decoded datasets (608). For example, an aggregated decoded dataset can be generated using an aggregated decoded dataset engine, based on ordering the first decoded dataset within one or more other decoded datasets as described above. The aggregated decoded dataset may contain data similar to the data used to generate reduced sequences, such as reduced sequences 332. The aggregated decoded dataset may be used in other processes or operations depending on the implementation.

[0119] Figure 7 is a flowchart showing one embodiment of a process 700 for determining a method for compressing quality scores. Process 700 can be carried out by one or more electronic systems, for example, system 300 in Figure 3 or system 100 in Figure 1.

[0120] Process 700 includes obtaining genetic data from a genetic sequencer (702). Various forms of genetic sequencers are known to those skilled in the art. For example, a decision engine can obtain genetic data corresponding to one or more quality scores generated by a given genetic sequencer.

[0121] Process 700 includes determining the number of unique quality scores in the genetic data (704). For example, depending on the type, model, or specific software of the sequencer, the quality score of a base call may be represented by one or more symbols or values. The number of unique symbols or values ​​used to represent one or more quality scores can then be used to determine which compression method is used for the quality scores of the genetic data.

[0122] Process 700 includes a first decision pathway corresponding to the determination that there are unique quality scores of 8 or less in the genetic data obtained from the genetic sequencer, and a second decision pathway corresponding to the determination that there are unique quality scores greater than 8 in the genetic data obtained from the genetic sequencer. If there are unique quality scores of 8 or less in the genetic data, the quality scores of the genetic data can be compressed by performing the process corresponding to process 200 shown in Figure 2. If there are unique quality scores greater than 8 in the genetic data, the quality scores of the genetic data can be compressed by performing the process corresponding to process 400 shown in Figure 4.

[0123] In some implementations, other thresholds are used to determine which compression method to use. For example, instead of 8 intrinsic quality scores, the system might determine that 7, 9, or 10 intrinsic quality scores are required to perform the process corresponding to process 400 shown in Figure 4. In general, the system can use any suitable threshold or decision model to determine which of several different compression methods is used to compress a given set of data, such as a given set of quality scores.

[0124] The genomic data referred to herein (e.g., input data 102) may include, for example, nucleotide sequences, deoxyribonucleic acid (DNA) sequences, ribonucleic acid (RNA) sequences, and amino acid sequences, for example, but not limited to these. While the description herein is quite detailed with respect to genomic information in the form of nucleotide sequences, it will be understood by those skilled in the art that, although there may be some modifications, the ordered data sequences of this specification can be implemented for other genomic data.

[0125] Figure 8 is a graphical representation of experimental results for a process to encode (e.g., compress) an array of quality scores having a first data format. Figure 8 shows the results from the compression of the dataset "SRR6882909_1.fastq". The dataset "SRR6882909_1.fastq" is formatted using the Q4 format described above. The quality scores in the dataset "SRR6882909_1.fastq" contain four unique quality scores. Chart 802 shows the relative size of the raw data corresponding to the dataset "SRR6882909_1.fastq", as well as compressed versions of the dataset "SRR6882909_1.fastq" using various techniques including gzip level 9 compression, zstd level 11 compression, and Lena Q4 compression. Lena Q4 compression corresponds to process 100 shown in Figure 1 and the method described herein.

[0126] Chart 802 shows that the raw data of the dataset "SRR6882909_1.fastq" is 7402335856 bytes, the compressed version of the dataset "SRR6882909_1.fastq" using gzip level 9 compression is 417512395 bytes, the compressed version of the dataset "SRR6882909_1.fastq" using zstd level 11 compression is 452733689 bytes, and the compressed version of the dataset "SRR6882909_1.fastq" using Lena Q4 compression is 259865991 bytes. The compression achieved by Lena Q4 compression of the dataset "SRR6882909_1.fastq" is greater than the compression achieved by other alternative compression methods. Legend 806 indicates which bar corresponds to which compression method and which bar corresponds to the raw data in the dataset "SRR6882909_1.fastq".

[0127] Chart 804 shows the compression times for each compression method used on the dataset "SRR6882909_1.fastq". Chart 804 shows that it takes 1980 seconds to generate a compressed version of the dataset "SRR6882909_1.fastq" using gzip level 9 compression, 108 seconds to generate a compressed version of the dataset "SRR6882909_1.fastq" using zstd level 11 compression, and 36 seconds to generate a compressed version of the dataset "SRR6882909_1.fastq" using Lena Q4 compression. The time required to compress the dataset "SRR6882909_1.fastq" using Lena Q4 compression is less than the compression times achieved by the other alternative compression methods.

[0128] Figure 9 is a graphical representation of experimental results for the process of encoding (e.g., compressing) an array of quality scores having a second data format. Figure 9 shows the results from the compression of the dataset "ERR1744700_1.fastq". The dataset "ERR1744700_1.fastq" is formatted using the Q40 format described above. The quality scores in the dataset "ERR1744700_1.fastq" contain more than 4 unique quality scores. Chart 902 shows the relative size of the raw data corresponding to the dataset "ERR1744700_1.fastq", as well as compressed versions of the dataset "ERR1744700_1.fastq" using various techniques, including gzip level 9 compression, zstd level 11 compression, and Lena Q40 compression. Lena Q40 compression corresponds to process 300 shown in Figure 3.

[0129] Chart 902 shows that the raw data of the dataset "ERR1744700_1.fastq" is 5033592178 bytes, the compressed version of the dataset "ERR1744700_1.fastq" using gzip level 9 compression is 1289564690 bytes, the compressed version of the dataset "ERR1744700_1.fastq" using zstd level 11 compression is 1290828665 bytes, and the compressed version of the dataset "ERR1744700_1.fastq" using Lena Q40 compression is 1228518456 bytes. The compression achieved by Lena Q40 compression of the dataset "ERR1744700_1.fastq" is greater than the compression achieved by other alternative compression methods. Legend 906 indicates which bar corresponds to which compression method and which bar corresponds to the raw data in the dataset "ERR1744700_1.fastq".

[0130] Chart 904 shows the compression times for each compression method used on the dataset "ERR1744700_1.fastq". Chart 904 shows that it takes 3762 seconds to generate a compressed version of the dataset "ERR1744700_1.fastq" using gzip level 9 compression, 306 seconds to generate a compressed version of the dataset "ERR1744700_1.fastq" using zstd level 11 compression, and 101 seconds to generate a compressed version of the dataset "ERR1744700_1.fastq" using Lena Q40 compression. The time required to compress the dataset "ERR1744700_1.fastq" using Lena Q40 compression is less than the compression times achieved by the other alternative compression methods.

[0131] Figure 10 shows the components of a computer system 1000 that can be used to implement a system for generating medical analysis using joint models based on multivariate data.

[0132] Computing device 1000 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Computing device 1050 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other similar computing devices. In addition, computing device 1000 or 1050 may include a Universal Serial Bus (USB) flash drive. The USB flash drive may store an operating system and other applications. The USB flash drive may include input / output components, such as a wireless transmitter or USB connector, which can be inserted into a USB port of another computing device. The components shown herein, the connections and relationships of these components, and the functions of these components are meant to be examples only and are not intended to limit the forms of implementation of the invention described and / or claimed herein.

[0133] The computing device 1000 includes a processor 1002, memory 1004, storage device 1008, a high-speed interface 1008 connected to memory 1004 and a high-speed expansion port 1010, and a low-speed bus 1014 and a low-speed interface 1012 connected to storage device 1008. Each of the components 1002, 1004, 1008, 1008, 1010, and 1012 is interconnected using various buses and can be mounted on a common motherboard or in other configurations as appropriate. The processor 1002 processes instructions for execution within the computing device 1000, including instructions stored in memory 1004 or on storage device 1008, and can display graphical information for a GUI on an external input / output device such as a display 1016 connected to the high-speed interface 1008. In other implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple types of memory as appropriate. Furthermore, multiple computing devices 1000 can be connected so that each device provides the necessary computing portion, for example, as a server bank, a group of blade servers, or a multiprocessor system.

[0134] The memory 1004 stores information within the computing device 1000. In one implementation, the memory 1004 is a volatile memory unit or a plurality of volatile memory units. In another implementation, the memory 1004 is a non-volatile memory unit or a plurality of non-volatile memory units. The memory 1004 may also be another form of computer-readable medium, such as a magnetic disk or an optical disk.

[0135] The storage device 1008 can provide large-capacity storage for the computing device 1000. In one implementation, the storage device 1008 may be or contain a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, flash memory or other similar solid-state memory device, or an array of devices including devices in a storage area network or other configuration. The computer program product may be tangibly implemented within the information carrier. The computer program product may also contain instructions that, when executed, perform one or more of the methods described above. The information carrier is a computer-readable medium or machine-readable medium, such as memory 1004, the storage device 1008, or memory on the processor 1002.

[0136] The high-speed controller 1008 manages bandwidth-intensive computations of the computing device 1000, while the low-speed controller 1012 manages low-bandwidth-intensive computations. This assignment of functions is just one example. In one implementation, the high-speed controller 1008 is connected, for example, to memory 1004, display 1016, and a high-speed expansion port 1010 that can accept various expansion cards (not shown) via a graphics processor or accelerator. In this implementation, the low-speed controller 1012 is connected to the storage device 1008 and the low-speed expansion port 1014. The low-speed expansion port, which can include various communication ports such as USB, Bluetooth, Ethernet, and wireless Ethernet, can be coupled to one or more input / output devices such as a keyboard, pointing device, microphone / speaker pair, scanner, or networking device such as a switch or router, for example, via a network adapter. The computing device 1000 can be implemented in several different forms, as shown in the figure. For example, the computing device can be implemented as a standard server 1020, or it can be implemented multiple times in a group of such servers. The computing device can also be implemented as part of a rack server system 1024. In addition, the computing device can be implemented in a personal computer such as a laptop computer 1022. Alternatively, components from computing device 1000 can be combined with other components in a mobile device (hidden), such as device 1050. Each of these devices can contain one or more computing devices 1000, 1050, and the entire system can consist of multiple computing devices 1000, 1050 communicating with each other.

[0137] The computing device 1000 can be implemented in several different forms, as shown in the figure. For example, the computing device can be implemented as a standard server 1020, or can be implemented multiple times in a group of such servers. The computing device can also be implemented as part of a rack server system 1024. In addition, the computing device can be implemented in a personal computer such as a laptop computer 1022. Alternatively, components from the computing device 1000 can be combined with other components in a mobile device (not shown), such as device 1050. Each of these devices can contain one or more computing devices 1000 and 1050, and the entire system can consist of multiple computing devices 1000 and 1050 communicating with each other.

[0138] The computing device 1050 includes, among other components, a processor 1052, memory 1064, and input / output devices such as a display 1054, a communication interface 1066, and a transceiver 1068. The device 1050 may also include storage devices, such as a microdrive or other devices, to provide additional storage. Each of the components 1050, 1052, 1064, 1054, 1066, and 1068 are interconnected using various buses, and some of the components can be implemented on a common motherboard or in other configurations as appropriate.

[0139] Processor 1052 can execute instructions within the computing device 1050, including instructions stored in memory 1064. The processor can be implemented as a chipset of chips including separate and multiple analog and digital processors. In addition, the processor can be implemented using one of several architectures. For example, processor 1010 can be a CISC (Complex Instruction Set Computer) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor. The processor can provide coordination of other components of device 1050, such as user interface control, applications run by device 1050, and wireless communication by device 1050.

[0140] The processor 1052 can communicate with the user via the control interface 1058 and the display interface 1056 connected to the display 1054. The display 1054 can be, for example, a TFT (Thin-Film-Transistor Liquid Crystal Display), an OLED (Organic Light Emitting Diode) display, or other suitable display technology. The display interface 1056 may include appropriate circuitry for driving the display 1054 to present graphical and other information to the user. The control interface 1058 can receive commands from the user and translate these commands for input to the processor 1052. In addition, an external interface 1062 can be provided for communication with the processor 1052 to enable short-range wireless communication between device 1050 and other devices. The external interface 1062 can be provided, for example, for wired communication in some implementations and for wireless communication in other implementations, and multiple interfaces can also be used.

[0141] Memory 1064 stores information within the computing device 1050. Memory 1064 can be implemented as one or more computer-readable media, volatile memory units, or non-volatile memory units. Furthermore, an expansion memory 1074 can be provided and connected to device 1050 via an expansion interface 1072, which may include, for example, a SIMM (Single In-Line Memory Module) card interface. Such an expansion memory 1074 can provide additional storage space for device 1050, or it can store applications or other information for device 1050. Specifically, the expansion memory 1074 may contain instructions to execute or complement the processes described above, and may also contain secure information. Therefore, for example, the expansion memory 1074 can be provided as a security module for device 1050 and can be programmed with instructions that enable secure use of device 1050. In addition, secure applications can be provided via SIMM cards, along with additional information such as placing identification information on the SIMM card in a hack-proof manner.

[0142] The memory may include, for example, flash memory and / or NVRAM memory, as will be described later. In one implementation, the computer program product is tangibly implemented within an information carrier. When executed, the computer program product contains instructions that perform one or more methods, such as those described above. The information carrier is a computer-readable or machine-readable medium, such as memory 1064, extended memory 1074, or memory on processor 1052, which can be received via transceiver 1068 or external interface 1062.

[0143] Device 1050 can communicate wirelessly via a communication interface 1066, which may include digital signal processing circuitry if necessary. The communication interface 1066 can provide communication under various modes or protocols, including, in particular, GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA®, CDMA2000, or GPRS. Such communication can be performed, for example, via a radio frequency transceiver 1068. In addition, short-range communication is possible, such as using Bluetooth, Wi-Fi, or other such transceivers (not shown). Furthermore, a GPS (Global Positioning System) receiver module 1070 can provide device 1050 with additional navigation-related and location-related radio data, which can be used as appropriate by applications running on device 1050.

[0144] Device 1050 can also communicate audibly using audio codec 1060, which can receive speech information from the user and convert this speech information into usable digital information. Audio codec 1060 can also generate audible sound for the user, for example, through a speaker in the handset of device 1050. Such sound may include sounds from voice telephone calls, recorded sounds such as voice messages and music files, and sounds generated by applications running on device 1050.

[0145] The computing device 1050 can be implemented in several different forms, as shown in the figure. For example, the computing device can be implemented as a mobile phone 1080. Alternatively, the computing device can be implemented as part of a smartphone 1082, a personal digital assistant, or other similar mobile device.

[0146] Various implementations of the systems and methods described herein can be realized in digital electronic circuits, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations of such implementations. These various implementations may include implementations in one or more computer programs that are dedicated or general-purpose and are executable and / or interpretable on a programmable system including at least one programmable processor coupled to receive and transmit data and instructions from a storage system, at least one input device, and at least one output device.

[0147] These computer programs (also known as programs, software, software applications, or code) contain machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus, and / or device, such as magnetic disks, optical disks, memory, programmable logic devices (PLDs) used to provide machine instructions and / or data to a programmable processor, and include machine-readable medium that receives machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0148] To provide user interaction, the systems and technologies described herein can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device, such as a mouse or trackball, on which the user can provide input to the computer. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and the input from the user can be received in any form, including acoustic input, speech input, or tactile input.

[0149] The systems and technologies described herein can be implemented in a computing system including a backend component, such as a data server; in a computing system including a middleware component, such as an application server; or in a computing system including a frontend component, such as a client computer having a graphical user interface or web browser that allows a user to interact with the implementation of the systems and technologies described herein; or in any combination of such backend, middleware, or frontend components. The components of the system can be interconnected by digital data communication, such as any form or medium of a communication network. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), and the Internet.

[0150] A computing system can include clients and servers. Clients and servers are generally remote to each other and typically interact via a communication network. The relationship between a client and a server arises from computer programs running on each computer that have a client-server relationship with each other.

[0151] Several embodiments have been described. However, it will be understood that various modifications can be made without departing from the spirit and scope of the present invention. In addition, the logical flow depicted in the figures does not require a specific order or sequence shown to achieve the desired result. Furthermore, other steps can be provided or steps can be removed from the described flow, and other components can be added to or removed from the described system. Accordingly, other embodiments are within the scope of the following claims.

[0152] All embodiments and functional operations of the present invention described herein can be implemented in digital electronic circuits, computer software, firmware, or hardware, or one or more combinations thereof, including the structures disclosed herein and their structural equivalents. Embodiments of the present invention can be implemented as one or more modules of computer program instructions coded on a computer-readable medium for execution by a data processing device or for controlling the operation of a data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects a machine-readable propagating signal, or one or more combinations thereof. The term “data processing device” encompasses all devices and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, a device may include code that makes up an execution environment for the computer program in question, such as processor firmware, a protocol stack, a database management system, an operating system, or code that constitutes one or more combinations thereof. A propagating signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a suitable receiving device.

[0153] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs, modules, components, subroutines, or other units suitable for use in a computing environment. Computer programs do not necessarily correspond to files in a file system. A program can be stored in part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subprograms, or parts of code). Computer programs can be deployed and run on one or more computers located in one place or distributed across multiple locations and interconnected by a network.

[0154] The processes, operations, and / or logical flows described herein may be implemented by one or more central processing units (CPUs) or graphics processing units (GPUs) that execute one or more computer software instructions to realize the functions of the processes, operations, and / or logical flows described herein. The processes, operations, and / or logical flows may also be implemented within hardware circuits. For example, in some implementations, the operations of the Disclosure may be implemented by a processing engine that uses logic gates of a field programmable gate array (FPGA) programmed to realize the functions of the processes, operations, and / or logical flows described herein. In another embodiment, the operations of the Disclosure may be implemented by a processing engine that uses logic gates of an application-specific integrated circuit (ASIC) configured to realize the functions of the processes, operations, and / or logical flows described herein. In yet another implementation, some of the parts of the processes, operations, and / or logical flows may be implemented by one or more CPUs or one or more GPUs, and some of the parts of the processes, operations, and / or logical flows may be implemented in hardware circuits and in any order.

[0155] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, and one or more arbitrary processors in any type of digital computer. Generally, a processor receives instructions and data from read-only memory, random-access memory, or both. Essential elements of a computer are a processor for executing instructions, and one or more memory devices for storing instructions and data. Generally, a computer also includes, or is operably connected to, one or more mass storage devices for storing data, such as receiving data from magnetic, magneto-optical disks, or optical disks, transmitting data to mass storage devices, or both. However, a computer is not required to have such devices. Furthermore, a computer can be incorporated into other devices, for example, tablet computers, mobile phones, personal digital assistants (PDAs), portable audio players, and Global Positioning System (GPS) receivers. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices like EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Processors and memory can be supplemented by or incorporated into special-purpose logic circuits.

[0156] To provide user interaction, embodiments of the present invention can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device, such as a mouse or trackball, on which the user can provide input to the computer. User interaction can also be provided using other types of devices, for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, including acoustic input, speech input, or tactile input.

[0157] Embodiments of the present invention can be implemented, for example, in a computing system including a backend component as a data server, or in a computing system including a middleware component, for example, an application server, or in a computing system including a frontend component, for example, a client computer having a graphical user interface or web browser that allows a user to interact with the implementation of the present invention, or in any combination of such backend, middleware, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), such as the Internet.

[0158] A computing system can include clients and servers. Clients and servers are generally remote to each other and typically interact via a communication network. The relationship between a client and a server arises from computer programs running on each computer that have a client-server relationship with each other.

[0159] While this specification contains many details, these should not be construed as limiting the scope of the invention or what may be claimed, but rather as descriptions of features specific to particular embodiments of the invention. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any preferred partial combination. Furthermore, features may be described above as functioning in a particular combination, and even if initially claimed as such, one or more features from the claimed combination may, in some cases, be removed from the combination, and the claimed combination may relate to a partial combination or a variation of a partial combination.

[0160] Similarly, while operations are depicted in a specific order in the drawings, it should not be understood that such operations must be performed in a specific order or sequentially as shown, or that all exemplary operations must be performed, in order to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can usually be integrated together in a single software product or packaged within multiple software products.

[0161] In each example where a specific file format is mentioned, it may be replaced by another file type or format. For example, an HTML file may be replaced by XML, JSON, plain text, or another type of file. Furthermore, where a table or hash table is mentioned, other data structures (such as spreadsheets, relational databases, or structured files) may be used.

[0162] Specific embodiments of the present invention have been described. Other embodiments are within the scope of the following claims. For example, the steps listed in the claims can be performed in a different order, and the desired results can still be achieved.

[0163] Other Embodiments Specific embodiments of the present invention have been described. Other embodiments are within the scope of the following claims. For example, the steps listed in the claims can be performed in a different order, and the desired results can still be achieved. [Explanation of Symbols]

[0164] 100, 300 systems 1000 computing devices

Claims

1. Obtaining a coded dataset generated by coding a read sequence containing data corresponding to multiple base calls generated by a nucleic acid sequencing device, wherein coding uses a value representing a number of different quality scores, each quality score indicating the likelihood that a particular base call of the read sequence was correctly generated by the nucleic acid sequencing device. The decoding dataset is generated using a value representing the number of different quality scores, Ordering the decoded datasets within one or more other decoded datasets, A method comprising generating an aggregated decoded dataset based on the decoded dataset and one or more other decoded datasets.

2. Using a value representing the number of different quality scores to generate the decoded dataset is, The method according to claim 1, comprising repeatedly dividing an integer value representing the coded dataset by a value representing the number of different quality scores.

3. The method according to claim 2, comprising generating an integer value representing the coded dataset using an integer value representing each of the numbers of different quality scores and a value representing the number of different quality scores.

4. This includes generating a second decoded dataset using a value representing the number of different quality scores, The method according to claim 1, wherein ordering the decrypted datasets within one or more other decrypted datasets includes ordering the decrypted dataset and the second decrypted dataset.

5. Ordering the aforementioned decoded dataset and the aforementioned second decoded dataset is The method according to claim 4, comprising ordering the decoded dataset before the second decoded dataset in response to the determination in the read sequence that the decoded dataset corresponds to a read that occurs before the read of the second decoded dataset.

6. Determining that the decoded dataset within the read sequence corresponds to a read that occurs before the read of the second decoded dataset means that The method according to claim 5, comprising extracting location data indicating the location of the decoded dataset and the location of the second decoded dataset from the coded dataset.

7. This includes generating a second decoded dataset using a value representing the number of different quality scores, The method according to claim 1, wherein generating an aggregated decrypted dataset based on the decrypted dataset and the one or more other decrypted datasets includes aggregating the decrypted dataset and the second decrypted dataset.

8. Generating the aggregated and decoded dataset is The method according to claim 7, comprising generating a version of the read sequence that includes data corresponding to the plurality of base calls generated by the nucleic acid sequencing device.

9. Obtaining the aforementioned coded dataset is The method according to claim 1, comprising obtaining data from an initial coding process or data from a compression process following an initial coding process.

10. The method according to claim 9, wherein the compression process includes an implementation of prediction by partial matching (PPMD) of a range encoder.

11. Obtaining the aforementioned coded dataset is The method according to claim 1, comprising obtaining coded data generated by encoding the reading array using a number in base x, wherein x is an integer corresponding to a value representing the number of different quality scores.

12. The method according to claim 1, wherein the value representing the number of different quality scores is equal to 3.

13. The method according to claim 1, comprising generating the coded dataset using a coding process selected from two or more coding processes.

14. One or more non-temporary computer-readable storage media encoded using instructions, wherein, when executed by one or more computers, the instructions cause the one or more computers to perform an operation, and the operation is Obtaining a coded dataset generated by coding a read sequence containing data corresponding to multiple base calls generated by a nucleic acid sequencing device, wherein coding uses a value representing a number of different quality scores, each quality score indicating the likelihood that a particular base call of the read sequence was correctly generated by the nucleic acid sequencing device. The decoding dataset is generated using a value representing the number of different quality scores, Ordering the decoded datasets within one or more other decoded datasets, A medium comprising generating an aggregated decoded dataset based on the decoded dataset and one or more other decoded datasets.

15. Using a value representing the number of different quality scores to generate the decoded dataset is, The medium according to claim 14, comprising repeatedly dividing an integer value representing the coded dataset by a value representing the number of different quality scores.

16. The medium according to claim 15, wherein the operation includes generating an integer value representing the coded dataset using an integer value representing each of the numbers of different quality scores and a value representing the number of different quality scores.

17. The operation includes generating a second decoded dataset using a value representing the number of different quality scores, The medium according to claim 14, wherein ordering the decrypted datasets within one or more other decrypted datasets includes ordering the decrypted dataset and the second decrypted dataset.

18. Ordering the aforementioned decoded dataset and the aforementioned second decoded dataset is The medium according to claim 17, comprising ordering the decoded datasets before the second decoded dataset in response to the determination in the read sequence that the decoded datasets correspond to reads that occur before the reads of the second decoded datasets.

19. Determining that the decoded dataset within the read sequence corresponds to a read that occurs before the read of the second decoded dataset means that The medium according to claim 18, comprising extracting location data indicating the location of the decoded dataset and the location of the second decoded dataset from the coded dataset.

20. A system comprising one or more computers and one or more storage devices storing instructions, wherein, when executed by the one or more computers, the instructions are operable to cause the one or more computers to perform an action, and the action is Obtaining a coded dataset generated by coding a read sequence containing data corresponding to multiple base calls generated by a nucleic acid sequencing device, wherein coding uses a value representing a number of different quality scores, each quality score indicating the likelihood that a particular base call of the read sequence was correctly generated by the nucleic acid sequencing device. The decoding dataset is generated using a value representing the number of different quality scores, Ordering the decoded datasets within one or more other decoded datasets, A system comprising generating an aggregated decoded dataset based on the decoded dataset and one or more other decoded datasets.

Images