Data compression method, data decompression method, device and computer system

Abstract

The present disclosure provides a data compression method, comprising: obtaining a quality value sequence from sequencing data; obtaining a priority score for any quality value in the quality value sequence, wherein the priority score represents the degree of influence of the quality value on a genotype associated with the sequencing data; shifting a bit plane of the quality value based on the priority score to obtain an updated quality value sequence; and encoding the updated quality value sequence based on a bit plane coding algorithm to obtain a compressed bit stream of the quality value sequence. The present disclosure also provides a data decompression method, a data compression device, a data decompression device, and a computer system.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a data compression method, a data decompression method, a data compression device, a data decompression device, and a computer system. Background Art

[0002] In the field of bioinformatics / genomics, high-throughput sequencing data obtained through genome sequencing usually occupies a large amount of data and needs to be compressed and transmitted for subsequent genome data processing and analysis.

[0003] Existing technologies typically divide sequencing data into different data blocks based on their different data fields or data types, and then compress them separately. Currently, only lossless encoding or lossy encoding with a fixed amount of data loss is possible. This results in limited flexibility and prevents scalable encoding from lossless to varying degrees of lossiness. Summary of the Invention

[0004] One aspect of the present disclosure provides a data compression method, comprising: obtaining a quality value sequence in sequencing data; obtaining a priority score for any quality value in the quality value sequence, wherein the priority score represents a degree of influence of the any quality value on a genotype associated with the sequencing data; shifting a bit plane of the any quality value based on the priority score to obtain an updated quality value sequence; and encoding the updated quality value sequence based on a bit plane coding algorithm to obtain a compressed bit stream of the quality value sequence.

[0005] Optionally, the above-mentioned obtaining of the priority score of any quality value in the quality value sequence includes: setting a first deviation amount and a second deviation amount for any quality value; calculating a first probability of a genotype associated with the sequencing data when any quality value is increased by the first deviation amount; calculating a second probability of a genotype associated with the sequencing data when any quality value is reduced by the second deviation amount; and, obtaining the priority score of any quality value based on the first deviation amount, the second deviation amount, the first probability and the second probability.

[0006] Optionally, the calculating the first probability of the genotype associated with the sequencing data and the calculating the second probability of the genotype associated with the sequencing data include: calculating the posterior probability of the genotype associated with the sequencing data based on a Bayesian algorithm.

[0007] Optionally, the above-mentioned method of obtaining the priority score of any quality value includes: calculating a third deviation between the first probability and the second probability; taking the sum of the first deviation and the second deviation as a fourth deviation; and, based on the ratio of the third deviation to the fourth deviation, obtaining the priority score of any quality value.

[0008] Optionally, the above-mentioned calculation of the first probability of the genotype associated with the sequencing data and the calculation of the second probability of the genotype associated with the sequencing data include: when the genotype associated with the sequencing data includes multiple genotypes, respectively calculating the first probability and the second probability of any genotype among the multiple genotypes; calculating the third deviation between the first probability and the second probability of any genotype; taking the sum of the first deviation and the second deviation as the fourth deviation; and, obtaining the priority score of any quality value based on the ratio of the mean of the third deviation corresponding to each genotype in the multiple genotypes to the fourth deviation, or, when the third deviation corresponding to any genotype is greater than the third deviation corresponding to other genotypes in the multiple genotypes, obtaining the priority score of any quality value based on the ratio of the third deviation corresponding to any genotype to the fourth deviation.

[0009] Optionally, the above-mentioned shifting of the bit plane of any quality value based on the priority score includes: when the priority score is less than or equal to zero, determining the shift amount to be 0; when the priority score is greater than zero and less than a first value, determining the shift amount to be the rounded result of the priority score; when the priority score is greater than or equal to the first value, determining the shift amount to be the first value; and moving the bit plane of any quality value by the shift amount to obtain the updated quality value sequence.

[0010] Optionally, the sequencing data includes a plurality of short read sequences arranged in a predetermined order, the short read sequences include a plurality of quality values, and the quality values in the plurality of short read sequences constitute the quality value sequence.

[0011] Optionally, the above method further includes: before shifting the bit plane of any quality value based on the priority score, dividing the any quality value into intervals according to the attribute information of the any quality value, the attribute information including at least one of the following: whether the alignment direction of the short read sequence where the any quality value is located is forward alignment or reverse alignment, whether the short read sequence where the any quality value is located is the first sequence or the second sequence, and the position of the any quality value on the short read sequence where it is located; and, for any quality value within any interval, mapping the any quality value to an updated quality value based on a predetermined mapping rule, so that the maximum updated quality value within the any interval is less than or equal to the maximum quality value within the any interval. Then the above-mentioned shifting the bit plane of any quality value based on the priority score includes: shifting the bit plane of the updated quality value based on the priority score.

[0012] Optionally, the above-mentioned shifting of the bit plane of any updated quality value based on the priority score includes: for any interval, determining the maximum bit plane regarding any interval based on the maximum updated quality value within the any interval; for any updated quality value, calculating the binary data of any updated quality value, taking the maximum bit plane of the interval to which the any updated quality value belongs as the highest bit of the binary data; shifting the bit plane of the binary data based on the priority score of the quality value corresponding to the any updated quality value; and, the updated quality value sequence is constituted by the binary data of each updated quality value after shifting.

[0013] Optionally, the above-mentioned encoding of the updated quality value sequence based on the bit plane coding algorithm includes: scanning the updated quality value sequence in order from high to low in the bit plane, and recording the values of the scanned bits in sequence, wherein if the value of the scanned bit does not exist, skipping the bit.

[0014] Optionally, determining the maximum bit-plane for any interval based on the maximum update quality value within the interval includes using the most significant bit of the maximum update quality value as the maximum bit-plane. And / or, calculating the binary data of any update quality value includes converting the difference between the update quality value and the minimum update quality value within the interval to which the update quality value belongs into binary form to obtain the binary data.

[0015] Another aspect of the present disclosure provides a data decompression method, comprising: obtaining a compressed bitstream of a quality value sequence, the quality value sequence being divided into one or more intervals, the quality value sequence including multiple quality values; obtaining a maximum bit-plane in any interval and a priority score for any quality value; decoding the compressed bitstream based on the maximum bit-plane, the priority score, and a bit-plane encoding algorithm to obtain the quality value sequence; and obtaining sequencing data matching the quality value sequence based on the quality value sequence. The priority score represents a bit-plane displacement of the quality value, and the priority score represents a degree of influence of the quality value on a genotype associated with the sequencing data.

[0016] Optionally, obtaining the compressed bitstream of the quality value sequence includes: obtaining all compressed bitstreams of the quality value sequence. Decoding the compressed bitstream based on the maximum bit-plane, the priority score, and the bit-plane coding algorithm includes: determining, for any quality value, a bit distribution of the quality value based on the maximum bit-plane of each interval, the maximum bit-plane of the interval to which the quality value belongs, and the priority score of the quality value; and, based on the bit distribution, sequentially filling the values of the compressed bitstream into each bit in descending order of the bit plane to obtain the quality value sequence.

[0017] Optionally, obtaining the compressed bitstream of the quality value sequence includes: obtaining the first predetermined number of bits of the compressed bitstream of the quality value sequence. Decoding the compressed bitstream based on the maximum bit-plane, the priority score, and the bit-plane coding algorithm includes: determining, for any quality value, a bit distribution of the quality value based on the maximum bit-plane of each interval, the maximum bit-plane of the interval to which the quality value belongs, and the priority score of the quality value; filling the values of the first predetermined number of bits into each bit in descending order of the bit plane based on the bit distribution; and, for any quality value, if there is a bit without a value, reconstructing the quality value to obtain the quality value sequence.

[0018] Optionally, the above-mentioned reconstruction of any quality value includes: for any quality value, setting the value of all bits without values in any quality value to 0, to obtain a first value of the quality value; setting the value of all bits without values in any quality value to 1, to obtain a second value of the quality value; and, using the first value as the reconstructed value of any quality value, or using the second value as the reconstructed value of any quality value, or using the average of the first value and the second value as the reconstructed value of any quality value.

[0019] Optionally, the method further includes: after obtaining the quality value sequence, obtaining a predetermined mapping rule; and reversely mapping the quality value sequence to an original quality value sequence based on the predetermined mapping rule. Obtaining sequencing data matching the quality value sequence includes obtaining sequencing data matching the original quality value sequence.

[0020] Another aspect of the present disclosure provides a data compression device, comprising: a first acquisition module, a second acquisition module, a displacement module, and an encoding module. The first acquisition module is configured to acquire a quality value sequence from sequencing data. The second acquisition module is configured to acquire a priority score for any quality value in the quality value sequence, wherein the priority score represents the degree of influence of any quality value on a genotype associated with the sequencing data. The displacement module is configured to shift a bit plane of any quality value based on the priority score to obtain an updated quality value sequence. Furthermore, the encoding module is configured to encode the updated quality value sequence based on a bit plane encoding algorithm to obtain a compressed bit stream of the quality value sequence.

[0021] Another aspect of the present disclosure provides a data decompression device, comprising: a third acquisition module, a fourth acquisition module, a decoding module, and a matching module. The third acquisition module is used to obtain a compressed bit stream of a quality value sequence, wherein the quality value sequence is divided into one or more intervals, and the quality value sequence includes multiple quality values. The fourth acquisition module is used to obtain the maximum bit plane of any interval and the priority score of any quality value. The decoding module is used to decode the compressed bit stream based on the maximum bit plane, the priority score, and the bit plane coding algorithm to obtain the quality value sequence. And, the matching module is used to obtain sequencing data that matches the quality value sequence based on the quality value sequence. Wherein, the priority score represents the degree of influence of any quality value on the genotype related to the sequencing data.

[0022] Another aspect of the present disclosure provides a computer system, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the method described above when executing the program.

[0023] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0024] Another aspect of the present disclosure provides a computer program, which includes computer-executable instructions. When the instructions are executed, the computer program is used to implement the method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] For a more complete understanding of the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 Schematically illustrates an exemplary system architecture of a data compression method, a data decompression method, a data compression device, and a data decompression device according to an embodiment of the present disclosure;

[0027] Figure 2A The following schematically shows a flow chart of a data compression method according to an embodiment of the present disclosure;

[0028] Figure 2B Schematically shows a bit plane representation diagram of a quality value according to an embodiment of the present disclosure;

[0029] Figure 2C Schematically shows a mapping relationship diagram of interval quality values according to an embodiment of the present disclosure;

[0030] Figure 2D Schematically shows a structural diagram of a quality data block according to an embodiment of the present disclosure;

[0031] Figure 3 The following schematically shows a flow chart of a data decompression method according to an embodiment of the present disclosure;

[0032] Figure 4 Schematically shows a block diagram of a data compression device according to an embodiment of the present disclosure;

[0033] Figure 5 A block diagram schematically shows a data processing device according to another embodiment of the present disclosure;

[0034] Figure 6 Schematically shows a block diagram of a data decompression device according to an embodiment of the present disclosure;

[0035] Figure 7 A block diagram schematically shows a data decompression device according to another embodiment of the present disclosure; and

[0036] Figure 8 The block diagram schematically shows a computer system according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0038] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0039] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0040] When expressions such as “at least one of A, B, and C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, and C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). When expressions such as “at least one of A, B, or C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, or C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.).

[0041] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks in the block diagrams and / or flow charts or their combinations can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, can create a device for implementing the functions / operations described in these block diagrams and / or flow charts. The technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable storage medium having instructions stored thereon, which can be used by an instruction execution system or in combination with an instruction execution system.

[0042] Figure 1 The exemplary system architecture of the data compression method, data decompression method, data compression device, and data decompression device according to the embodiment of the present disclosure is schematically shown. The exemplary system architecture 100 is applicable to the data compression method and device, and / or, to the data decompression method and device. It should be noted that, Figure 1The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0043] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0044] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104. Various client applications can be installed on terminal devices 101, 102, and 103, such as bioinformatics applications (such as sequence alignment software), data processing applications (such as data compression applications, data decompression applications, etc.) (for example only).

[0045] The terminal devices 101 , 102 , and 103 may be various electronic devices with certain computing capabilities, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0046] Server 105 can be a server or server cluster with certain computing capabilities for providing various services. For example, server 105 can receive data from terminal devices 101, 102, and 103, analyze and process the received data, and feed back the processing results to terminal devices 101, 102, and 103.

[0047] It should be noted that the data compression method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the data compression device provided in the embodiments of the present disclosure can generally be set in the server 105. The data compression method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the data compression device provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0048] The data compression method provided in the embodiment of the present disclosure may also be executed by one or more of the terminal devices 101, 102, and 103. Accordingly, the data compression apparatus provided in the embodiment of the present disclosure may also be provided in one or more of the terminal devices 101, 102, and 103.

[0049] Furthermore, / or, the data decompression method provided by the embodiments of the present disclosure may generally be executed by the server 105. Accordingly, the data decompression apparatus provided by the embodiments of the present disclosure may generally be disposed in the server 105. The data decompression method provided by the embodiments of the present disclosure may also be executed by a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the data decompression apparatus provided by the embodiments of the present disclosure may also be disposed in a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, 103 and / or the server 105.

[0050] The data decompression method provided in the embodiment of the present disclosure may also be executed by one or more of the terminal devices 101, 102, and 103. Accordingly, the data decompression apparatus provided in the embodiment of the present disclosure may also be provided in one or more of the terminal devices 101, 102, and 103.

[0051] It should be understood that Figure 1 The numbers of terminal devices, networks and servers in the embodiment are merely illustrative. Any number of terminal devices, networks and servers may be provided according to actual needs.

[0052] Figure 2A The flowchart of the data compression method according to the embodiment of the present disclosure is schematically shown.

[0053] like Figure 2A As shown, the method may include the following operations S201 to S204.

[0054] In operation S201 , a quality value sequence in sequencing data is obtained.

[0055] In the field of bioinformatics / genomics, the high-throughput sequencing data obtained by genome sequencing is mainly stored in, for example, FASTQ files as input for subsequent genome data processing and analysis processes. FASTQ files are a collection of short read sequences (reads) in text format. Each read includes: identification information, base sequence, an optional description line (which can be ignored in subsequent processing to save storage space), and a quality value sequence. Wherein, each quality value in the quality value sequence of a read represents the detection reliability of each base in the base sequence of the read, i.e., the reliability of each base in the base sequence of the read used to describe the read. Due to distortion in the tandem order relationship between different reads in the FASTQ file, the first step in most genome analysis processes is to align the original read to a reference genome sequence to obtain aligned sequencing data. The sequencing data described in this operation S201 can be sequencing data composed of a plurality of reads whose tandem order relationship is correct after alignment, and the quality value sequence in the acquired sequencing data can be the quality value sequence in the tandem order data. Compared with base sequences, quality value sequences often occupy a larger amount of data. Therefore, the data compression method according to the embodiment of the present disclosure compresses the quality value sequences of sequencing data to facilitate the transmission and interaction of sequencing data in subsequent processing.

[0056] In operation S202 , a priority score of any quality value in the quality value sequence is obtained.

[0057] For each quality value in the quality value sequence, the priority score of the quality value represents the degree of influence of the quality value on the genotype associated with the above-mentioned sequencing data. This operation S202 measures the degree of influence of each quality value on the genotype associated with the sequencing data in which the quality value sequence is located by obtaining the priority score of each quality value in the quality value sequence. For example, if the degree of influence of a quality value on the genotype is large, a small change in the quality value may cause a change in the genotype. If the degree of influence of a quality value on the genotype is small, a large change in the quality value may not cause a change in the genotype.

[0058] In operation S203, the bit plane of any quality value is shifted based on the priority score to obtain an updated quality value sequence.

[0059] Each quality value in the obtained quality value sequence can be converted into a binary format for representation. Each quality value includes one or more bits, which are divided into multiple bit planes from top to bottom in descending order of bit positions. In operation S203, for each quality value in the obtained quality value sequence, the bit plane of the quality value is shifted based on the priority score of the quality value. For example, the bit planes of some quality values in the quality value sequence are shifted upward by x1 bits, some by x2 bits, some by x3 bits, some by x4 bits, and so on, to obtain an updated quality value sequence.

[0060] In operation S204, the updated quality value sequence is encoded based on a bit-plane coding algorithm to obtain a compressed bit stream of the quality value sequence.

[0061] In this operation S204, a bit-plane encoding algorithm is used to scan the updated quality value sequence based on the bit plane to obtain a corresponding compressed bit stream, which is used as the data compression result of the quality value sequence in the sequencing data obtained in operation S201.

[0062] Those skilled in the art will understand that Figure 2A The method shown, when compressing the quality value sequence in the comparison sequencing data, performs a bit-plane shift on the quality value according to the degree of influence of the quality value on the genotype, and then performs bit-plane encoding on the quality value sequence after the bit-plane shift to obtain a compressed bit stream of the quality value sequence. Thus, in the obtained compressed bit stream, the more forward bits carry information about the quality value that has a greater degree of influence on the genotype. In this way, even if only a portion of the compressed bit stream is received during lossy encoding, important information that affects the genotype will be preferentially retained. Based on this compressed bit stream, more accurate sequencing data can be restored without affecting the subsequent genome data processing and analysis process. Therefore, the data compression method according to the embodiment of the present disclosure realizes scalable compression of the quality value of the comparison sequencing data.

[0063] In one embodiment of the present disclosure, obtaining a priority score for any quality value in the sequence of quality values for the aligned sequencing data may include: setting a first deviation and a second deviation for the quality value; and calculating a first probability of a genotype associated with the aligned sequencing data when the quality value increases by the first deviation. If the quality value decreases by the second deviation, calculating a second probability of a genotype associated with the aligned sequencing data. The priority score for the quality value is obtained based on the first deviation, the second deviation, the first probability, and the second probability.

[0064] According to this embodiment, assuming that the sequencing data consists of multiple reads covering site 1 after alignment with the reference genome, and the coverage rate of the sequencing data for site 1 is N, the base sequence n in the sequencing data can be expressed as:

[0065]

[0066] The quality value sequence q in the sequencing data can be expressed as:

[0067]

[0068] Where i represents the number of bases and quality values, and bases and quality values correspond one to one. Quality value q i Used to describe the base n i The reliability of the base n i The sequencing error rate is P error , then the quality value q i It can be represented as an example:

[0069]

[0070] That is, in this example, the quality value is equal to the product of the logarithm of the sequencing error rate of the corresponding base and -10, rounded down. In some cases, for the convenience of format storage and processing, the value obtained by formula (3) can be converted to ASCII code. Regardless of the format of the quality value, when executing the data compression method according to the present disclosure, the original quality value needs to be converted to binary format.

[0071] With quality value q i Take the following example to illustrate the process of obtaining the priority score for each quality value: For example, for the quality value q i Set the first deviation Δ1 and the second deviation Δ2. i When the first deviation Δ1 is added, the quality value is changed from q i becomes When the first probability P1 of the genotype associated with site 1 is calculated. i When the second deviation Δ2 is reduced, the quality value is changed from q i becomes When , calculate the second probability P2 of the genotype associated with site 1. Based on the first deviation Δ1, the second deviation Δ2, the first probability P1 and the second probability P2, the quality value q is obtained i Priority score.

[0072] In one embodiment of the present disclosure, obtaining the priority score of any quality value based on the first deviation, the second deviation, the first probability, and the second probability may include: calculating a third deviation between the first probability and the second probability, taking the sum of the first deviation and the second deviation as a fourth deviation, and obtaining the priority score of any quality value based on the ratio of the third deviation to the fourth deviation.

[0073] For example, for the quality value q above i For example, the third deviation between the first probability P1 and the second probability P2 can be expressed as D[P1, P2], where the function D[] is used to measure the deviation or distortion between the genotype probabilities caused by the change in mass value. The fourth deviation can be expressed as (Δ1+Δ2). The ratio of the third deviation to the fourth deviation is calculated as r i :

[0074] r i =D[P1, P2] / (Δ1+Δ2) (4)

[0076] Therefore, in this example, the ratio r i Get the quality value q i The priority score l i :

[0077] l i =log2(r i / r0) (5)

[0079] Among them, r0 is a preset constant, which means that the quality value q i The distortion rate of the corresponding genotype when there is no distortion. i That is the quality value q i The priority score will be used to calculate q in bit plane coding i In other examples, formula (5) can be set as needed. In principle, it is only necessary to characterize the priority score and the above ratio r i The corresponding relationship between them is sufficient.

[0080] For example, as an optional embodiment, the above-mentioned calculation of the first probability of the genotype associated with the sequencing data and the calculation of the second probability of the genotype associated with the sequencing data may include: when the genotype associated with the sequencing data includes multiple genotypes, respectively calculating the first probability and the second probability of any genotype in the multiple genotypes. On this basis, the above-mentioned obtaining the priority score of any quality value based on the first deviation, the second deviation, the first probability and the second probability may include: first, calculating the third deviation between the first probability and the second probability of any genotype in the multiple genotypes; and taking the sum of the first deviation and the second deviation as the fourth deviation. Then, the priority score is calculated in one of the following ways: (1) obtaining the priority score of any quality value based on the ratio of the average of the third deviations corresponding to each genotype in the multiple genotypes to the above-mentioned fourth deviation; or (2) when the third deviation corresponding to any genotype is greater than the third deviation corresponding to other genotypes in the multiple genotypes, obtaining the priority score of any quality value based on the ratio of the third deviation corresponding to the any genotype to the above-mentioned fourth deviation.

[0081] According to this embodiment, when the sequencing data corresponds to multiple genotypes, the priority score can be calculated using the average of the third deviation amounts corresponding to the multiple genotypes, or the priority score can be calculated using the maximum third deviation amount corresponding to the multiple genotypes to obtain a priority score that can reflect the degree of influence of the quality value on the genotype.

[0082] The following is a specific example to illustrate the process of calculating the priority score of the quality value in the above embodiment. Since the process of calculating the priority score involves the calculation of genotype probability, the calculation process of genotype probability is first illustrated here.

[0083] Typically, genotype probabilities can be calculated from observed data using a Bayesian statistical model. Genotype variations include both germline mutations and somatic mutations. Existing technologies only support genotype probability calculations in germline mutation detection, but not in somatic mutation detection. In this example, this model will be used to expand support for detection of both germline and somatic mutations. Furthermore, in other embodiments, other models can be flexibly selected for genotype probability calculation based on downstream applications.

[0084] Specifically, in embodiments of the present disclosure, the posterior probability of a genotype associated with the sequencing data can be calculated based on a Bayesian algorithm. For example, the calculation of the first probability of a genotype associated with the sequencing data and the calculation of the second probability of a genotype associated with the sequencing data can both be performed based on a Bayesian algorithm.

[0085] The genotype of site 1 in the genome corresponding to the sequencing data can be represented by a random variable G, and A represents the allele that can be found at site 1. The genotype can be represented as: G = (A1, ..., A γ ,...,A Γ ), where allele A γ From the base Allele alphabet Γ is the chromosome multiple of the species. The set of all possible genotypes is The base number is This can be obtained by calculating all possible combinations (including combinations of the same alleles):

[0086]

[0087] For example, in DNA sequencing, the allele alphabet Cardinality For diploid organisms (Γ = 2), we have These ten Possible genotypes. As an optional embodiment, in order to make the model support somatic variation at the same time, the variable s∈S={0,1} can be introduced to indicate whether the current position contains somatic variation (s=1) or does not contain somatic variation (s=0). In this case, the set of genotypes Expanded to twice the original value, that is, G=(A1, ..., A γ ,...,A Γ )×s.

[0088] Considering the above-mentioned set of reads with coverage N for site 1, base sequence n (see formula (1)), and quality value sequence q (see formula (2)), the posterior distribution of genotype G is calculated by the genotype uncertainty model. Based on Bayesian theory, the posterior probability of genotype G is:

[0089]

[0090]

[0091] Among them, the likelihood probability P(n|G;q) is:

[0092]

[0093] P(n in formula (8) i |G;q i ) is a given genotype G and quality value q i In the case of i Likelihood probability. Assuming base n i The probability of taking values in all alleles is equal, and the likelihood probability can be calculated as follows (9):

[0094]

[0095] P in formula (9) a is the prior probability of the unknown allele a, which can be calculated by the following formula (10):

[0096]

[0097] in is the set of possible somatic cell variant genotypes. In order to make the model support somatic cell variants at the same time, the parameter P is introduced in formula (10) s For germ cell mutation, when s=0, P s Fixed to 0, for somatic cell variation, that is, when s = 1, P s Indicates the allele frequency of somatic mutations. In specific implementation, Ps can take values in the range of [0, 1] according to the compressed data type.

[0098] In addition, P(n i |a;q i ) assumes that the true base is allele a and the parameter is the quality value q i When n is observed i The likelihood probability can be calculated using the following formula (11):

[0099]

[0100] Therefore, by substituting formulas (8) to (11) into formula (7), the probability of the genotype can be calculated. Then, the process of calculating the priority score of any quality value is further illustrated.

[0101] Based on the genotype probability, the priority score of each quality value can be calculated. The data compression algorithm according to the embodiment of the present disclosure calculates a suitable scaling factor for each quality value data to base n. i The quality value q iScaling (bit-plane shifting) is performed to change the priority of the encoding order, so that the quality values that are important to the genotype probability are encoded first in the bit-plane sequence encoding. This scaling factor can also be called a priority score. In the bit-plane encoding, this priority score will be used as the bit-plane shift of the quality value data.

[0102] As an example, the quality value q i Set the same first deviation and second deviation, assuming that the first deviation and second deviation are both Δ. Define the quality value q i The distortion rate is:

[0103] r i =D[π(n,q i+ ),π(n,q i- )] / 2Δ (12)

[0105] in, is a γ-dimensional vector that records the different genotypes {G1, ..., G γ Function D: R γ →R is used to calculate the deviation or distortion between two different posterior probability vectors due to compression error, that is, the third deviation mentioned above.

[0106] The mass value q in the mass value sequence q i Let (q i +Δ), we can get the quality value sequence q with error i+ , the quality value sequence q i+ Any quality value in It can be expressed as:

[0107]

[0108] Similarly, the quality value q in the quality value sequence q i Let (q i -Δ), we can get the quality value sequence q with error i- , the quality value sequence q i- Any quality value in It can be expressed as:

[0109]

[0110] In obtaining the quality value q i The distortion rate r i After that, the quality value q can be calculated by the formula (5) above i The priority score l i =log2(ri / r0).

[0111] In the above process of calculating the priority score of the quality value, function D is used to measure the third offset of the genotype probability. Any suitable distance measurement function can be used as function D as needed. For example, if the p-norm distance is used for measurement, function D can be expressed as:

[0112]

[0113] In other embodiments, if the genotype with the highest posterior probability is of greatest interest, the change in the posterior probability of the genotype due to the change in the quality value may be used as the third offset, that is:

[0114]

[0115] Among them, G M is the most likely genotype, i.e.

[0116] After obtaining the priority scores of each quality value in the quality value sequence, the bit planes of each quality value can be shifted based on the priority scores of each quality value to obtain an updated quality value sequence. In one embodiment of the present disclosure, the above-mentioned shifting of the bit plane of any quality value based on the priority score of any quality value includes: for any quality value in the quality value sequence, first determining the displacement amount based on the priority score of any quality value. Specifically, when the priority score is less than or equal to zero, the displacement amount is determined to be 0. When the priority score is greater than zero and less than a first value, the displacement amount is determined to be the rounded result of the priority score. When the priority score is greater than or equal to the first value, the displacement amount is determined to be the first value. Then, the bit plane of any quality value is shifted by the above-mentioned displacement amount to obtain an updated quality value sequence.

[0117] Using the example above, we can get the quality value q i The priority score l i Then, assuming the first value is 6, based on the priority score l i The displacement s can be determined using formula (17): i :

[0118]

[0119] In actual observations, for bases that are correctly matched to the reference genome sequence, the displacement s calculated by the corresponding quality value is iIt is usually 0, and this phenomenon is particularly evident when the sequencing data has a high coverage of the site and most of the bases are well matched to the reference genome sequence. For mismatched bases, inserted and / or deleted bases (which can be called "InDels"), the displacement of the corresponding quality value is very important. In order to save workload and improve data processing efficiency, as an optional embodiment, during the data compression process, the displacement table only stores the displacement of the quality value related to the mismatched or InDels bases. Among them, the displacement table is used as auxiliary information for lossless coding and is transmitted to the decoding end, so that after the binary compressed bit stream is received, the decoding end can correctly reconstruct the quality value sequence.

[0120] The data compression method according to the embodiment of the present disclosure is a quality value sequence constructed based on all the received aligned sequencing data that are aligned to a continuous gene position on the reference genome sequence, which can also be called data stacking blocks (pileups). In practical applications, in addition to the case of alignment mismatch, the case of InDels can also be considered. Optionally, in order to handle InDels, when an insertion or deletion occurs at the current gene position, two new letters I and D can be introduced into the allele alphabet, specifically including: (1) Since the inserted base on the read has no corresponding position on the reference sequence, the inserted base needs to be merged with the base after them and given a pseudo-base type "I"; (2) relative to the position of the vacant or deleted base on the reference genome, the pseudo-base type "D" is assigned. Therefore, a quality value sequence considering the InDels situation is obtained. Based on this quality value sequence, the possible genotype probability can be calculated to determine the bit plane displacement of each quality value. The process of determining the displacement amount is logically the same as the process of determining the displacement amount described above, and will not be repeated here.

[0121] As described above, in an embodiment of the present disclosure, sequencing data may include multiple short read sequences arranged in a predetermined order, each short read sequence includes multiple quality values, and the quality values in the multiple short read sequences constitute a quality value sequence, which serves as a compression object of the data compression method according to an embodiment of the present disclosure.

[0122] Furthermore, in one embodiment of the present disclosure, the data compression method according to the embodiment of the present disclosure may also include: before shifting the bit plane of any quality value based on the priority score, dividing the interval for any quality value according to the attribute information of any quality value. Wherein, for any quality value in the quality value sequence, the attribute information includes at least one of the following: (1) the alignment direction of the short read sequence where the quality value is located is forward alignment or reverse alignment (reverse alignment is also called "reverse complement"); (2) the short read sequence where the quality value is located is the first sequence or the second sequence in the double-end sequencing data; (3) the position of the quality value on the short read sequence where it is located. The quality value can be divided into different intervals based on the above one or more attribute information.

[0123] After dividing each quality value into different intervals, the above-mentioned shifting of the bit plane of any quality value based on the priority score may include: for any interval, determining the maximum bit plane of any interval based on the maximum quality value in the any interval; for any quality value, calculating the binary data of the any quality value, and taking the maximum bit plane of the interval to which the any quality value belongs as the highest bit of the binary data of the any quality value; shifting the bit plane of the binary data of the any quality value based on the priority score of the any quality value; and, forming an updated quality value sequence with the binary data of each quality value after the shift.

[0124] In a specific embodiment, determining the maximum bit-plane for any interval based on the maximum quality value within the interval may include: using the most significant bit of the maximum quality value within the interval as the maximum bit-plane for the interval. And / or calculating the binary data of any quality value may include: converting the difference between the quality value and the minimum quality value within the interval to which the quality value belongs into binary form to obtain the binary data corresponding to the quality value.

[0125] On this basis, as an optional embodiment, the above-mentioned encoding of the updated quality value sequence based on the bit plane coding algorithm includes: scanning the updated quality value sequence in order from high to low according to the bit plane, and recording the values of the scanned bits in sequence, wherein if the value of the scanned bit does not exist, the bit is skipped.

[0126] For example, for any interval, determine the minimum quality value q in the interval min and the maximum mass value q max For any quality value q in this interval, use formula (18) to convert the quality value q into binary form:

[0127]

[0128] Where M is the maximum bit plane corresponding to the interval, satisfying 2 M ≥(q max -q min )>2 (M-1) , b i ∈{0, 1} represents the binary quality bit of the quality value q.

[0129] Figure 2B A diagram schematically shows a bit plane representation of a quality value according to an embodiment of the present disclosure.

[0130] like Figure 2B As shown in (1), the binary form of each quality value before the shift is represented in the bit plane in the order from the most significant bit (MSB) to the least significant bit (LSB). For example, each column corresponds to a quality value row, and there are six columns in total. The quality values of the first and second columns belong to interval 1, and the maximum bit plane M of interval 1 is 6. The quality values of the third and fourth columns belong to interval 2, and the maximum bit plane M of interval 2 is 5. The quality values of the fifth and sixth columns belong to interval 3, and the maximum bit plane M of interval 3 is 4. The displacement corresponding to the quality value of each column is also shown below {1, 0, 3, 2, 0, 1}.

[0131] like Figure 2B As shown in (2), the bit plane of the quality value of each column shown in (1) is shifted according to the corresponding displacement amount. For example, the bit plane of the quality value of the first column is shifted upward by 1 bit, the bit plane of the quality value of the third column is shifted upward by 3 bits, the bit plane of the quality value of the fourth column is shifted upward by 2 bits, and the bit plane of the quality value of the sixth column is shifted upward by 1 bit. The other vacant bits are marked with the default symbol "X". After completing the above shift, the updated quality value sequence is obtained. Then the updated quality value sequence can be bit-plane encoded. The process of bit-plane sequential encoding is to encode all bits starting from the highest bit plane of the bit plane of the updated quality value sequence to LSB (0) in a "horizontal" manner as shown in the figure. It should be noted that if some bits do not take values after the shift (such as marked as "X"), these bits are skipped. Figure 2B In the example shown in (2), the obtained compressed bit stream is "1001110110101010".

[0132] More preferably, for any quality value in any interval, the quality value can be mapped to an updated quality value based on a predetermined mapping rule, so that the maximum updated quality value in the interval where the quality value is located is less than or equal to the maximum quality value in the interval.

[0133] Figure 2C The diagram schematically shows a mapping relationship diagram of interval quality values according to an embodiment of the present disclosure.

[0134] like Figure 2C As shown in the figure, assuming that the quality values divided into the same interval include: {1, 4, 5, 6, 9, 10, 11, 12}, the minimum quality value q in the interval is min is 1, the maximum mass value q max The discrete mass values in this interval are not evenly distributed, and the mass values 2, 3, 7, and 8 are missing. The mass values in this interval are mapped to evenly distributed discrete mass values, and we get: {1, 2, 3, 4, 5, 6, 7, 8}. The minimum updated mass value is equal to the original minimum mass value q min Same, maximum updated quality value q' max is 8, q' max <q max .

[0135] It can be seen that according to this embodiment, a new smaller q' is adopted through the above mapping max Replace q max , thereby reducing the number of bit planes required for the binary representation of the quality value to further improve the compression efficiency. In addition, Figure 2C As shown, for each quality value in the original quality value range, a binary bit can be used to indicate whether the quality value actually exists (for example, in the above example, quality values 2, 3, 7, and 8 are marked as "0"). This binary bit sequence, used to indicate whether the original quality value actually exists, is losslessly encoded as auxiliary information and transmitted to the decoder. After receiving the compressed bit stream, the decoder can reconstruct the mapping relationship between the quality values in the range based on this binary bit sequence.

[0136] After mapping and converting the quality values divided into each interval to obtain corresponding updated quality values, shifting the bit plane of any quality value based on the priority score may include: shifting the bit plane of the updated quality value of any quality value based on the priority score.

[0137] Among them, as an optional embodiment, the above-mentioned shifting of the bit plane of the updated quality value of any quality value based on the priority score may include: for any interval, determining the maximum bit plane regarding any interval based on the maximum updated quality value in the any interval; for any updated quality value, calculating the binary data of any updated quality value, and taking the maximum bit plane of the interval to which the any updated quality value belongs as the highest bit of the binary data of the any updated quality value; shifting the bit plane of the binary data of any updated quality value based on the priority score of the quality value corresponding to the any updated quality value; and, the binary data of each updated quality value after the shift constitutes an updated quality value sequence.

[0138] In a specific embodiment, determining the maximum bit-plane for any interval based on the maximum update quality value within the interval may include: using the most significant bit of the maximum update quality value within the interval as the maximum bit-plane for the interval. Additionally or alternatively, calculating the binary data of any update quality value may include: converting the difference between the update quality value and the minimum update quality value within the interval to which the update quality value belongs into binary form to obtain the binary data corresponding to the update quality value.

[0139] On this basis, as an optional embodiment, the above-mentioned encoding of the updated quality value sequence based on the bit plane coding algorithm includes: scanning the updated quality value sequence in order from high to low according to the bit plane, and recording the values of the scanned bits in sequence, wherein if the value of the scanned bit does not exist, the bit is skipped.

[0140] The process of bit plane shifting and encoding the quality value after mapping transformation is the same as that described above. Figure 2B The logic of the process of bit-plane shifting and encoding the quality values that have not been mapped is the same as shown, and will not be repeated here. It should be noted that it has been verified that when bit-plane encoding is performed on the quality values after shifting, the expected values of the genotype error rate at different positions of the reference genome are the same. Therefore, the data compression method according to the embodiment of the present disclosure will not introduce higher error rates at other gene positions due to excessive allocation of coding bits to certain gene positions.

[0141] After encoding the updated quality value sequence using a bit-plane coding algorithm, a compressed bitstream of the quality value sequence in the sequencing data is obtained. This compressed bitstream is scalable and can be called a scalable bitstream. Part or all of the scalable bitstream and its associated context information are encoded into a quality data block. This quality data block is sent to the decoder, which decodes part or all of the scalable bitstream based on the context information.

[0142] Figure 2D An exemplary structural diagram of a quality data block according to an embodiment of the present disclosure is schematically shown.

[0143] like Figure 2DAs shown, the quality data block includes a compressed bitstream and context information, which can include header information and auxiliary information. The header information stores an integer representing the total number of bytes in the quality data block. The auxiliary information for lossless coding includes an auxiliary information block for the minimum / maximum interval quality value table, an auxiliary information block for the interval quality value mapping, and an auxiliary information block for the displacement table. These fields enable correct decoding of the compressed bitstream. The specific meaning of each auxiliary information block has been explained above and will not be repeated here.

[0144] Figure 3 A flowchart of a data decompression method according to an embodiment of the present disclosure is schematically shown. The data decompression method is used to Figure 2A The compressed bit stream obtained by the data compression method shown is decoded.

[0145] like Figure 3 As shown, the method includes the following operations S301 to S304.

[0146] In operation S301 , a compressed bit stream of a quality value sequence is obtained, where the quality value sequence is divided into one or more intervals, and the quality value sequence includes a plurality of quality values.

[0147] The specific meanings of the above-mentioned quality value sequence, compressed bit stream and one or more intervals have been explained above and will not be repeated here.

[0148] In operation S302 , a maximum bit-plane of any interval and a priority score of any quality value are obtained.

[0149] Among them, the priority score of a quality value represents the displacement of the bit plane of the quality value, and represents the degree of influence of the quality value on the genotype related to the sequencing data in which it is located. The specific meanings of the maximum bit plane of any interval and the priority score of any quality value have been explained above and will not be repeated here. The decoding end can be based on Figure 2D The auxiliary information block of the displacement table in the quality data block shown in FIG. 1 obtains the priority score or displacement amount of each quality value. The decoding end can obtain the priority score or displacement amount of each quality value according to the following example. Figure 2D The auxiliary information block of the interval quality value minimum / maximum table in the quality data block shown obtains the maximum bit plane of each interval.

[0150] In operation S303, the compressed bit stream is decoded based on the maximum bit-plane, the priority score, and a bit-plane coding algorithm to obtain the quality value sequence.

[0151] In operation S304 , sequencing data matching the quality value sequence is obtained based on the quality value sequence.

[0152] In this operation S304, based on the bases corresponding to the quality values in the encoded quality value sequence, a corresponding base sequence can be obtained. The quality value sequence and the base sequence can constitute corresponding sequencing data. The sequencing data is the sequencing data after alignment with the reference genome sequence, which is suitable for subsequent data analysis.

[0153] visible, Figure 3 The method shown in FIG. 1 describes the decoding process of the compressed bit stream from the decoding end side, which is similar to the decoding process of FIG. Figure 2A The data compression process shown in FIG. The data compression process performs bit-plane shifting on the quality values based on their impact on the genotype, and then bit-plane encoding on the bit-plane-shifted quality value sequence, resulting in a compressed bitstream of the quality value sequence. The transmission process of the compressed bitstream saves bandwidth, improves transmission efficiency, and enhances security. Furthermore, in this compressed bitstream, the more forward bits carry information about quality values that have a greater impact on the genotype. This way, even if the decoding end only receives a portion of the compressed bitstream, important information that affects the genotype is preferentially retained. Therefore, based on this compressed bitstream, relatively accurate sequencing data can be restored without affecting subsequent genomic data processing and analysis processes.

[0154] In one embodiment of the present disclosure, if all compressed bit streams are obtained, the quality value sequence can be losslessly reconstructed at the decoding end. The above-mentioned acquisition of the compressed bit stream of the quality value sequence may include: acquiring all compressed bit streams of the quality value sequence. Then the above-mentioned process of decoding the compressed bit stream based on the maximum bit plane, the priority score, and the bit plane coding algorithm may include: for any quality value, first determine the bit distribution of any quality value based on the maximum bit plane of each interval, the maximum bit plane of the interval to which the any quality value belongs, and the priority score of the any quality value, and then, based on the bit distribution, fill the values of the compressed bit stream into each bit in sequence according to the bit plane from high to low to obtain a quality value sequence.

[0155] In another embodiment of the present disclosure, if a partial compressed bit stream is obtained, the quality value can be losslessly reconstructed based on the predicted value of the quality value, thereby obtaining a quality value sequence. The above-mentioned obtaining of the compressed bit stream of the quality value sequence may include: obtaining the first predetermined number of bits of the compressed bit stream of the quality value sequence. Then the above-mentioned decoding of the compressed bit stream based on the maximum bit plane, the priority score, and the bit plane coding algorithm may include: for any quality value, first determining the bit distribution of any quality value based on the maximum bit plane of each interval, the maximum bit plane of the interval to which the any quality value belongs, and the priority score of the any quality value; then, based on the bit distribution, filling the values of the first predetermined number of bits into each bit in order from high to low in the bit plane; then, for any quality value, if there is a bit without a value, reconstruct the any quality value to obtain a quality value sequence.

[0156] Optionally, the above-mentioned reconstruction of any quality value may include: for any quality value, setting the value of all bits without a value in any quality value to 0, thereby obtaining a first value of the quality value. Setting the value of all bits without a value in any quality value to 1, thereby obtaining a second value of the quality value. Then, reconstructing the quality value in one of the following ways: (1) using the first value as the reconstructed value of any quality value; or, (2) using the second value as the reconstructed value of any quality value; or, (3) using the average of the first value and the second value as the reconstructed value of any quality value.

[0157] For example, if the maximum bit plane of the compressed bit stream is M, when k bit planes are received (k<M), for any quality value q, the value range of the quality value q is [q1,q r ], the minimum quality value of the interval to which the quality value q belongs is q min ,but

[0158]

[0159]

[0160] In this case, it is assumed that the quality value q is in the range [q1,q r ] has a uniform posterior probability distribution, and the quality value q can be reconstructed according to the minimum mean square error, so that Right now

[0161]

[0162] In addition, the data decompression method according to the embodiment of the present disclosure also supports reconstructing the quality value q in other ways. For example, (optimistic reconstruction) or (Pessimistic reconstruction).

[0163] As mentioned above, when compressing the quality value sequence, sometimes the mapping transformation of the quality values in each interval is performed. To cope with this situation, in one embodiment of the present disclosure, the data decompression method according to the embodiment of the present disclosure may further include: after obtaining the quality value sequence, obtaining a predetermined mapping rule, for example, Figure 2D A predetermined mapping rule is obtained from a mapping information block for interval quality values in the quality data block shown; and based on the predetermined mapping rule, the quality value sequence is reversely mapped into the original quality value sequence. Obtaining sequencing data matching the quality value sequence includes obtaining sequencing data matching the original quality value sequence. The quality value mapping conversion process has been described above; the reverse mapping process is logically opposite and will not be further described here.

[0164] In order to speed up the encoding and decoding speed, the solution according to the embodiment of the present disclosure can adopt a fixed bit plane displacement method. For example, for each short read sequence, the priority score of the quality value corresponding to the site of "X" and "I" on the reference genome sequence can be pre-set, such as directly shifting upward by 6 bit planes. The priority scores of the quality values corresponding to other sites are set to default values, that is, they are not improved. Then, according to the simplified priority, all quality values of all short read sequences are bit-plane encoded, transmitted, and subsequently decoded.

[0165] Figure 4 The block diagram of the data compression device according to the embodiment of the present disclosure is schematically shown.

[0166] like Figure 4 As shown, the data compression device 400 includes: a first acquisition module 401 , a second acquisition module 402 , a displacement module 403 , and an encoding module 404 .

[0167] The first acquisition module 401 is used to obtain a quality value sequence in sequencing data.

[0168] The second acquisition module 402 is configured to acquire a priority score of any quality value in the quality value sequence, where the priority score represents the degree of influence of any quality value on a genotype associated with the sequencing data.

[0169] The shift module 403 is configured to shift the bit plane of any quality value based on the priority score to obtain an updated quality value sequence.

[0170] The encoding module 404 is configured to encode the updated quality value sequence based on a bit-plane coding algorithm to obtain a compressed bit stream of the quality value sequence.

[0171] Figure 5 The figure schematically shows a block diagram of a data compression device according to another embodiment of the present disclosure.

[0172] like Figure 5 As shown, the data compression device 500 includes: a first acquisition module 501, a second acquisition module 502, a displacement module 503, and an encoding module 504. The first acquisition module 501, the second acquisition module 502, the displacement module 503, and the encoding module 504 respectively have the same functions as the first acquisition module 401, the second acquisition module 402, the displacement module 403, and the encoding module 404, and the repeated parts are not repeated here.

[0173] In one embodiment of the present disclosure, the second acquisition module 502 includes: a setting submodule 5021 , a first calculation submodule 5022 , a second calculation submodule 5023 , and a third calculation submodule 5024 .

[0174] The setting submodule 5021 is used to set a first deviation and a second deviation for any of the quality values. The first calculation submodule 5022 is used to calculate a first probability of a genotype associated with the sequencing data when any of the quality values is increased by the first deviation. The second calculation submodule 5023 is used to calculate a second probability of a genotype associated with the sequencing data when any of the quality values is decreased by the second deviation. The third calculation submodule 5024 is used to obtain a priority score for any of the quality values based on the first deviation, the second deviation, the first probability, and the second probability.

[0175] Optionally, the process of the first calculation submodule 5022 calculating the first probability of the genotype associated with the sequencing data and the process of the second calculation submodule 5023 calculating the second probability of the genotype associated with the sequencing data can specifically be to calculate the posterior probability of the genotype associated with the sequencing data based on the Bayesian algorithm.

[0176] As an optional embodiment, the third calculation submodule 5024 is specifically used to: calculate the third deviation between the first probability and the second probability; take the sum of the first deviation and the second deviation as the fourth deviation; and obtain the priority score of any quality value based on the ratio of the third deviation and the fourth deviation.

[0177] As an optional embodiment, the process in which the first calculation submodule 5022 calculates the first probability of the genotype associated with the sequencing data and the second calculation submodule 5023 calculates the second probability of the genotype associated with the sequencing data can specifically be: when the genotype associated with the sequencing data includes multiple genotypes, the first probability and the second probability of any one of the multiple genotypes are calculated respectively.

[0178] The third calculation submodule 5024 is specifically used to first calculate the third deviation between the first probability and the second probability of any genotype, and use the sum of the first deviation and the second deviation as the fourth deviation. Then, the priority score is calculated in one of the following ways: (1) the priority score of any quality value is obtained based on the ratio of the mean of the third deviation corresponding to each genotype in the plurality of genotypes to the fourth deviation; or (2) when the third deviation corresponding to any genotype is greater than the third deviation corresponding to other genotypes in the plurality of genotypes, the priority score of any quality value is obtained based on the ratio of the third deviation corresponding to the genotype to the fourth deviation.

[0179] In one embodiment of the present disclosure, the displacement module 503 includes: a first determination submodule 5031 and a first displacement submodule 5032 .

[0180] The first determination submodule 5031 is configured to, when the priority score is less than or equal to zero, determine the shift amount to be 0; when the priority score is greater than zero and less than a first value, determine the shift amount to be the rounded result of the priority score; and when the priority score is greater than or equal to the first value, determine the shift amount to be the first value. The first shift submodule 5032 is configured to shift the bit plane of any quality value by the shift amount to obtain the updated quality value sequence.

[0181] In one embodiment of the present disclosure, sequencing data includes a plurality of short read sequences arranged in a predetermined order, the short read sequences include a plurality of quality values, and the quality values in the plurality of short read sequences constitute the quality value sequence.

[0182] Furthermore, as an optional embodiment, the data compression device 500 further includes: an interval division module 505 and a mapping module 506 .

[0183] The interval division module 505 is used to divide the interval for any quality value according to the attribute information of any quality value before the displacement module 503 shifts the bit plane of any quality value based on the priority score. The attribute information includes at least one of the following: (1) the alignment direction of the short read sequence where any quality value is located is forward alignment or reverse alignment; (2) the short read sequence where any quality value is located is the first sequence or the second sequence; and (3) the position of any quality value on the short read sequence where it is located. The mapping module 506 is used to map any quality value in any interval to an updated quality value based on a predetermined mapping rule, so that the maximum updated quality value in any interval is less than or equal to the maximum quality value in any interval. On this basis, the displacement module 503 is specifically used to shift the bit plane of the updated quality value based on the priority score.

[0184] As an optional embodiment, the displacement module 503 includes: a second determination submodule 5033 , a fourth calculation submodule 5034 , a second displacement submodule 5035 , and a reconstruction submodule 5036 .

[0185] The second determination submodule 5033 is configured to determine, for each of the intervals, the maximum bit plane associated with the interval based on the maximum updated quality value within the interval. The fourth calculation submodule 5034 is configured to calculate, for each of the update quality values, the binary data of the update quality value, using the maximum bit plane of the interval to which the update quality value belongs as the most significant bit of the binary data. The second shift submodule 5035 is configured to shift the bit planes of the binary data based on the priority score of the quality value corresponding to the update quality value. Furthermore, the reconstruction submodule 5036 is configured to construct the updated quality value sequence from the shifted binary data of each update quality value.

[0186] On this basis, as an optional embodiment, the encoding module 504 is specifically used to scan the updated quality value sequence in order from high to low in the bit plane, and record the values of the scanned bits in sequence, wherein if the value of the scanned bit does not exist, the bit is skipped.

[0187] As an optional embodiment, the second determination submodule 5033 is specifically configured to use the most significant bit of the maximum update quality value as the maximum bit plane. And / or, the fourth calculation submodule 5034 is specifically configured to convert the difference between any update quality value and the minimum update quality value within the interval to which the any update quality value belongs into binary form to obtain the binary data.

[0188] Figure 6The block diagram of the data decompression device according to the embodiment of the present disclosure is schematically shown.

[0189] like Figure 6 As shown, the data decompression device 600 includes: a third acquisition module 601 , a fourth acquisition module 602 , a decoding module 603 , and a matching module 604 .

[0190] The third acquisition module 601 is used to acquire a compressed bit stream of a quality value sequence, where the quality value sequence is divided into one or more intervals and includes multiple quality values.

[0191] The fourth acquisition module 602 is configured to acquire the maximum bit-plane of any interval and the priority score of any quality value.

[0192] The decoding module 603 is configured to decode the compressed bit stream based on the maximum bit plane, the priority score, and a bit plane coding algorithm to obtain the quality value sequence.

[0193] The matching module 604 is configured to obtain sequencing data that matches the quality value sequence based on the quality value sequence, wherein the priority score represents the degree of influence of any quality value on the genotype associated with the sequencing data.

[0194] Figure 7 A block diagram of a data decompression device according to another embodiment of the present disclosure is schematically shown.

[0195] like Figure 7 As shown, the data decompression device 700 includes: a third acquisition module 701, a fourth acquisition module 702, a decoding module 703, and a matching module 704. The third acquisition module 701, the fourth acquisition module 702, the decoding module 703, and the matching module 704 respectively have the same functions as the third acquisition module 601, the fourth acquisition module 602, the decoding module 603, and the matching module 604, and the repeated parts are not repeated here.

[0196] In one embodiment of the present disclosure, the third acquisition module 701 includes a first acquisition submodule 7011, which is specifically configured to acquire all compressed bit streams of the quality value sequence.

[0197] The decoding module 703 includes a first determination submodule 7031 and a first restoration submodule 7032. The first determination submodule 7031 is configured to determine, for any quality value, the bit distribution of the quality value based on the maximum bit-plane of each interval, the maximum bit-plane of the interval to which the quality value belongs, and the priority score of the quality value. Furthermore, the first restoration submodule 7032 is configured to, based on the bit distribution, sequentially fill the compressed bitstream values into each bit in descending order of bit-planes to obtain the quality value sequence.

[0198] In one embodiment of the present disclosure, the third acquisition module 701 includes a second acquisition submodule 7012, which is specifically configured to acquire the first predetermined number of bits of the compressed bit stream of the quality value sequence.

[0199] The decoding module 703 includes a second determination submodule 7033, a second recovery submodule 7034, and a padding submodule 7035. The second determination submodule 7033 is configured to determine, for any quality value, the bit distribution of the quality value based on the maximum bit-plane of each interval, the maximum bit-plane of the interval to which the quality value belongs, and the priority score of the quality value. The second recovery submodule 7034 is configured to sequentially fill the values of the first predetermined number of bits into each bit in descending order of bit-planes based on the bit distribution. Furthermore, the padding submodule 7035 is configured to reconstruct any quality value, if any bit has no value, to obtain the quality value sequence.

[0200] Specifically, as an optional embodiment, the filling submodule 7035 is specifically configured to, for any quality value, first, set the value of all bits without a value in the quality value to 0, thereby obtaining a first value of the quality value; and then set the value of all bits without a value in the quality value to 1, thereby obtaining a second value of the quality value. Next, reconstruct the quality value in one of the following ways: (1) using the first value as the reconstructed value of the quality value; or, (2) using the second value as the reconstructed value of the quality value; or, (3) using the average of the first value and the second value as the reconstructed value of the quality value.

[0201] Furthermore, in one embodiment of the present disclosure, the data decompression apparatus 700 further includes a reverse mapping module 705 configured to, after obtaining the quality value sequence, obtain a predetermined mapping rule; and reversely map the quality value sequence to an original quality value sequence based on the predetermined mapping rule. The matching module 704 is specifically configured to obtain sequencing data that matches the original quality value sequence.

[0202] It should be noted that the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each module / unit / sub-unit in the device part embodiment are the same or similar to the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0203] According to the modules, submodules, units, and subunits of the embodiments of the present invention, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be split into multiple modules for implementation. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in an appropriate combination of any of them. Alternatively, according to the modules, submodules, units, and subunits of the embodiments of the present invention, one or more thereof can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0204] For example, any multiple of the first acquisition module 501, the second acquisition module 502, the displacement module 503, the encoding module 504, the interval division module 505, and the mapping module 506 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first acquisition module 501, the second acquisition module 502, the displacement module 503, the encoding module 504, the interval division module 505, and the mapping module 506 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 501, the second acquisition module 502, the displacement module 503, the encoding module 504, the interval division module 505, and the mapping module 506 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0205] Figure 8 The block diagram schematically shows a computer system suitable for implementing the above-described method according to an embodiment of the present disclosure. Figure 8 The computer system shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0206] like Figure 8 As shown, the computer system 800 includes a processor 810 and a computer-readable storage medium 820. The computer system 800 can execute the method according to the embodiment of the present disclosure.

[0207] Specifically, the processor 810 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 810 may also include onboard memory for cache purposes. The processor 810 may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiments of the present disclosure.

[0208] The computer-readable storage medium 820 may be, for example, a non-volatile computer-readable storage medium, specific examples of which include but are not limited to: magnetic storage devices, such as magnetic tapes or hard disks (HDDs); optical storage devices, such as compact disks (CD-ROMs); memories, such as random access memories (RAMs) or flash memories; and the like.

[0209] The computer-readable storage medium 820 may include a computer program 821 , which may include code / computer-executable instructions that, when executed by the processor 810 , cause the processor 810 to perform a method according to an embodiment of the present disclosure or any variation thereof.

[0210] Computer program 821 may be configured to include, for example, computer program code comprising computer program modules. For example, in an exemplary embodiment, the code in computer program 821 may include one or more program modules, such as module 821A, module 821B, and so on. It should be noted that the division method and number of modules are not fixed, and those skilled in the art may use appropriate program modules or combinations of program modules according to actual circumstances. When these program module combinations are executed by processor 810, processor 810 may perform the method according to the embodiments of the present disclosure or any variation thereof.

[0211] According to an embodiment of the present invention, at least one of the first acquisition module 501, the second acquisition module 502, the displacement module 503, the encoding module 504, the interval division module 505, and the mapping module 506 can be implemented as a reference Figure 8The computer program module described herein, when executed by the processor 810, can implement the data compression method and the data decompression method described above.

[0212] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0213] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0214] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0215] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, even if such combinations or combinations are not explicitly described in this disclosure. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0216] Although the present disclosure has been shown and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made to the present disclosure without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the above-described embodiments, but should be determined not only by the appended claims but also by the equivalents of the appended claims.

Claims

1. A data compression method, comprising: Obtain quality value sequences in sequencing data; Obtaining a priority score for any quality value in the quality value sequence, wherein the priority score represents a degree of influence of any quality value on a genotype associated with the sequencing data; shifting the bit plane of any quality value based on the priority score to obtain an updated quality value sequence; and encoding the updated quality value sequence based on a bit plane coding algorithm to obtain a compressed bit stream of the quality value sequence, Wherein, the obtaining the priority score of any quality value in the quality value sequence includes: setting a first deviation and a second deviation for any quality value; When any of the quality values is increased by the first deviation amount, calculating a first probability of a genotype associated with the sequencing data; When any one of the quality values is reduced by the second deviation amount, a second probability of a genotype associated with the sequencing data is calculated; and a priority score of the any one of the quality values is obtained based on the first deviation amount, the second deviation amount, the first probability, and the second probability.

2. The method according to claim 1, wherein The calculating a first probability of a genotype associated with the sequencing data and the calculating a second probability of a genotype associated with the sequencing data include: calculating a posterior probability of a genotype associated with the sequencing data based on a Bayesian algorithm.

3. The method according to claim 1, wherein The obtaining of the priority score of any quality value comprises: calculating a third deviation between the first probability and the second probability; The sum of the first deviation and the second deviation is used as a fourth deviation; and the priority score of any quality value is obtained based on the ratio of the third deviation to the fourth deviation.

4. The method according to claim 1, wherein The calculating the first probability of the genotype associated with the sequencing data and the calculating the second probability of the genotype associated with the sequencing data include: when the genotype associated with the sequencing data includes a plurality of genotypes, respectively calculating the first probability and the second probability of any one of the plurality of genotypes; Obtaining the priority score of any quality value includes: Calculating a third deviation between the first probability and the second probability of any one of the genotypes; The sum of the first deviation and the second deviation is taken as the fourth deviation; and the priority score of any quality value is obtained based on the ratio of the mean of the third deviation corresponding to each genotype in the multiple genotypes to the fourth deviation, or, when the third deviation corresponding to any genotype is greater than the third deviation corresponding to other genotypes in the multiple genotypes, the priority score of any quality value is obtained based on the ratio of the third deviation corresponding to any genotype to the fourth deviation.

5. The method according to claim 1, wherein The shifting the bit plane of any quality value based on the priority score includes: when the priority score is less than or equal to zero, determining the shift amount to be 0; When the priority score is greater than zero and less than a first value, determining the displacement amount to be a rounded result of the priority score; When the priority score is greater than or equal to the first value, the displacement is determined to be the first value; and the bit plane of any quality value is shifted by the displacement to obtain the updated quality value sequence.

Images