Intelligent compression system and method for sequencing data

Abstract

The application discloses an intelligent compression system and method for sequencing data, and relates to the field of intelligent compression. The system first acquires a plurality of short gene sequences containing ID, Base and Quality data, and then compresses the DNA sequencing data to be stored. By matching the short gene sequences to be compressed with spliced sequences, if the Base data are all successfully matched, the short gene sequences are directly compressed; if part of the Base data are matched, the short gene sequences are split into matched and unmatched sequences for compression. For the unmatched sequences, the sequences are further split into a subsequence set, and a refined subset is obtained based on group feature distribution guided sequence selection, and finally, the refined subset is compressed and encoded. The method effectively improves the compression rate, especially when processing the unmatched sequences, the method reduces redundant information through fine screening, improves the overall storage efficiency and data retrieval speed, and enhances the effective mining capacity for long-range redundancy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent compression, and more specifically, to an intelligent compression system and method for sequencing data. BACKGROUND

[0002] With the rapid development of high-throughput sequencing technology, the scale of DNA sequencing data generation presents exponential growth. A single sequencing experiment can produce hundreds of GB to TB level of raw data, which contains a large number of short gene sequences. These data are usually composed of three parts of core information: sequence identification (ID data), base sequence (Base data) and sequencing quality score (Quality data). In order to reduce the storage and transmission cost, data compression has become a key technical requirement in the sequencing data analysis process. However, the existing compression method may cause the whole sequence to be unable to be effectively compressed due to the difference of individual positions when processing short gene sequences (reads), which affects the overall compression efficiency.

[0003] Therefore, patent CN118522351A proposes a DNA sequencing data storage method and system based on artificial intelligence, which proposes a reference-free compression model based on sequence assembly for efficient compression of DNA sequencing data. First, a plurality of short gene sequences containing ID, Base and Quality data are obtained, and the reference-free compression model is used for matching and compression in the splicing process. When the Base data of the short gene sequence is completely matched, the offset and length are recorded to realize compression; when partially matched, the matched and unmatched sequences are split for compression, thereby effectively improving the overall compression rate and optimizing the storage efficiency.

[0004] The patent uses ASCII compression when compressing the unmatched sequence. However, the Base data itself only contains 4 base symbols (A / T / C / G), but when the traditional method uses ASCII encoding, each base character occupies 1 byte (8 bits) of storage space, and the actual information density is extremely low (only 2 bits are needed to encode 4 states), directly causing inherent waste of storage space. Although general compression algorithms (such as gzip) can compress repeated ASCII characters, the dictionary model they rely on is difficult to identify the biological characteristics of base sequences (such as local overlap, repeated fragments or variation patterns), resulting in insufficient mining ability for long-range redundancy. In addition, the core goal of compression is to remove redundancy, and for the unmatched sequence segment, although it does not match the "spliced sequence, there is still redundancy (small character set) inside, and ASCII encoding fails to utilize this internal redundancy, which is only a direct representation (or close to direct representation) of the original data, without realizing effective compression of this part of data.

[0005] Therefore, an intelligent compression scheme for sequencing data is desired. SUMMARY

[0006] To solve the above technical problems, the present application is proposed.

[0007] According to an aspect of the present application, a method for intelligent compression of sequencing data is provided, comprising:

[0008] obtaining to-be-stored DNA sequencing data containing a plurality of short gene sequences, the short gene sequences including ID data, Base data and Quality data;

[0009] performing compression processing on the to-be-stored DNA sequencing data, comprising: extracting to-be-compressed short gene sequences from the to-be-stored DNA sequencing data and obtaining a spliced sequence; matching the to-be-compressed short gene sequences with the spliced sequence; in response to the Base data of the to-be-compressed short gene sequences being completely matched successfully, compressing the Base data of the to-be-compressed short gene sequences; in response to the Base data of the to-be-compressed short gene sequences being partially matched successfully, splitting the Base data of the to-be-compressed short gene sequences into matched sequences and unmatched sequences and then compressing them respectively;

[0010] storing the compressed DNA sequencing data;

[0011] wherein, the compression of the unmatched sequences comprises: sequence segmentation of the unmatched sequences to obtain a set of unmatched subsequences; population feature distribution guided sequence selection of the set of unmatched subsequences to obtain a selected subset of unmatched subsequences; compression encoding of the selected subset of unmatched subsequences.

[0012] According to another aspect of the present application, a system for intelligent compression of sequencing data is provided, comprising:

[0013] a sequencing data acquisition module for obtaining to-be-stored DNA sequencing data containing a plurality of short gene sequences, the short gene sequences including ID data, Base data and Quality data;

[0014] a sequencing data processing module for performing compression processing on the to-be-stored DNA sequencing data, wherein the sequencing data processing module is configured to: extract to-be-compressed short gene sequences from the to-be-stored DNA sequencing data and obtain a spliced sequence; match the to-be-compressed short gene sequences with the spliced sequence; in response to the Base data of the to-be-compressed short gene sequences being completely matched successfully, compress the Base data of the to-be-compressed short gene sequences; in response to the Base data of the to-be-compressed short gene sequences being partially matched successfully, split the Base data of the to-be-compressed short gene sequences into matched sequences and unmatched sequences and then compress them respectively;

[0015] a compression module, configured to store the compressed DNA sequencing data.

[0016] The sequencing data processing module is further configured to: perform sequence segmentation on the unmatched sequence to obtain a set of unmatched subsequences; perform population feature distribution guided sequence selection on the set of unmatched subsequences to obtain a selected subset of the unmatched subsequences; and perform compression encoding on the selected subset of the unmatched subsequences.

[0017] Compared with the prior art, the intelligent compression system and method for sequencing data provided by the application first obtains a plurality of short gene sequences containing ID, Base and Quality data, and then performs compression processing on the DNA sequencing data to be stored. By matching the short gene sequence to be compressed with the spliced sequence, if the Base data is all matched successfully, the short gene sequence is directly compressed; if part of the Base data is matched, the short gene sequence is split into matched and unmatched sequences for compression. For the unmatched sequence, the unmatched sequence is further segmented into a set of subsequences, and a selected subset of the set of subsequences is obtained through population feature distribution guided sequence selection, and finally the selected subset is compressed and encoded. This method effectively improves the compression ratio, especially when processing the unmatched sequence, through fine selection, the redundant information is reduced, the overall storage efficiency and data retrieval speed are improved, and the effective mining ability of long-range redundancy is also enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application, taken in conjunction with the accompanying drawings. The drawings provided in the specification and the embodiments of the present application together serve to provide a further understanding that enables others skilled in the art to make or use the present application. The drawings provided are for illustrative purposes and are not intended to limit the present application. In the drawings, the same reference numerals generally refer to the same components or steps throughout the drawings.

[0019] Figure 1 A flowchart of the intelligent compression method for sequencing data according to the embodiments of the present application.

[0020] Figure 2 A flowchart of the compression of the unmatched sequence in the intelligent compression method for sequencing data according to the embodiments of the present application.

[0021] Figure 3 A flowchart of the population feature distribution guided sequence selection on the set of unmatched subsequences to obtain a selected subset of the unmatched subsequences in the intelligent compression method for sequencing data according to the embodiments of the present application.

[0022] Figure 4A flow chart of individual and population distribution sequence selection on the set of encoding vectors of the unmatched sub-sequences to obtain the set of encoding vectors of the unmatched sub-sequences as the selected subset of the unmatched sub-sequences in the intelligent compression method of the sequencing data according to the embodiments of the present application.

[0023] Figure 5 A block diagram of the intelligent compression system of the sequencing data according to the embodiments of the present application. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be understood that each step described in the method embodiments of the present disclosure can be performed in different orders, and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0026] Based on this, in the technical solutions of the present application, an intelligent compression method of sequencing data is proposed. Figure 1 A flow chart of the intelligent compression method of the sequencing data according to the embodiments of the present application. As shown in Figure 1 The intelligent compression method of the sequencing data according to the embodiments of the present application includes: S110, obtaining to-be-stored DNA sequencing data containing a plurality of short gene sequences, the short gene sequences including ID data, Base data and Quality data; S120, performing compression processing on the to-be-stored DNA sequencing data; S130, storing the compressed DNA sequencing data.

[0027] In step S110, the DNA sequencing data to be stored containing a plurality of short gene sequences is acquired, the short gene sequences including ID data, Base data and Quality data. In particular, the short gene sequences are sequencing read results of a fragment of the original DNA chain corresponding to the DNA sequencing data to be stored. It should be understood that the short gene sequences contain a variety of key information such as ID data, Base data and Quality data. The ID data is used to identify each short gene sequence, facilitating differentiation and tracking in the process of data processing and analysis; the Base data records the base arrangement order of the gene, which is the core information for studying the function, variation, etc. of the gene; the Quality data reflects the quality of each base sequencing, which is helpful for evaluating the reliability of the data. Only when the complete short gene sequences containing these information are acquired, can comprehensive and accurate data analysis and processing be performed to meet the needs of subsequent compression and other related researches.

[0028] In the process of DNA sequencing, since the DNA molecule is usually very long, even the third-generation sequencing technology cannot cover the genetic information of the entire genome. Therefore, it is necessary to break the DNA chain into fragments, and the length of these fragments is generally 150-300 bp. Sequencing is performed for each fragment, and the sequencing read data obtained is the short gene sequence.

[0029] In actual operation, the shotgun method is usually used to cut the gene fragments into a plurality of short gene sequences with shorter length, and all these short gene sequences constitute a short gene sequence sample library. The result of DNA sequencing is generally presented in the FASTQ file format, and the FASTQ file is composed of a plurality of read data, each read data being a short gene sequence.

[0030] Taking specific FASTQ file data as an example, “GTACGGGGGAATTCTCCAGATTACTTCCATTTCCGCCCAAGCT” corresponding to “@SRR6790711.11 / 1” is the Base data, the first line “@SRR6790711.11 / 1” is the ID data, and the fourth line of quality score sequence is the Quality data. Extracting these information from the FASTQ file obtains the short gene sequences containing ID data, Base data and Quality data, thereby achieving the acquisition of the DNA sequencing data to be stored containing a plurality of short gene sequences.

[0031] In step S120, the to-be-stored DNA sequencing data is compressed. Accordingly, the DNA sequencing technology develops rapidly, and the data output increases explosively. If such a large amount of data is not compressed, the cost of hardware equipment required for storage will be extremely high, and in the data transmission process, a large amount of network bandwidth will be occupied, resulting in low transmission efficiency and a substantial increase in cost. Moreover, the DNA sequencing data has redundancy. On the one hand, the internal genome sequence has high repeatability, and there are identical or similar fragments in a large number of short gene sequences; on the other hand, the ID data and Quality data in the FASTQ file have relatively low redundancy, but there is a large amount of redundant information that can be mined in the Base data. For example, there may be repeated base sequence fragments in the Base data of different short gene sequences, which provides the possibility for data compression.

[0032] Specifically, in the embodiment of the present application, step S120, the to-be-stored DNA sequencing data is compressed, comprising: extracting a to-be-compressed short gene sequence from the to-be-stored DNA sequencing data, and obtaining a spliced sequence; matching the to-be-compressed short gene sequence with the spliced sequence; in response to the Base data of the to-be-compressed short gene sequence being completely matched successfully, compressing the Base data of the to-be-compressed short gene sequence; in response to the Base data of the to-be-compressed short gene sequence being partially matched successfully, splitting the Base data of the to-be-compressed short gene sequence into matched sequences and unmatched sequences for compression respectively.

[0033] More specifically, the to-be-compressed short gene sequence is extracted from the to-be-stored DNA sequencing data, and a spliced sequence is obtained. It should be understood that the DNA sequencing data contains a large number of short gene sequences, which are the sequencing read results of the original DNA chain fragments. If these data are not effectively processed, the storage and transmission costs will be difficult to bear. Moreover, the data has redundancy and has compression potential, so it is necessary to extract the short gene sequence from the overall data for targeted compression. The spliced sequence is a sequence formed by continuously splicing the uncompressed short gene sequences when the to-be-stored DNA sequencing data is compressed. Specifically, in the compression process, each short gene sequence is continuously compressed and spliced, and this spliced sequence is recorded. It serves as a reference for the matching of subsequent to-be-compressed short gene sequences. In the sequence splicing process, the short gene sequences are continuously spliced to form longer and longer spliced sequences, providing a basis for the matching and compression of subsequent short gene sequences.

[0034] In particular, in one specific example of the present application, the method for obtaining the stitched sequence is as follows: in the initial state, the stitched sequence (DB) is empty. When processing the first short gene sequence (read1), its Base data is "ATCGATCG". Since there is no data in the stitched sequence (DB) at this time, no matching segment can be found, so the complete Base data "ATCGATCG" of read1 is directly appended to DB, and the updated stitched sequence (DB) becomes "ATCGATCG".

[0035] Subsequently, the second short gene sequence (read2) is processed, and its Base data is "ATCGTTTA". In the matching process, it is found that the first 4 characters "ATCG" of read2 completely match the "ATCG" at the beginning of the stitched sequence (DB), but the subsequent "TTTA" part cannot be found in the stitched sequence (DB). According to the processing rule, the matched "ATCG" part records its position information in the stitched sequence (DB) (offset = 0, matching length length = 4), and the unmatched "TTTA" is appended to the end of DB after being compressed by encoding. The updated DB is extended to "ATCGATCGTTTA".

[0036] This process embodies the dynamic generation mechanism of the stitched sequence (DB): by gradually accumulating the unmatched Base data segments, a constantly expanding reference sequence library is constructed, thereby providing a matching basis for repeated patterns for subsequent data compression. The unmatched part in each newly processed short gene sequence is appended to the end of the stitched sequence (DB), so that the stitched sequence (DB) can adaptively cover the characteristics of the input data, and finally achieve efficient compression.

[0037] More specifically, the short gene sequence to be compressed is matched with the stitched sequence. Accordingly, the amount of DNA sequencing data is huge, and the storage and transmission cost is high. There is a large amount of redundant information in these data, especially in the repeated regions between short gene sequences. Matching the short gene sequence to be compressed with the stitched sequence can find and utilize these redundancies to better compress and store the sequence.

[0038] More specifically, in response to the Base data of the short gene sequence to be compressed being completely matched, the Base data of the short gene sequence to be compressed is compressed. It can be understood that there is a large amount of redundant information in the DNA sequencing data, and many Base data of short gene sequences are repeated. When the Base data of the short gene sequence to be compressed is completely matched, it means that this part of data has been recorded in the stitched sequence, which has the condition to use a special compression method, thereby avoiding repeated storage and reducing the amount of data.

[0039] Specifically, in the process of handling the DNA sequencing data containing multiple short gene sequences to be stored, if the Base data of the short gene sequence to be compressed is matched successfully in its entirety, the compression process is closely centered around the assembled sequence. The assembled sequence is continuously assembled from the uncompressed short gene sequences during the compression process, serving as the key reference for the subsequent matching of the short gene sequence to be compressed.

[0040] When it is determined that the Base data of the short gene sequence to be compressed is matched successfully in its entirety with the assembled sequence, the offset and length of the Base data in the assembled sequence are recorded. The offset represents the starting position of the Base data in the assembled sequence, and the length indicates the continuous length of the Base data in the assembled sequence. To illustrate with a specific example, if the assembled sequence is “ATCGATCG” and the Base data “ATCGATCG” of the new short gene sequence to be compressed is completely matched, the offset of the Base data in the assembled sequence is recorded as 0 (because the matching starts from the beginning of the assembled sequence), and the length is recorded as 8 (the number of bases of “ATCGATCG”). This recording method is simple and efficient, as it avoids the repeated storage of the completely matched Base data and greatly reduces the data volume by recording the offset and length.

[0041] In the storage and subsequent data processing stage, the recorded offset and length information can be used to accurately locate and restore the corresponding Base data. This compression method fully utilizes the redundancy of short gene sequences in DNA sequencing data, effectively improving the storage efficiency of data. Compared with traditional methods, it avoids the waste of space caused by repeated storage of the same Base data, providing a more optimal solution for the storage and management of large-scale DNA sequencing data, and helping to improve the efficiency of the entire DNA sequencing data analysis process and promote the development of related fields such as bioinformatics research.

[0042] More specifically, in response to the partial matching success of the Base data of the short gene sequence to be compressed, the Base data of the short gene sequence to be compressed is split into matched sequences and unmatched sequences for compression. Accordingly, since traditional compression is performed in units of short gene sequences (reads), if there are local differences in the base sequence of the short gene sequence, the entire short gene sequence may not be effectively compressed. Splitting and processing the partially matched Base data can break through this limitation of using reads as the smallest unit, and adopt more appropriate compression methods for data with different matching conditions, improving the overall compression efficiency. In particular, by splitting the partially matched Base data, the offset and length of the matched sequence in the assembled sequence are recorded and compressed.

[0043] Specifically, when the Base data part of the short gene sequence to be compressed is matched successfully, the longest matching sequence in the Base data is determined first. Based on the matching result of the Base data of the short gene sequence to be spliced and the spliced sequence, the sequence composed of the matched data in the Base data is taken as a candidate sequence, and the length thereof is recorded.

[0044] Next, the first character of the current Base data is removed to obtain new Base data, which is matched with the spliced sequence. If the length of the sequence composed of the matched data in the new Base data is greater than the length of the current candidate sequence, the candidate sequence and the length thereof are updated; otherwise, they remain unchanged. This process is repeated until a preset iteration termination condition is met, such as the length of the new Base data being less than the initial length of the candidate sequence, or the difference between the length of the new Base data and the length of the sequence composed of the matched data therein being less than a preset threshold (the preset threshold is 3 in the document). At this time, the candidate sequence is the longest matching sequence.

[0045] After the longest matching sequence is determined, the data corresponding to the longest matching sequence in the Base data of the short gene sequence to be spliced is divided into a matched sequence, and the remaining data is divided into an unmatched sequence. For the matched sequence, the offset and length thereof in the spliced sequence are recorded for compression. This is because the matched sequence has been recorded in the spliced sequence. By recording the offset and length, repeated storage is avoided, and efficient compression is achieved.

[0046] More specifically, in the embodiments of the present application, the unmatched sequence is compressed. It should be understood that the unmatched part, although not completely identical to the spliced sequence, also has compression space due to its small character set. After splitting and compression, these redundant information can be more accurately mined and utilized, and more efficient compression can be achieved.

[0047] Thus, in view of the technical problems in the background art, the technical concept of the present application is to introduce a one-hot encoding and population distribution sequence selection mechanism. By converting the unmatched sub-sequences into one-hot encoding vectors, the 8-bit storage requirement for each base character in the original ASCII encoding is reduced to a feature space matching the number of base types, fundamentally eliminating character set redundancy. In particular, when dealing with metagenomic sequencing data and other sequencing data lacking reference genomes, by performing individual and population distribution selection on the encoding vectors of the unmatched sub-sequences, the base combination preferences implied in the population sequencing data can be mined, such as frequently occurring short sequence motifs or population-shared variation patterns, thereby breaking through the limitations of traditional methods that rely only on single sequence internal repeats. This double selection mechanism not only preserves the local features of individual sequences, but also captures long-range correlations across samples through population distribution modeling, effectively solving the problem that traditional dictionary compression algorithms cannot identify biological features. Then, the initialized entropy encoder can perform dynamic probability modeling on the selected encoding vector distribution, which is more suitable for the base composition characteristics of different sequencing projects than the fixed encoding table, so that the compression efficiency of the unmatched sequences is no longer limited by the universal compression framework of ASCII characters. The final compressed data not only avoids the inherent space waste of the original ASCII encoding, but also realizes deep mining of biological redundancy through feature extraction and distribution optimization, significantly improving storage efficiency while maintaining data integrity.

[0048] Figure 2 A flowchart of a process for compressing the unmatched sequences in the intelligent compression method for sequencing data according to an embodiment of the present application. As shown in Figure 2 the process of compressing the unmatched sequences includes: S210, performing sequence segmentation on the unmatched sequences to obtain a set of unmatched sub-sequences; S220, performing sequence selection based on population feature distribution guidance on the set of unmatched sub-sequences to obtain a selected subset of unmatched sub-sequences; and S230, performing compression encoding on the selected subset of unmatched sub-sequences.

[0049] Specifically, step S210, sequence segmentation is performed on the unmatched sequence to obtain a set of unmatched subsequences. Accordingly, considering that the unmatched sequence is usually composed of local variations, sequencing errors or unknown sequence fragments. These sequences, although cannot be completely matched with the assembled sequence, but their internal may still imply local duplication or population shared base combination pattern. However, when the traditional compression method directly compresses the whole unmatched sequence, due to the large length of the unsegmented sequence and the complex internal structure, it is difficult to effectively identify the fine-grained redundant features. For example, the unmatched sequence may contain multiple short motifs, but the global statistical characteristics of the long sequence will mask these local patterns, causing the general compression algorithm to only rely on limited dictionary matching or statistical models, and unable to fully exploit the potential relevance in biological sense. Based on this, the present application separates these short feature fragments of different microbial genomes into independent subsequences by performing sequence segmentation on the unmatched sequence to obtain a set of unmatched subsequences.

[0050] Specifically, step S220, sequence selection based on population feature distribution guidance is performed on the set of unmatched subsequences to obtain a selected subset of unmatched subsequences. Figure 3 For the sequence selection based on population feature distribution guidance on the set of unmatched subsequences to obtain a selected subset of unmatched subsequences in the intelligent compression method of sequencing data according to the embodiment of the present application. Specifically, in the embodiment of the present application, as shown in Figure 3 S220, sequence selection based on population feature distribution guidance is performed on the set of unmatched subsequences to obtain a selected subset of unmatched subsequences, including: S221, one-hot encoding is performed on each unmatched subsequence in the set of unmatched subsequences to obtain a set of unmatched subsequence encoding vectors; S222, individual and population distribution sequence selection is performed on the set of unmatched subsequence encoding vectors to obtain a set of encoding vectors of the unmatched subsequences as the selected subset of the unmatched subsequences.

[0051] More specifically, step S221, each of the set of unmatched sub-sequences is one-hot encoded to obtain a set of unmatched sub-sequence encoding vectors. It should be understood that considering that the Base data only contains A / T / C / G four types of base symbols, the traditional ASCII I encoding represents each base character with a single byte (8 bits), resulting in a 25% information density deficiency of the theoretical value (only 2 bits are required to encode 4 states). This encoding method not only directly wastes storage space, but more importantly, it forcibly maps the biological symbols of bases to the general character set, destroying the inherent structural features of the base sequence. For example, the ASCII encoding cannot distinguish the complementary pairing relationship of bases (such as the natural association of A-T, C-G), nor can it represent the possible biological rules between adjacent bases (such as the high-frequency methylation region of CpG island), making it difficult for subsequent compression algorithms to effectively identify and utilize these potential bioinformatics redundancies. Therefore, in the technical solution of the present application, each of the set of unmatched sub-sequences is one-hot encoded to obtain a set of unmatched sub-sequence encoding vectors. That is, this step converts each unmatched sub-sequence into a one-hot encoding vector matching the number of base types, for example, mapping a k-length unmatched sub-sequence to a 4xk-dimensional binary vector (each base position corresponds to four dimensions of unique bit activation of A / T / C / G). This encoding method reconstructs the data representation from two levels: first, it directly represents the discrete attributes of bases through binary vectors, eliminating the character set redundancy of ASCII encoding; second, it expands the vector dimension to convert the linear structure of the original sequence into a distributed representation in a high-dimensional feature space, laying the foundation for subsequent mining of statistical rules of base combinations. For example, in metagenomic data, a certain unmatched sub-sequence "ATCG" is one-hot encoded, and its vector representation can explicitly distinguish the feature activation state of each base position, allowing high-frequency k-mer patterns (such as "ATC" or "TCG") to form identifiable cluster distributions in the vector space.

[0052] Figure 4 A flowchart for individual and population distribution sequence selection of the set of unmatched sub-sequence encoding vectors in the intelligent compression method of sequencing data according to the embodiments of the present application to obtain a set of unmatched sub-sequence encoding vectors as the selected subset of the unmatched sub-sequences. More specifically, in the embodiments of the present application, as shown in FIG. 2B, step S222, the set of unmatched sub-sequence encoding vectors is subjected to individual and population distribution sequence selection to obtain a set of unmatched sub-sequence encoding vectors as the selected subset of the unmatched sub-sequences. Figure 4As shown, in step S222, individual and population distribution sequence selection is performed on the set of unmatched subsequence encoding vectors to obtain a set of unmatched subsequence encoding vectors as a selected subset of the unmatched subsequence. The step S222 includes: S222-1, extracting an ith unmatched subsequence encoding vector from the set of unmatched subsequence encoding vectors as an unmatched subsequence individual encoding vector; S222-2, calculating an unmatched subsequence individual semantic increment operator of the unmatched subsequence individual encoding vector based on the set of unmatched subsequence encoding vectors; and S222-3, determining whether to delete the ith unmatched subsequence encoding vector based on a comparison between the unmatched subsequence individual semantic increment operator and a preset threshold. Furthermore, although the one-hot encoding of the unmatched subsequence eliminates the character set redundancy of the ASCII encoding, the high-dimensional encoding vector thereof can still contain a large amount of noise or low-information-content features. For example, in metagenomic sequencing data, the unmatched subsequence can be derived from gene fragments of different species or random sequencing errors, and the distribution of these fragments at the population level is highly heterogeneous: some subsequences can represent conserved regions shared by the population (such as functional gene core motifs), while others can be individual sample-specific random variations or sequencing noise. If the encoding vectors of all unmatched subsequences are directly compressed, not only will the encoding complexity increase due to the retention of invalid features, but the probability model of the entropy encoder can also deviate from the true distribution due to noise interference, thereby reducing the compression efficiency. The traditional method only relies on the local statistical properties (such as k-mer frequency) of a single sequence, which is difficult to distinguish between population-shared patterns and individual noise, resulting in a compression algorithm that cannot accurately focus on truly compressible bioinformatics features. Therefore, the present application performs individual and population distribution sequence selection on the set of unmatched subsequence encoding vectors to obtain a set of unmatched subsequence encoding vectors as a selected subset of the unmatched subsequence.

[0053] That is, first, the vector requiring feature selection is extracted. Then, by constructing a population distribution semantic graph, the base combination preference across samples (such as high-frequency short sequence motifs or population-shared variation patterns) is captured from a global perspective, and the encoding vector of the unmatched subsequence is mapped into the implicit semantic space to reveal its functional relevance in the population distribution. For example, the encoding vector of a certain unmatched subsequence may exhibit a high-dimensional pattern shared by multiple samples in the mapping with the population, indicating that it belongs to a cross-sample conserved region rather than random noise. Subsequently, for the individual feature vector of each subsequence, the contribution to the overall semantic expression is quantified by modulating the mapping to the population semantic space (individual semantic increment operator). If the semantic increment of a certain subsequence is lower than the preset threshold, it means that its removal has little effect on the population semantic expression ability, and it can be determined as a redundant or noise feature to be removed; otherwise, it is retained as a key feature. For example, a random variation subsequence caused by a sequencing error is isolated in the population distribution, and its individual semantic increment is significantly lower than the functional motif shared by the population, so it is automatically filtered by the screening mechanism.

[0054] In particular, step S222-1, the ithunmatched subsequence encoding vector is extracted from the set of unmatched subsequence encoding vectors as an individual encoding vector of the unmatched subsequence.

[0055] It can be understood that the unique value of each subsequence in the population context needs to be finely evaluated. For example, in metagenomic sequencing, a certain unmatched subsequence may have cross-sample conservation because it carries functional gene fragments of a certain species, but its expression frequency or variation pattern in a single sample may be significantly different from other subsequences. If only relying on global feature screening at the population level, the specific contribution of individual subsequence in the local sample may be ignored, for example, a low-frequency but functionally critical variation fragment may be misjudged as redundant noise due to low group co-occurrence frequency, resulting in loss of key biological information or suboptimal selection of compression efficiency. Therefore, the present application extracts the ithunmatched subsequence encoding vector from the set of unmatched subsequence encoding vectors as an individual encoding vector of the unmatched subsequence, thereby focusing on the feature expression of a single subsequence, thereby providing the accuracy of feature selection.

[0056] More specifically, in the embodiments of the present application, step S222-2, based on the set of unmatched subsequence encoding vectors, calculating an unmatched subsequence individual semantic increment operator of the unmatched subsequence individual encoding vector based on the set of unmatched subsequence encoding vectors, comprises: S222-21, calculating an unmatched group distribution semantic graph of the set of unmatched subsequence encoding vectors; S222-22, mapping the unmatched subsequence individual encoding vector to a feature space of the unmatched group distribution semantic graph to obtain an unmatched subsequence individual modulation encoding vector; S222-23, calculating an unmatched subsequence individual semantic increment operator of the unmatched subsequence individual modulation encoding vector relative to the set of unmatched subsequence encoding vectors.

[0057] In particular, step S222-21, calculating an unmatched group distribution semantic graph of the set of unmatched subsequence encoding vectors, the process can be represented by the formula as follows:

[0058] X = {x1, x2,..., xi,..., xj,..., xn} i ,...,x n}

[0059]

[0060] wherein X is the set of unmatched subsequence encoding vectors, x1, x2, x i , x j and x n are the 1st, 2nd, i-th, j-th and n-th unmatched subsequence encoding vectors in the set of unmatched subsequence encoding vectors, W r is a learnable weight matrix, b is a bias vector, [·; ·] is vector splicing, r i,j is an unmatched subsequence linear aggregation vector between x i and x j , is a feature value at the h-th position in r i,j , T is the number of feature values in r i,j , sigmoid is a sigmoid activation function, R(x i , x j ) is an unmatched subsequence semantic correlation between x i and x j , i.e. the value at the (i, j) position in the unmatched group distribution semantic graph, R(x1, x1), R(x1, x n ), R(x n , x1)…R(x n , x n are feature values at various positions in the unmatched group distribution semantic graph, and M is the unmatched group distribution semantic graph.

[0061] Accordingly, the set of unmapped subsequence encoding vectors implies complex cross-sample correlations. For example, in metagenomic sequencing data, unmapped subsequences can originate from conserved gene fragments of different microorganisms or environmental-specific variations, which can exhibit high-frequency co-occurrence or functionally complementary distribution patterns at the population level. However, relying solely on single sequences cannot capture long-range correlations across samples, resulting in the inability to identify redundant patterns common to the population. For example, a certain unmapped subsequence can appear as a low-frequency random fragment in a single sample, but repeatedly appears in multiple samples of the population, forming an implicit functional motif or structural feature. If this cross-sample redundancy is not effectively mined, it will directly limit the improvement of compression efficiency. Based on this, the present application dynamically captures global correlations between features by calculating the unmapped population distribution semantic graph of the set of unmapped subsequence encoding vectors.

[0062] In particular, in step S222-22, the unmapped subsequence individual encoding vector is mapped to the feature space of the unmapped population distribution semantic graph to obtain an unmapped subsequence individual modulation encoding vector. This process can be represented by the formula:

[0063]

[0064] where x i is the i-th unmapped subsequence encoding vector in the set of unmapped subsequence encoding vectors, M is the unmapped population distribution semantic graph, is matrix multiplication, v i is the x i corresponding unmapped subsequence individual modulation encoding vector.

[0065] It should be understood that although the individual encoding vector of the unmapped subsequence has characterized its local base features through one-hot encoding, its independent analysis can ignore its potential semantic association in population distribution. For example, a certain individual subsequence can have a base mutation due to sequencing errors, and its independent encoding vector only reflects the local variation characteristics of the current sample, but cannot reflect whether the variation has cross-sample co-occurrence or functional conservation in the population. If feature selection is directly based on the individual encoding vector of the unmapped subsequence, it can lead to misjudgment due to the lack of population context: a functional variation that appears frequently across samples is mistakenly deleted as noise, or a random noise that only exists in a single sample is retained, thereby affecting the recognition accuracy of the compression algorithm for true redundant patterns. Therefore, the unmapped subsequence individual encoding vector is mapped to the feature space of the unmapped population distribution semantic graph to obtain an unmapped subsequence individual modulation encoding vector, so that not only the biological characteristics of the individual subsequence are retained, but also its potential redundancy across samples is revealed through population semantic information.

[0066] In particular, in the embodiments of the present application, the step S222-23 of calculating the unmatched subsequence individual semantic increment operator of the unmatched subsequence individual modulation coding vector with respect to the set of unmatched subsequence coding vectors comprises:

[0067] The scale-invariant correlation and semantic conjugate mapping optimization is performed on the unmatched subsequence individual modulation coding vector to obtain an unmatched subsequence individual optimized modulation coding vector, and the process can be represented by a formula as follows:

[0068]

[0069] wherein x i is the i-th unmatched subsequence coding vector in the set of unmatched subsequence coding vectors, v i is the corresponding unmatched subsequence individual modulation coding vector of x i , T is a transposition operation, (·) ⊙-1 is the reciprocal of the eigenvalue at each position in the calculation vector, M i is the corresponding unmatched subsequence scale-invariant correlation matrix of x i , ln is the logarithmic function value with the natural constant e as the base, x' i is the corresponding unmatched subsequence global correlation abnormal dimension vector of x i , and is the point multiplication according to the position, v' i is the corresponding unmatched subsequence individual optimized modulation coding vector of x i .

[0070] Based on the set of unmatched subsequence coding vectors, the individual semantic energy ratio calculation of the unmatched subsequence individual optimized modulation coding vector is performed with reference to the group to obtain the unmatched subsequence individual semantic increment operator, and the process can be represented by a formula as follows:

[0071]

[0072] wherein x i is the i-th unmatched subsequence coding vector in the set of unmatched subsequence coding vectors, v' i is the corresponding unmatched subsequence individual optimized modulation coding vector of x i , v' ik is the eigenvalue at the k-th position in v' i , x i,k is the eigenvalue at the k-th position in x i , L is the number of eigenvalues in v' i and x i , and v' i and x i have the same length, and n is the number of vectors in the set of unmatched subsequence coding vectors, and λi is x i The corresponding unmatched sub-sequence individual semantic energy ratio, arctan is the inverse tangent function, π is the ratio of a circle, is x i The corresponding unmatched sub-sequence individual semantic increment operator.

[0073] In particular, since the unmatched group distribution semantic graph M captures the feature value distribution based on local correlation of feature vectors, feature joint occurrence frequency and feature space configuration features through the pairwise calculation of unmatched sub-sequence encoding vectors x i ,x j When, there is inevitably a global universal heterogeneity phenomenon. Specifically, the semantic neighborhood energy configuration generated by the local correlation defect produces a phase deviation in the energy transfer process of the local semantic background field, resulting in a significant system error in the overall-local difference significance measure of independent individuals x i .

[0074] Therefore, the unmatched sub-sequence scale invariant correlation matrix By taking the individual feature x i as the topological invariant of the unmatched group distribution semantic graph M, the β function analytical expansion based on the global semantic conjugate mapping is implemented. This process derives x' i = ln(v i )⊙[ln(x i ) ⊙-1 ] as a universal critical dimension representation, establishing an abnormal dimension correction mechanism for global correlation. Finally, with the help of the energy scale dimension transmission path M'-M, the local disturbance of neighborhood energy is constrained in the invariant scale dimension space, significantly improving the local-global migration universalization efficiency of the unmatched sub-sequence individual modulation encoding vector v i based on the unmatched group distribution semantic graph M.

[0075] ​Accordingly, although the modulation and coding vector of the unmatched subsequence individual has fused the group semantic information, its contribution to the overall data redundancy still lacks quantitative evaluation standards. For example, in metagenomic sequencing, a certain individual subsequence may carry a conserved metabolic pathway gene fragment across samples, and its modulation and coding vector is highly associated with multiple functional motifs in the group semantic graph. If only relying on group co-occurrence frequency for screening, low-frequency but key functional fragments (such as rare variants of antibiotic resistance genes) may be ignored, leading to the deletion of high semantic value features or the retention of a large number of non-contributing random fragments in the compression process, thereby weakening the biological perception ability of the compression algorithm. Therefore, the individual semantic energy ratio calculation based on the group reference of the unmatched subsequence individual optimization modulation and coding vector to the set of the unmatched subsequence coding vectors is used to achieve accurate identification and priority division of biological redundancy in the compression process, and the individual semantic increment operator of the unmatched subsequence is obtained.

[0076] In particular, in step S222-3, whether to delete the i-th unmatched subsequence coding vector is determined based on the comparison between the unmatched subsequence individual semantic increment operator and the preset threshold, which can be expressed by the formula as follows:

[0077]

[0078] wherein x i is the i-th unmatched subsequence coding vector in the set of unmatched subsequence coding vectors, is the corresponding unmatched subsequence individual semantic increment operator of x i , mask(x i ) is the feature selection of x i , and θ is the preset threshold.

[0079] Finally, whether to delete the i-th unmatched subsequence coding vector is determined based on the comparison between the unmatched subsequence individual semantic increment operator and the preset threshold. By setting an adjustable threshold as the boundary standard of semantic contribution, the increment operator value of each unmatched subsequence is compared with the threshold: if the operator value is higher than the threshold, it indicates that the contribution of the subsequence to the group semantic expression is significant, and it needs to be retained to maintain data integrity; otherwise, it is determined as a redundant or noise feature to be removed. For example, in environmental metagenomic analysis, a certain unmatched subsequence may carry specific marker genes of a new virus and have a high semantic increment value, so even if it appears at a low frequency in the current sample, it will still be retained due to exceeding the threshold; while a high-frequency but unknown functional repeat fragment may be considered as redundant and efficiently removed due to the increment value being lower than the threshold.

[0080] Specifically, step S230, the refined subset of the unmatched sub-sequences is compressed and encoded. Specifically, in the embodiments of the present application, the refined subset of the unmatched sub-sequences is compressed and encoded, comprising: using an initialized entropy encoder to compress and encode the refined subset of the unmatched sub-sequences. It should be understood that after the previous sequence segmentation, one-hot encoding and sequence selection steps, the refined subset of the unmatched sub-sequences has specific distribution characteristics. The entropy encoder can dynamically model the probability according to the probability distribution of the data, which can adapt to the base composition characteristics of different sequencing projects, unlike the traditional fixed coding table which has limitations.

[0081] Specifically, using an initialized entropy encoder to compress and encode the refined subset of the unmatched sub-sequences can be achieved by the following way: in the technical solutions of the present application, the entropy encoder plays a key role in the entire compression process, which can dynamically model the probability according to the probability distribution of the data. This feature makes it show strong adaptability when processing different sequencing projects.

[0082] In particular, the initialized entropy encoder is obtained through the preliminary analysis of the sequencing data and the setting of the model parameters. When facing a specific sequencing project, a large number of sequencing data samples similar to the project are first collected. The unmatched sub-sequences in these samples are sorted and counted, and the distribution rules of the bases and base combinations are analyzed. Based on these statistical information, the initial parameters of the entropy encoder are determined. For example, according to the frequency of different base combinations in the samples, the initial encoding length and encoding rules are set. Then, the entropy encoder is initialized using these initial parameters. In the initialization process, the characteristics of different sequencing projects, such as the preference of base composition and the degree of data redundancy, are also considered to optimize and adjust the encoder. The initialized entropy encoder obtained in this way can better adapt to the characteristics of the refined subset of the unmatched sub-sequences in a specific sequencing project, providing strong support for subsequent compression and encoding work. In practical applications, the initialized entropy encoder can quickly and accurately compress and encode the refined subset of the unmatched sub-sequences, improve the efficiency and quality of the entire sequencing data compression processing, effectively reduce the space required for data storage, while ensuring the integrity of the data, facilitating subsequent bioinformatics research and analysis.

[0083] In the implementation of compression encoding, the refined subset of the unmatched subsequence has specific distribution characteristics due to the previous sequence segmentation, one-hot encoding, and sequence selection steps. The initialized entropy encoder analyzes the frequency information of the base or base combination in the subset based on these characteristics. For example, in actual metagenomic sequencing data, certain specific base combinations may appear frequently in the refined subset of the unmatched subsequence, while others may appear infrequently. The entropy encoder uses this frequency difference to adopt shorter encoding for the frequently appearing part, because these parts appear frequently in the data, and using short encoding can significantly reduce the required storage space; for the infrequently appearing part, longer encoding is used, which ensures the integrity of the data and achieves efficient compression as a whole.

[0084] For a more intuitive understanding, assume that the base combination "ATC" appears frequently in the refined subset of the unmatched subsequence, and the entropy encoder may assign it a shorter code such as "00"; while for the low-frequency "GCTA" combination, a longer code such as "1010" may be assigned. In this way, the entropy encoder can flexibly adjust the encoding strategy according to the specific situation of the refined subset, achieving efficient compression of data.

[0085] In addition, the entropy encoder has a significant advantage over traditional fixed encoding tables. Traditional fixed encoding tables cannot be adjusted according to the base composition characteristics of different sequencing projects, and when faced with complex and diverse sequencing data, they often cannot fully utilize the redundancy of the data, resulting in low compression efficiency. The entropy encoder can adapt to the base composition characteristics of different sequencing projects, dynamically model the probability for each sequencing project, and thus more accurately capture the redundant information in the data, achieving better compression effect.

[0086] In summary, the compression of the unmatched sequences is illustrated by introducing a one-hot encoding and population distribution sequence selection mechanism. By converting the unmatched sub-sequences into one-hot encoding vectors, the 8-bit storage requirement for each base character in the original ASCII encoding is reduced to a feature space matching the number of base types, fundamentally eliminating character set redundancy. In particular, when dealing with sequencing data such as metagenomics that lack reference genomes, by individually and population distribution selecting the encoding vectors of the unmatched sub-sequences, the base combination preferences implied in the population sequencing data, such as frequently occurring short sequence motifs or population-shared variation patterns, can be mined, thereby breaking through the limitations of traditional methods that rely solely on single sequence internal repeats. This double selection mechanism not only preserves the local features of individual sequences, but also captures long-range correlations across samples through population distribution modeling, effectively solving the problem that traditional dictionary compression algorithms cannot identify biological features. Then, the initialized entropy encoder can dynamically model the probability of the selected encoding vector distribution, which is more suitable for the base composition characteristics of different sequencing projects than the fixed encoding table, so that the compression efficiency of the unmatched sequences is no longer limited by the universal compression framework of ASCII characters. The final compressed data not only avoids the inherent space waste of the original ASCII encoding, but also realizes the deep mining of biological redundancy through feature extraction and distribution optimization, significantly improving the storage efficiency while maintaining data integrity.

[0087] In step S130, the compressed DNA sequencing data is stored. It should be understood that after completing the compression processing of the DNA sequencing data, the key link of storing the compressed data is entered. DNA sequencing data contains rich biological genetic information and has irreplaceable important value in many fields such as bioinformatics research, medical diagnosis, and species evolution research. For example, in the medical field, by analyzing these data, disease-related gene mutations can be found, providing a strong basis for early diagnosis and personalized treatment of diseases; in the study of species evolution, it is helpful to understand the evolutionary history and relationship of species. Therefore, proper storage of the compressed DNA sequencing data is a necessary measure to preserve its data value and provide solid support for subsequent research and application.

[0088] From the actual operation point of view of storage, the data processed by compression will be saved according to specific storage formats and rules. These formats and rules are carefully designed according to the characteristics of the data and the needs of subsequent use. For example, when storing, the classification and indexing of data will be fully considered to facilitate subsequent fast and accurate data retrieval. In a large-scale data storage environment, the storage space will be reasonably planned to ensure that the small volume of compressed data can more efficiently utilize the space of the storage device, thereby reducing storage costs.

[0089] In this application, there is a special data storage module to perform the storage task. These storage modules have powerful functions, can orderly store the compressed DNA sequencing data on various storage media such as hard disk, cloud storage, etc. And the storage module will also manage and maintain the data to ensure the security and integrity of the data. This means that in subsequent research, researchers can use the corresponding retrieval tools and interfaces to easily obtain the original data for reanalysis according to the needs. With the continuous progress of research technology and methods, it is extremely likely to find new biological characteristics and rules by reanalyzing the stored data, which further highlights the importance of properly storing the compressed DNA sequencing data.

[0090] In summary, the intelligent compression method of sequencing data based on the embodiments of the present application is illustrated, which first acquires a plurality of short gene sequences containing ID, Base and Quality data, and then compresses the DNA sequencing data to be stored. By matching the short gene sequence to be compressed with the already spliced sequence, if the Base data is all matched successfully, it is directly compressed; if it is partially matched, it is split into matched and unmatched sequences for compression respectively. For the unmatched sequence, it is further divided into a subsequence set, and a refined subset is obtained based on the population characteristic distribution guided sequence selection, and finally the refined subset is compressed and encoded. This method effectively improves the compression ratio, especially when dealing with unmatched sequences, by fine screening to reduce redundant information, improve overall storage efficiency and data retrieval speed, and also enhance the effective mining ability of long-range redundancy.

[0091] Figure 5 The block diagram of the intelligent compression system of sequencing data according to the embodiments of the present application. As shown in Figure 5 The intelligent compression system of sequencing data 100 according to the embodiments of the present application, comprising: a sequencing data acquisition module 110, used for acquiring DNA sequencing data to be stored containing a plurality of short gene sequences, the short gene sequence including ID data, Base data and Quality data; a sequencing data processing module 120, used for compressing the DNA sequencing data to be stored, wherein the sequencing data processing module is used for: extracting short gene sequences to be compressed from the DNA sequencing data to be stored, and acquiring already spliced sequences; matching the short gene sequences to be compressed with the already spliced sequences; in response to the Base data of the short gene sequences to be compressed being all matched successfully, compressing the Base data of the short gene sequences to be compressed; in response to the Base data of the short gene sequences to be compressed being partially matched successfully, splitting the Base data of the short gene sequences to be compressed into matched sequences and unmatched sequences for compression respectively; a compression module 130, used for storing the compressed DNA sequencing data.

[0092] Specifically, in the embodiments of the present application, the sequencing data processing module is further configured to: perform sequence segmentation on the unmatched sequence to obtain a set of unmatched subsequences; perform population feature distribution guided sequence selection on the set of unmatched subsequences to obtain a selected subset of unmatched subsequences; and perform compression encoding on the selected subset of unmatched subsequences.

[0093] Here, those skilled in the art can understand that the specific operations of each step in the above-described intelligent compression system for sequencing data have been described in detail above with reference to the description of the intelligent compression method for sequencing data, and therefore repeated descriptions thereof will be omitted. Figures 1 to 4

[0094] As described above, the intelligent compression system for sequencing data 100 according to the embodiments of the present disclosure can be implemented in various wireless terminals, such as a server with intelligent compression algorithm for sequencing data, etc. In one possible implementation, the intelligent compression system for sequencing data 100 according to the embodiments of the present disclosure can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the intelligent compression system for sequencing data 100 can be a software module in the operating system of the wireless terminal, or can be an application program developed for the wireless terminal; of course, the intelligent compression system for sequencing data 100 can also be one of the many hardware modules of the wireless terminal.

[0095] Alternatively, in another example, the intelligent compression system for sequencing data 100 and the wireless terminal can also be separate devices, and the intelligent compression system for sequencing data 100 can be connected to the wireless terminal through a wired and / or wireless network, and transmit interactive information in an agreed data format.

[0096] In summary, it is intended that the above detailed description be considered exemplary only and not limiting, and that the above-described embodiments are to be understood to have been presented for purposes of illustration and not limitation, and that the scope of the present application is to be limited only by the appended claims. Having now described the present application in accordance with the embodiments described above, it is to be understood that the application is not to be limited to what has been presented, and that changes and modifications can be made by those skilled in the art without departure from the scope of the present application.​

Claims

1. A method of intelligent compression of sequencing data, characterized in that, The method comprises: acquiring to-be-stored DNA sequencing data containing a plurality of short gene sequences, the short gene sequences including ID data, Base data and Quality data; performing compression processing on the to-be-stored DNA sequencing data, comprising: extracting to-be-compressed short gene sequences from the to-be-stored DNA sequencing data and acquiring spliced sequences; matching the to-be-compressed short gene sequences with the spliced sequences; in response to the Base data of the to-be-compressed short gene sequences being all matched successfully, compressing the Base data of the to-be-compressed short gene sequences; in response to the Base data of the to-be-compressed short gene sequences being partially matched successfully, splitting the Base data of the to-be-compressed short gene sequences into matched sequences and unmatched sequences and then respectively compressing the matched sequences and the unmatched sequences; storing the compressed DNA sequencing data; wherein the compression on the unmatched sequences comprises: performing sequence segmentation on the unmatched sequences to obtain a set of unmatched subsequences; performing population feature distribution guided sequence selection on the set of unmatched subsequences to obtain a selected subset of the unmatched subsequences; and performing compression encoding on the selected subset of the unmatched subsequences; wherein the population feature distribution guided sequence selection on the set of unmatched subsequences to obtain a selected subset of the unmatched subsequences comprises: one-hot encoding each of the unmatched subsequences in the set of unmatched subsequences to obtain a set of unmatched subsequence encoding vectors; performing individual and population distribution sequence selection on the set of unmatched subsequence encoding vectors to obtain a set of encoding vectors of the unmatched subsequences as the selected subset of the unmatched subsequences; wherein the individual and population distribution sequence selection on the set of unmatched subsequence encoding vectors to obtain a set of encoding vectors of the unmatched subsequences as the selected subset of the unmatched subsequences comprises: extracting an i-th unmatched subsequence encoding vector from the set of unmatched subsequence encoding vectors as an unmatched subsequence individual encoding vector; based on the set of unmatched subsequence encoding vectors, calculating an unmatched subsequence individual semantic increment operator of the unmatched subsequence individual encoding vector, wherein the individual semantic increment operator represents the contribution degree of the individual to the overall semantic expression by modulation mapping to a population semantic space; based on a comparison between the unmatched subsequence individual semantic increment operator and a preset threshold, determining whether to delete the i-th unmatched subsequence encoding vector.

2. The method of intelligent compression of sequencing data according to claim 1, wherein, The short gene sequence is a sequencing read result of a fragment of an original DNA strand corresponding to the to-be-stored DNA sequencing data.

3. The method of intelligent compression of sequencing data as claimed in claim 1, wherein, based on the set of unmatched subsequence encoding vectors, calculating an unmatched subsequence individual semantic increment operator of the unmatched subsequence individual encoding vector comprises: calculating an unmatched population distribution semantic graph of the set of unmatched subsequence encoding vectors; mapping the unmatched subsequence individual encoding vector to a feature space of the unmatched population distribution semantic graph to obtain an unmatched subsequence individual modulation encoding vector; and performing individual and population distribution sequence selection on the set of unmatched subsequence encoding vectors to obtain a set of encoding vectors of the unmatched subsequences as the selected subset of the unmatched subsequences. calculating an unmatched sub-sequence individual semantic increment operator of the unmatched sub-sequence individual modulation coding vector relative to the set of unmatched sub-sequence coding vectors.

4. The method of intelligent compression of sequencing data according to claim 3, wherein, calculating an unmatched sub-sequence individual semantic increment operator of the unmatched sub-sequence individual modulation coding vector relative to the set of unmatched sub-sequence coding vectors, comprising: scaling invariant correlation and semantic conjugate mapping optimization on the unmatched sub-sequence individual modulation coding vector to obtain an unmatched sub-sequence individual optimized modulation coding vector; group-referenced individual semantic energy ratio calculation on the unmatched sub-sequence individual optimized modulation coding vector based on the set of unmatched sub-sequence coding vectors to obtain the unmatched sub-sequence individual semantic increment operator.

5. The method of intelligent compression of sequencing data as claimed in claim 1, wherein, compressively encoding the selected subset of unmatched sub-sequences, comprising: using an initialized entropy encoder to compressively encode the selected subset of unmatched sub-sequences.

6. An intelligent compression system of sequencing data, characterized in that, comprising: a sequencing data acquisition module, configured to acquire to-be-stored DNA sequencing data containing a plurality of short gene sequences, the short gene sequences including ID data, Base data, and Quality data; a sequencing data processing module, configured to perform compression processing on the to-be-stored DNA sequencing data, wherein the sequencing data processing module is configured to: extract to-be-compressed short gene sequences from the to-be-stored DNA sequencing data and acquire spliced sequences; match the to-be-compressed short gene sequences with the spliced sequences; in response to successful full matching of Base data of the to-be-compressed short gene sequences, compress the Base data of the to-be-compressed short gene sequences; and in response to successful partial matching of the Base data of the to-be-compressed short gene sequences, split the Base data of the to-be-compressed short gene sequences into matched sequences and unmatched sequences and compress the matched sequences and the unmatched sequences respectively; a compression module, configured to store the compressed DNA sequencing data; wherein the sequencing data processing module is further configured to: perform sequence segmentation on the unmatched sequences to obtain a set of unmatched sub-sequences; perform group feature distribution guided sequence selection on the set of unmatched sub-sequences to obtain a selected subset of unmatched sub-sequences; and compressively encode the selected subset of unmatched sub-sequences; wherein the group feature distribution guided sequence selection on the set of unmatched sub-sequences to obtain the selected subset of unmatched sub-sequences comprises: one-hot encoding each unmatched sub-sequence in the set of unmatched sub-sequences to obtain a set of unmatched sub-sequence coding vectors; individual and group distribution sequence selection on the set of unmatched sub-sequence coding vectors to obtain the set of unmatched sub-sequence coding vectors as the selected subset of unmatched sub-sequences; wherein the individual and group distribution sequence selection on the set of unmatched sub-sequence coding vectors to obtain the set of unmatched sub-sequence coding vectors as the selected subset of unmatched sub-sequences comprises: extracting an i-th unmatched sub-sequence coding vector from the set of unmatched sub-sequence coding vectors as an unmatched sub-sequence individual coding vector; Based on the set of the unmatched subsequence encoding vectors, an unmatched subsequence individual semantic increment operator of the unmatched subsequence individual encoding vector is calculated, wherein the individual semantic increment operator represents a degree of contribution to the overall semantic expression by modulating the mapping to the group semantic space; Based on the comparison between the unmatched subsequence individual semantic increment operator and a preset threshold, it is determined whether to delete the i-th unmatched subsequence encoding vector.

Images