Method and system for marking redundant gene data and computer readable storage medium
By processing gene data files in blocks in parallel and using a multi-stage pipeline, the inefficiency of existing gene redundancy labeling software on large-scale datasets is solved, achieving efficient gene redundancy labeling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INST OF COMPUTING TECH CHINESE ACAD OF SCI
- Filing Date
- 2025-12-29
- Publication Date
- 2026-05-01
AI Technical Summary
Existing gene redundancy marker software is inefficient in processing large-scale datasets, mainly due to its single-threaded implementation and reliance on global sorting, which leads to excessive computational and resource overhead.
The gene data file is divided into multiple data blocks, and the redundant gene marker process is set as a multi-stage parallel pipeline. Each pipeline stage is allocated multiple processing threads. The results are corrected by dividing the data blocks and processing them in a multi-threaded parallel pipeline, combined with the overlapping areas of adjacent data blocks.
It greatly improves the efficiency of gene redundancy labeling on large-scale datasets, makes full use of computing resources, avoids the performance bottleneck caused by global sorting, and achieves fast and efficient gene redundancy labeling.
Smart Images
Figure CN121963869A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of gene data analysis technology, specifically to methods and systems for identifying gene redundancy information. Background Technology
[0002] The statements in this section are merely to provide background information in relation to this application to aid in understanding it, and such background information does not necessarily constitute prior art.
[0003] In the workflow of gene sequence analysis, gene redundancy labeling is an indispensable and crucial preprocessing step. Its function is to identify redundant gene sequence data generated by PCR (Polymerase Chain Reaction) amplification bias and human factors in sequencing technology. These redundant sequences originate from amplification products of the same template DNA, are not only completely identical in sequence but also completely overlap in alignment coordinates on the reference genome. They are not representations of the genetic characteristics of the biological sample itself but rather technical noise introduced during experimental procedures. In quantitative gene expression analysis, these redundant sequences can artificially inflate the reading count of specific genes, leading to an overestimation of expression levels. In scenarios such as mutation detection, redundant sequences can distort the estimated values of allele frequencies and lead to false positive mutation results, masking or amplifying the true proportion of low-frequency mutations. Therefore, efficiently and accurately identifying and labeling gene redundancy information has become a core prerequisite for ensuring the reliability of gene sequence analysis results.
[0004] To address the problem of gene redundancy labeling, the dedicated software Picard MarkDuplicates, with its mature algorithmic logic and stable labeling performance, is widely recognized as the gold standard tool in this field. This software determines redundancy relationships by tracking features such as the start position and end-splitting information of sequences. However, its processing efficiency drops drastically when handling large-scale sequencing datasets of tens of gigabytes or even terabytes, severely hindering the progress and efficiency of gene sequence analysis. Summary of the Invention
[0005] The inventors discovered through research that the limitations of existing gene redundancy marker software tools in processing large-scale datasets are mainly due to their single-threaded implementation and the significant computational and resource overhead resulting from their reliance on global sorting.
[0006] In this regard, this application aims to provide a new scheme for labeling redundant gene data to improve the speed of gene redundancy labeling, thereby improving processing efficiency on large-scale datasets.
[0007] The objective of this application is achieved through the following technical solution: According to a first aspect of this application, a method for labeling redundant gene data is provided, comprising: dividing a gene data file to be labeled into multiple data blocks, wherein the gene data file to be labeled is sorted based on the starting coordinates of sequencing reads; setting the redundant gene labeling process as a multi-stage parallel pipeline, each execution step of the redundant gene labeling process corresponding to a pipeline stage in the multi-stage pipeline, and allocating multiple processing threads to each pipeline stage; providing each data block to the multi-stage parallel pipeline for redundant gene labeling processing; and correcting the redundant gene labeling processing results from the multi-stage parallel pipeline based on the overlapping area between adjacent data blocks.
[0008] Preferably, the redundant gene marker process includes at least the following execution steps: Step 1: Reading and parsing the data block; Step 2: Finding sequence pairing information; Step 3: Sorting the data block; Step 4: Redundancy information detection and outputting the marker result.
[0009] Preferably, the step of providing each data block to the multi-stage parallel pipeline for redundant gene marker processing includes: dividing the data block into multiple sub-data blocks; sequentially supplying each sub-data block to the first to third pipeline stages of the multi-stage parallel pipeline for processing; merging all sub-data blocks after processing by the third pipeline stage, and providing the merged data block to the fourth pipeline stage for processing.
[0010] Preferably, it further includes: allocating an independent processing thread for each sub-data block in the first to third pipeline stages.
[0011] Preferably, the third flow stage corresponding to step 3 is set to: sorting the sequencing reads in the data block according to the sorting rules; wherein the sorting rules are arranged in order of priority from high to low, including: by read direction mark, by reference sequence identifier of paired read, by start coordinate of paired read, by sequencing read insertion length, and by sequencing read name.
[0012] Preferably, the correction of the redundant gene labeling processing results from the multi-stage parallel pipeline based on the overlapping area between adjacent data blocks includes: setting an overlapping area based on the boundary of two adjacent data blocks; then traversing the overlapping area and checking whether there is a redundant read group spanning two data blocks based on the start coordinates and sequence of the sequencing read; if there is a redundant read group spanning two blocks, then merging the redundant read groups and re-labeling the redundancy attributes of the redundant read groups according to a preset redundancy determination rule.
[0013] According to a second aspect of this application, a method for labeling redundant gene data is provided, comprising: dividing a gene data file to be labeled into multiple data blocks, wherein the gene data file to be labeled is sorted based on the starting coordinates of sequencing reads; setting the redundant gene labeling process as a multi-stage parallel pipeline, each execution step of the redundant gene labeling process corresponding to a pipeline stage in the multi-stage pipeline, and allocating multiple processing threads to each pipeline stage; sequentially providing two data blocks from the multiple data blocks to the multi-stage parallel pipeline for redundant gene labeling processing; wherein the redundant gene labeling process includes at least the following execution steps: step 1: reading and parsing data blocks; step 2: finding sequence pairing information; step 3: sorting data blocks; step 4: redundancy information detection and outputting labeling results; step 5: overlapping region detection and correcting labeling results; and wherein the pipeline stage corresponding to step 5 only starts after receiving the labeling results for the two data blocks from the previous pipeline stage.
[0014] Preferably, the fifth pipeline stage corresponding to step 5 is set to detect the overlapping area between two data blocks and thereby correct the redundant gene marker processing results of the two data blocks from the previous pipeline stage.
[0015] According to a third aspect of this application, a system for labeling redundant gene data is provided, comprising a block processing module, a pipeline setting module, a redundancy labeling module, and a detection and correction module; wherein the block processing module is configured to divide the gene data file to be labeled into multiple data blocks, wherein the gene data file to be labeled is sorted based on the starting coordinates of sequencing reads; the pipeline setting module is configured to set the redundant gene labeling process as a multi-stage parallel pipeline, wherein each execution step of the redundant gene labeling process corresponds to a pipeline stage in the multi-stage pipeline, and multiple processing threads are allocated to each pipeline stage; the redundancy labeling module is configured to provide each data block to the multi-stage parallel pipeline for redundant gene labeling processing; and the detection and correction module is configured to correct the redundant gene labeling processing results from the multi-stage parallel pipeline based on the overlapping area between adjacent data blocks.
[0016] According to a fourth aspect of this application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, implements the methods as described in the first and second aspects.
[0017] Compared with existing technologies, the advantages of this application are mainly as follows: it makes full use of the data features of SAM / BAM files based on coordinate sorting, avoiding the performance bottleneck caused by global sorting; through data block partitioning, process execution step partitioning and multi-threaded parallel pipeline execution scheme, it makes full use of computing resources, greatly improves the running efficiency of the overall labeling process, and realizes rapid gene redundancy labeling on large-scale datasets. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart illustrating a method for labeling redundant gene data according to an embodiment of this application; Figure 2 This is a schematic diagram of the execution flow for performing redundant gene marker processing on a data block according to an embodiment of this application; Figure 3 This is a schematic diagram of the execution flow of marker redundant gene data according to an embodiment of this application; Figure 4 This is a schematic diagram of a multi-stage parallel pipeline for labeling redundant gene data according to an embodiment of this application. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided through specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.
[0020] As mentioned in the background section, Picard MarkDuplicates is a mainstream tool for identifying redundant sequences. It distinguishes between biological duplicates and technical duplicates based on the alignment information of sequencing reads. The inventors discovered in their research that this tool relies on global sorting based on BAM / SAM files. That is, after inputting the BAM / SAM file, a global sort must first be performed. The sorting rules are fixed and cannot be adjusted. The specific order is as follows (in descending order of priority): by chromosome number, by the start coordinates of the sequencing read, by read direction marker, and by the reference sequence identifier of the paired read (unpaired reads are considered as...). The sorting process is performed by starting coordinates of paired reads (unpaired reads are considered 0), by read insertion length (signed), and by read name (lexicographically). Global sorting requires significant memory and disk space; when processing terabytes of sequencing data, the sorting time is often several times that of the redundancy markers themselves. Furthermore, the tool operates entirely in a single thread; redundancy detection can only begin after the full sorting is complete, preventing full utilization of modern computers' multi-core computing resources for parallel processing.
[0021] Figure 1 This is a flowchart illustrating a method for labeling redundant gene data according to an embodiment of this application. Figure 1 As shown, the method includes: Step S101, dividing the gene data file to be labeled into multiple data blocks, wherein the gene data file to be labeled is sorted based on the starting coordinates of sequencing reads. In some embodiments, the gene data file is a BAM (Binary Alignment Map) / SAM (Sequence Alignment / Map) file. Sorting based on the starting coordinates of sequencing reads includes sorting the sequencing reads in the BAM / SAM file first by chromosome number and then by the starting coordinates of the sequencing reads. In some embodiments, the number of data blocks is determined by preset parameters. Step S102, setting the redundant gene labeling process as a multi-stage parallel pipeline, each execution step of the redundant gene labeling process corresponds to a pipeline stage in the multi-stage pipeline, and multiple processing threads are allocated to each pipeline stage. Step S103, providing each data block to the multi-stage parallel pipeline for redundant gene labeling processing, that is, starting multiple threads to process different data blocks respectively, and the multiple threads in the same pipeline stage are independent of each other and can be executed independently. Step S104: Correct the redundant gene marker processing results from the multi-stage parallel pipeline based on the overlapping area between adjacent data blocks.
[0022] This application fully utilizes the data characteristics of SAM / BAM files based on coordinate sorting, avoiding the performance bottleneck caused by global sorting. Through data block partitioning, process execution step partitioning, and multi-threaded parallel pipeline execution scheme, each stage of the pipeline can not only be multi-threaded, but also achieve throughput far exceeding that of a single-threaded pipeline through multi-level parallelism. It can make full use of computing resources, greatly improve the overall running efficiency of the labeling process, and realize rapid gene redundancy labeling on large-scale datasets.
[0023] Figure 2 This is a schematic diagram illustrating the execution flow of redundant gene marker processing for a data block according to an embodiment of this application. Figure 2 As shown, the redundant gene marker process includes at least the following execution steps: Step 1: Read and parse the data block; Step 2: Find sequence pairing information; Step 3: Sort the data blocks; Step 4: Redundancy detection, output the marking results.
[0024] In this pipeline, step 1 corresponds to the first pipeline stage, step 2 to the second pipeline stage, step 3 to the third pipeline stage, and step 4 to the fourth pipeline stage. The specific processing details and procedures involved in step 4 are existing technologies and will not be described in detail here. For each data block, operations such as reading and parsing, sequence matching, and data sorting are time-consuming (typically accounting for 90% of the runtime). Therefore, in some embodiments, key operations for each data block are optimized. Specifically, this includes: firstly, further refining the segmentation of each data block into multiple sub-data blocks. Then, each sub-data block is processed sequentially by the processing threads corresponding to the first to third pipeline stages. The sub-data blocks in the current data block that have undergone processing in the third pipeline stage (i.e., before redundancy detection) are merged, and the merged data block is processed by the processing thread corresponding to the fourth pipeline stage, outputting the labeling result of the data block. This embodiment of the application further refines the internal data division of the data block, greatly improving the efficiency of labeling redundant gene data.
[0025] Continue to refer to Figure 2 In one embodiment, a multi-threaded approach is used to accelerate the processing within data blocks. Specifically, for each data block, an independent processing thread is allocated to each sub-data block in the first to third pipeline stages. Each data block can be processed in parallel using N threads, where N can be determined by pre-set parameters. This embodiment of the application accelerates fine-grained data parallelism within data blocks, further fully utilizing computing resources and significantly improving the performance of the process of labeling redundant gene data.
[0026] In some embodiments, the second stream can be configured to determine two or more sequencing reads that are completely identical or meet a preset similarity threshold based on the base sequence of the sequencing reads in the data block.
[0027] In some embodiments, the third pipeline stage can be configured to sort the sequencing reads within the data block according to sorting rules, which are ordered from highest to lowest priority and include: by read direction marker, by reference sequence identifier of the paired read, by start coordinate of the paired read, by sequencing read insertion length, and by sequencing read name (lexicographical order).
[0028] Figure 3 This is a schematic diagram illustrating the execution flow of marker-redundant gene data according to an embodiment of this application. Figure 3As shown, in some embodiments, each data block is processed by the first to fourth pipeline threads using a guessing execution approach. Guessing execution means that during the processing of the current data block by the first to fourth pipeline threads, data overlap with other data blocks is not considered. Then, after all data blocks have been processed by the first to fourth pipeline threads, a detection and correction phase begins. This phase involves detecting overlapping areas in the marking results of adjacent data blocks and correcting the marking results based on the overlapping data. This embodiment of the application, based on the partitioned data blocks, uses a data block guessing execution combined with a detection and correction algorithm, which improves the overall efficiency of the marking process while ensuring the consistency of the marking results.
[0029] In some embodiments, step S104 may include: setting an overlapping region based on the boundary of two adjacent data blocks, for example, taking a 1000bp interval at the end of each data block and the beginning of the next data block as the overlapping region. Then, traversing the overlapping region, checking whether there is a group of the same redundant reads spanning two data blocks based on the start coordinates and sequence of the sequencing reads. That is, comparing reads with completely identical coordinates and sequences, such as a read part at the end of data block A and another part at the beginning of data block B. If a group of redundant reads spanning two blocks exists, the redundant read group is merged, and the redundancy attribute of the redundant read group is re-marked according to the redundancy determination rule. That is, if a group of redundant reads spanning two blocks is detected, the determination results of the two data blocks are corrected, including: merging reads belonging to the same group in the two data blocks; re-marking the redundancy attribute of this group of reads according to the redundancy determination rule (e.g., determining whether to mark it as redundant based on the quality value MAPQ); updating the results of the two data blocks to ensure that redundant reads spanning the boundary are not missed or mislabeled.
[0030] According to another embodiment of this application, a method for labeling redundant gene data is provided, comprising: dividing a gene data file to be labeled into multiple data blocks, wherein the gene data file to be labeled is sorted based on the starting coordinates of sequencing reads; setting the redundant gene labeling process as a multi-stage parallel pipeline, each execution step of the redundant gene labeling process corresponding to a pipeline stage in the multi-stage pipeline, and allocating multiple processing threads to each pipeline stage; and sequentially providing two data blocks from the multiple data blocks to the multi-stage parallel pipeline for redundant gene labeling processing. The redundant gene labeling process includes at least the following execution steps: Step 1: reading and parsing data blocks; Step 2: finding sequence pairing information; Step 3: sorting data blocks; Step 4: redundancy information detection and outputting labeling results; Step 5: overlapping region detection and correcting labeling results; and wherein the pipeline stage corresponding to Step 5 only starts after receiving the labeling results of two data blocks from the previous pipeline stage. This embodiment of the application does not require waiting for all data blocks to be processed, thereby further shortening the overall process runtime.
[0031] Figure 4 This is a schematic diagram illustrating the execution of a multi-stage parallel pipeline for labeling redundant gene data according to an embodiment of this application. Figure 4 As shown, taking the division into three data blocks as an example, the parallel pipelined processing of the three data blocks is illustrated, with each execution step handled by an independent processing thread. After processing data block 1, the processing thread corresponding to the first-level pipeline can proceed to process data block 2, while data block 1 enters the processing thread corresponding to the second-level pipeline. After processing data block 2, the processing thread corresponding to the first-level pipeline can proceed to process data block 3. After processing data block 1, the processing thread corresponding to the second-level pipeline can proceed to process data block 2, while data block 1 enters the processing thread corresponding to the third-level pipeline, and so on. Since the detection and correction stage (i.e., step 5) needs to detect the overlapping area of adjacent data blocks, it requires information from two data blocks for each input. Therefore, the processing thread corresponding to the fourth-level pipeline needs to process data block 1 and data block 2 before starting the processing thread of the fifth-level pipeline. In this embodiment, there is no need to wait for all data blocks to be processed. The detection and correction stage can begin after two data blocks have been processed by the fourth-level pipeline, making full use of computing resources and further improving the execution speed.
[0032] According to another embodiment of this application, a system for labeling redundant gene data is provided, comprising a block processing module, a pipeline setup module, a redundancy labeling module, and a detection and correction module. The block processing module is configured to divide the gene data file to be labeled into multiple data blocks, wherein the gene data file to be labeled is sorted based on the starting coordinates of sequencing reads. The pipeline setup module is configured to set the redundant gene labeling process as a multi-stage parallel pipeline, where each execution step of the redundant gene labeling process corresponds to a pipeline stage in the multi-stage pipeline, and multiple processing threads are allocated to each pipeline stage. The redundancy labeling module is configured to provide each data block to the multi-stage parallel pipeline for redundant gene labeling processing. The detection and correction module is configured to correct the redundant gene labeling processing results from the multi-stage parallel pipeline based on the overlapping area between adjacent data blocks.
[0033] This application's embodiments fully utilize the data characteristics of SAM / BAM files based on coordinate sorting, avoiding the performance bottleneck caused by global sorting. Through data block partitioning, process execution step partitioning, and multi-threaded parallel pipeline execution scheme, it makes full use of computing resources and greatly improves the overall running efficiency of the labeling process, enabling rapid gene redundancy labeling on large-scale datasets.
[0034] References to "various embodiments," "some embodiments," "one embodiment," or "embodiment," etc., in this specification refer to a specific feature, structure, or property described in connection with an embodiment that is included in at least one embodiment. Therefore, the appearance of the phrases "in various embodiments," "in some embodiments," "in one embodiment," or "in an embodiment," etc., throughout this specification does not necessarily refer to the same embodiment. Furthermore, specific features, structures, or properties can be combined in any suitable manner in one or more embodiments. Therefore, a specific feature, structure, or property shown or described in connection with one embodiment can be combined, in whole or in part, with features, structures, or properties of one or more other embodiments without limitation, provided that such combination is not illogical or inoperable.
[0035] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments or implementation methods. Similar or identical parts between the various embodiments of the present invention can be referred to mutually. The implementation principles and technical effects of the inventive concept can be referred to mutually, and will not be repeated here. Where there is no conflict, the various embodiments or implementation methods in this invention can be combined with each other.
[0036] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.
[0037] Embodiments of this application may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.
[0038] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.
[0039] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used in the embodiments of the present invention is chosen to best explain the principles, practical applications, or technological improvements in the market, or to enable other those skilled in the art to understand the disclosed embodiments of the present invention.
Claims
1. A method for labeling redundant gene data, wherein, include: The gene data file to be labeled is divided into multiple data blocks, and the gene data file to be labeled is sorted based on the starting coordinates of the sequencing reads. The redundant gene labeling process is set up as a multi-stage parallel pipeline, and each execution step of the redundant gene labeling process corresponds to a pipeline stage in the multi-stage pipeline, and multiple processing threads are allocated to each pipeline stage. Each data block is provided to the multi-stage parallel pipeline for redundant gene labeling processing; The redundant gene marker processing results from the multi-stage parallel pipeline are corrected based on the overlapping areas between adjacent data blocks.
2. The method according to claim 1, wherein, The redundant gene marker process includes at least the following steps: Step 1: Read and parse the data block; Step 2: Find sequence pairing information; Step 3: Sort the data blocks; Step 4: Redundancy detection, output the marking results.
3. The method according to claim 2, wherein, The step of providing each data block to the multi-stage parallel pipeline for redundant gene labeling processing includes: Divide the data block into multiple sub-data blocks; Each sub-data block is sequentially fed to the first to third pipeline stages of the multi-stage parallel pipeline for processing; All sub-data blocks processed in the third pipeline stage are merged, and the merged data blocks are then provided to the fourth pipeline stage for further processing.
4. The method according to claim 3, further comprising: Each sub-data block in the first to third pipeline stages is assigned an independent processing thread.
5. The method according to claim 2, wherein, The third flow stage corresponding to step 3 is set as follows: The sequencing reads within the data block are sorted according to the sorting rules; The sorting rules, arranged from highest to lowest priority, include: by read direction marker, by reference sequence identifier of paired read, by start coordinate of paired read, by sequencing read insertion length, and by sequencing read name.
6. The method according to any one of claims 1-5, wherein, Correcting redundant gene marker processing results from the multi-stage parallel pipeline based on overlapping regions between adjacent data blocks includes: Set the overlapping area based on the boundary of two adjacent data blocks; Then, the overlapping region is traversed, and the presence of the same redundant read group spanning two data blocks is checked based on the starting coordinates and sequence of the sequencing reads. If there are redundant read segments across blocks, the redundant read segments are merged, and their redundancy attributes are re-marked according to preset redundancy determination rules.
7. A method for labeling redundant gene data, wherein, include: The gene data file to be labeled is divided into multiple data blocks, and the gene data file to be labeled is sorted based on the starting coordinates of the sequencing reads. The redundant gene labeling process is set up as a multi-stage parallel pipeline, and each execution step of the redundant gene labeling process corresponds to a pipeline stage in the multi-stage pipeline, and multiple processing threads are allocated to each pipeline stage. Two data blocks from the plurality of data blocks are provided to the multi-stage parallel pipeline in sequence for redundant gene labeling processing; The redundant gene marker process includes at least the following execution steps: Step 1: Read and parse the data block; Step 2: Find sequence pairing information; Step 3: Sort the data blocks; Step 4: Redundancy detection, output marking results; Step 5: Detect and correct overlapping regions; Furthermore, the pipeline stage corresponding to step 5 only starts after receiving the marking results of two data blocks from the previous pipeline stage.
8. The method according to claim 7, wherein, The pipeline stage corresponding to step 5 is set to detect the overlapping area between two data blocks and correct the redundant gene marker processing results of the two data blocks from the previous pipeline stage accordingly.
9. A system for labeling redundant gene data, wherein, It includes a block processing module, a pipeline setting module, a redundancy marking module, and a detection and correction module; among which, The block processing module is configured to divide the gene data file to be labeled into multiple data blocks, wherein the gene data file to be labeled is sorted based on the starting coordinates of the sequencing reads. The pipeline setup module is configured to set the redundant gene labeling process as a multi-stage parallel pipeline, where each execution step of the redundant gene labeling process corresponds to a pipeline stage in the multi-stage pipeline, and multiple processing threads are allocated to each pipeline stage. The redundancy labeling module is configured to provide individual data blocks to the multi-stage parallel pipeline for redundant gene labeling processing; The detection and correction module is configured to correct the redundant gene marker processing results from the multi-stage parallel pipeline based on the overlapping area between adjacent data blocks.
10. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-8.