Method and system for identifying and removing redundant OTUs

Abstract

The invention discloses a method and a system for identifying and removing redundant OTUs, and relates to biological detection. The method comprises the following steps: comparing each sequence OTUi in a first OTU set with a species database by using blast to obtain a first matching record set; deleting the matching records with the coverage rate lower than a preset coverage rate threshold value in the first matching record set to obtain a second matching record set; sorting the matching records in the second matching record set according to the unique identifier of the reference sequence to obtain a third matching record set; connecting the matching records in the third matching record set in series according to a preset format to obtain a comparison signature of the OTUi; performing deduplication and merging on the comparison signatures of the OTUi to obtain a second OTU set; the first OTU set is an OTU set before redundancy elimination, the second OTU set is an OTU set after redundancy elimination, and the problem that annotation level redundancy cannot be eliminated through a traditional sequence similarity clustering method is effectively solved.

Description

A method and system for identifying and removing redundant OTUs Technical Field

[0001] This application relates to the field of biological detection, and more specifically, to a method and system for identifying and removing redundant OTUs. Background Technology

[0002] Amplicon sequencing is a high-throughput sequencing technique targeting specific DNA fragments, commonly used for environmental DNA metabarcoding analysis. It amplifies the target molecular marker sequence by PCR amplification of the target gene region in environmental samples using universal primers (such as primers for COI, 16S rRNA, 18S rRNA, or ITS genes). These amplified products are then fed into a high-throughput sequencing platform (such as Illumina MiSeq or NovaSeq), generating a large number of short reads. This method is particularly suitable for biodiversity assessment of mixed samples, enabling the simultaneous detection of genetic information from multiple organisms without isolating individual species. Amplicon sequencing efficiently reveals the structure of microbial, fungal, plant, or animal communities in environments such as water, soil, or sediments, making it a core tool in ecology, microbiome, and biomonitoring.

[0003] Sequences obtained from amplicon sequencing are typically short reads, generally between 150 and 500 bp in length. Due to PCR amplification and sequencing errors, these sequences may contain variations, including single-base substitutions, insertions / deletions, or amplification biases, resulting in slight differences in sequences even within the same species. To process this data, clustering is usually performed, grouping similar sequences together to form operational taxonomic units (OTUs). After obtaining representative sequences, species annotation (such as BLAST) is used to convert them into biologically meaningful species information.

[0004] In real-world DNA macrobarcoding analysis, especially for groups such as phytoplankton, zooplankton, or protists, a common and challenging problem is the over-division of OTUs / ASVs within the same species, where multiple different OTUs / ASVs are ultimately annotated under the same taxonomic name. The causes of this phenomenon mainly include: (1) natural genetic variations within an organism, such as heterogeneity of mitochondrial or plastid genes and intraspecific multicopy variations, which prevent sequences of the same species from being completely clustered into a single unit at a similarity threshold of 97%; (2) technical errors introduced during PCR amplification and sequencing, which further amplify small differences; (3) greedy clustering algorithms (such as UPARSE and VSEARCH's cluster_size) are sensitive to the sequence addition order and easily divide highly similar sequences into multiple clusters; (4) incomplete coverage or inconsistent annotation of reference databases, leading to similar sequences being matched with the same taxonomic name. This redundancy not only significantly increases the feature dimensions of downstream analysis (the number of OTUs can expand several times), increasing computational complexity and storage requirements, but may also lead to inflated estimates of community diversity indicators (such as the Shannon index and Chao1 richness estimates). Summary of the Invention

[0005] To address the common problem of redundant OTU sequences of the same species in amplicon sequencing data analysis in existing technologies, this application provides a method and system for identifying and removing redundant OTUs, effectively solving the annotation-level redundancy problem that traditional sequence similarity clustering methods cannot eliminate.

[0006] One aspect of this application provides a method for identifying and removing redundant OTUs, comprising: using BLAST to compare each sequence OTU_i in a first OTU set with a species database to obtain matching records of sequence OTU_i, which serve as a first set of matching records; the matching records include at least: a unique identifier of the reference sequence, a percentage of consistency, and a coverage rate;

[0007] Delete matching records in the first matching record set whose coverage is lower than a preset coverage threshold to obtain the second matching record set; sort the matching records in the second matching record set according to the unique identifier of the reference sequence to obtain the third matching record set; concatenate the matching records in the third matching record set according to a preset format to obtain the alignment signature of OTU_i; perform deduplication and merging on the alignment signature of OTU_i to obtain the second OTU set; wherein, the first OTU set is the OTU set before redundancy removal, and the second OTU set is the OTU set after redundancy removal.

[0008] OTUs are an abbreviation for Operational Taxonomic Units. In microbial ecology, environmental DNA metabarcoding, and amplicon sequencing (such as 16S rRNA, COI, or ITS gene sequencing) analyses, OTUs refer to groups of short sequences obtained from high-throughput sequencing, clustered based on sequence similarity. Each OTU represents an "operational" taxonomic unit and is usually considered to be approximately a species (or a higher taxonomic level). Sequences with ≥97% similarity are typically grouped into the same OTU.

[0009] Furthermore, it also includes: if the number of matching records is less than a preset threshold, then all matching records are returned as the first set of matching records.

[0010] Furthermore, the BLAST alignment employs the BLASTN algorithm; BLASTN (Nucleotide-NucleotideBLAST) is an algorithm specifically designed for nucleotide sequence alignment within the BLAST (Basic Local Alignment Search Tool) package. BLAST is the most commonly used and classic sequence alignment tool in the field of bioinformatics.

[0011] Furthermore, the species database is a blast database constructed from selected species barcode DNA sequence files; the species barcode DNA sequences include COI, 16S, or 18S.

[0012] Furthermore, the species type contained in the species barcode DNA sequence file is fish, benthic animals, phytoplankton, or zooplankton.

[0013] Furthermore, the formula for calculating coverage is: coverage = (end position of query sequence alignment - start position of query sequence alignment + 1) / total length of query sequence × 100%.

[0014] Furthermore, the preset coverage threshold ranges from 70% to 80%.

[0015] Furthermore, the preset format is: unique identifier of reference sequence | consistency percentage, with each sorted matching record concatenated by a semicolon. In particular, the preset format in this application transforms the species annotation information of OTUs into directly comparable string signatures through a standardized pairing concatenation method (reference sequence identifier + consistency percentage). This simplifies the problem of whether they belong to the same species, which originally required complex sequence alignment and similarity calculations, into a simple and efficient string equality judgment. Thus, while ensuring the accuracy of biological significance, it simplifies the algorithm and improves the efficiency of redundant OTU identification.

[0016] Furthermore, the alignment signatures of OTU_i are deduplicated and merged, including merging OTUs with completely identical alignment signatures into a single representative OTU. Specifically, the alignment signature is formed by concatenating the filtered and sorted matching records obtained after BLAST alignment of the OTU sequences according to a preset format (unique identifier of the reference sequence + connector + consistency percentage). The original OTU sequence with the highest consistency percentage among the OTUs with completely identical alignment signatures is used as the representative sequence of the representative OTU, resulting in the merged OTU set.

[0017] Another aspect of this application provides a system for identifying and removing redundant OTUs, comprising: an alignment module for aligning each sequence OTU_i in the OTU set to a species database using BLAST to obtain N matching records for sequence OTU_i, wherein each matching record includes at least: a unique identifier of the reference sequence, a percentage of consistency, and a coverage rate; a filtering module for filtering the matching records obtained by the alignment module, deleting matching records with a coverage rate lower than a threshold, and obtaining filtered matching records; a sorting module for sorting the filtered matching records obtained by the filtering module according to the unique identifier of the reference sequence, and obtaining sorted matching records; a signature generation module for concatenating the sorted matching records obtained by the sorting module according to a preset format to form an alignment signature of OTU_i; and a deduplication and merging module for deduplicating and merging the alignment signatures formed by the signature generation module, and obtaining a merged OTU set.

[0018] Compared to existing technologies, the advantages of this application are as follows: The method and system for identifying and removing redundant OTUs in this application, by constructing a unique alignment signature for each OTU based on reference sequence matching features (an ordered combination of a unique identifier of the reference sequence and a percentage of consistency), and merging OTUs with the same alignment signature, can accurately identify and eliminate redundant OTUs caused by biological variation, sequencing errors, and limitations of clustering algorithms at the species annotation level. Compared with traditional methods that rely solely on sequence similarity for clustering, this method significantly reduces the redundancy of the OTU set, effectively compresses the feature dimension, thereby improving the accuracy of community diversity assessment, avoiding inflated estimates of indicators such as the Shannon index and Chao1 richness, and significantly reducing the computational complexity and storage requirements of downstream ecological analysis, providing a more reliable data foundation for environmental DNA macrobarcoding analysis. Attached Figure Description

[0019] Figure 1 is a flowchart of a method for identifying and removing redundant OTUs in this application; Figure 2 is a partial diagram of OTUs.fas in Embodiment 1 of this application; Figure 3 is a partial diagram of OTUs.fas in Embodiment 2 of this application. Detailed Implementation

[0020] The present application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0021] Example 1, as shown in Figure 1, illustrates a method for identifying and removing redundant OTUs proposed in this application. This method aims to address the sequence redundancy problem caused by multiple OTUs corresponding to the same species in amplicon sequencing data. The method includes: performing BLAST alignment on each OTU sequence in the first OTU set; filtering by coverage based on the alignment results; sorting by unique identifier of the reference sequence; concatenating them according to a preset format to construct an alignment signature for each OTU sequence; and then performing deduplication and merging based on the alignment signature to finally obtain a simplified second OTU set.

[0022] For each sequence OTU_i in the first OTU set, the BLAST tool is used to compare it with the species database to obtain matching records for sequence OTU_i, which serve as the first set of matching records. Each matching record contains at least: a unique identifier for the reference sequence (accession), a percentage of identity, and coverage.

[0023] In this embodiment, the BLAST alignment uses the BLASTN algorithm, which aligns nucleic acid sequences to a nucleic acid database and is suitable for aligning DNA sequences obtained from amplicon sequencing. After querying each OTU_i sequence, the first N=20 best matching records are returned by default (i.e., the preset threshold N=20) to cover the most representative matching results in the database while avoiding unnecessary computational overhead. If the actual number of matching records is less than N, all matching records are returned as the first set of matching records. The value of N can be adjusted appropriately according to the database size and the diversity of the sequences to be analyzed. When there are a large number of similar sequences, N can be increased to improve the resolution of the alignment signature; the general value range is 10 to 200.

[0024] The BLAST alignment results are presented in tabular form. The columns, from left to right, are: query sequence ID, accession, alignment length, number of mismatches, query sequence alignment start position, query sequence alignment end position, expected value (E-value), identity percentage, and bit score. The accession number (Accession Number) is a unique identifier assigned to a nucleic acid sequence (DNA / RNA) or protein sequence in the NCBI database or local gene database. It's similar to a sequence's "ID number," used for permanent and unique referencing and retrieval. Identity represents the percentage of identical bases (or amino acids) between the query sequence and the database sequence within the alignment region. Identity (%) = (Number of identical bases / Alignment region length) × 100%.

[0025] The species database used is a local BLAST database constructed from selected species barcode DNA sequence files. Sequences can be downloaded from public databases (such as GenBank, RefSeq, or other authoritative barcode repositories) or obtained through self-sequencing. After organizing the target sequences into FASTA format files, the local BLAST database is constructed using the `makeblastdb` command. Species barcode DNA sequences include COI, 16S, or 18S sequences, suitable for barcode analysis of fish, benthic animals, phytoplankton, zooplankton, and other groups.

[0026] Delete matching records in the first matching record set whose coverage is lower than a preset coverage threshold to obtain the second matching record set.

[0027] The formula for calculating coverage is: Coverage = (End position of query sequence alignment - Start position of query sequence alignment + 1) / Total length of query sequence × 100%.

[0028] The default coverage threshold is 70%, meaning it retains matching records with a coverage rate of ≥70%. This threshold maintains high sensitivity (without losing too many true matches) while improving specificity, filtering out noise introduced by short segments or non-specific comparisons, and allowing for some missing ends, making it suitable for macro barcode annotation scenarios. For more stringent screening, the threshold can be increased according to actual needs, generally ranging from 70% to 80%.

[0029] The matching records in the second matching record set are sorted in descending order based on the unique accession of the reference sequence to obtain the third matching record set. The fixed sorting rule ensures that the same alignment result set has a consistent concatenation order across different OTUs, laying the foundation for accurate comparison of subsequent signatures.

[0030] The matching records in the third matching record set are concatenated according to a preset format to obtain the alignment signature of OTU_i. The preset format is: unique identifier of reference sequence | percentage of consistency, with each matching record separated by a semicolon. In this embodiment, the alignment signature is formed in the following format: XXX123|0.987; YYY456|0.952. This alignment signature comprehensively reflects the alignment information between OTU_i and the species database, and can be used as a unique and comparable fingerprint of the OTU at the species level.

[0031] The signatures of all OTUs are deduplicated and merged to obtain a second set of OTUs, which is the set of OTUs after redundancy removal. The specific rules are as follows: OTUs with completely identical signatures are regarded as redundant OTUs with the same annotation results and merged into a representative OTU; among the OTUs with completely identical signatures, the original OTU sequence with the highest identity value is selected as the representative sequence of the representative OTU, resulting in the merged second set of OTUs.

[0032] The first OTU set is the original OTU set before redundancy removal, and the second OTU set is the simplified OTU set after redundancy removal following the above steps.

[0033] Example 2: In this example, the raw sequencing data contains barcode DNA sequences of phytoplankton. The raw sequencing data obtained from next-generation sequencing is preprocessed. Simultaneously, a mapping file containing the correspondence between barcode sequences and sample IDs is created.

[0034] If it is paired-end sequencing, two sequencing files will be generated. You can use the command fastq_mergepairs in vsearch (v2.30.2) to merge the fastq files of forward and reverse sequencing.

[0035] The `split_libraries.py` script of QIIME1 is used to assign sequences to different samples based on the barcode and primer information in the mapping file, while performing quality filtering (removing low-quality sequences, excessively short / long sequences, and sequences with too many primer / barcode mismatches). The parameter `e2` is set to 2, meaning that bases with a quality score below 2 are considered errors.

[0036] The subsequent analysis was performed using vsearch (v2.30.2). The command `derep_fulllength` was used for redundancy removal; the command `sortbysize` was used to sort by abundance, with the parameter `minsize` set to 2 (minimum abundance 2); the command `cluster_size` was used for OTU clustering, with the parameter `id` set to 0.97 (97% similarity); the command `uchime3_denovo` was used to remove chimeras, ultimately generating the OTU representative sequence file `OTUs.fas` (Figure 2). The command `usearch_global` was used to generate the OTU abundance table `otus.uc`. Finally, the command `otutab` was used to generate the OTU table, i.e., `otus.table.txt` (Table 1).

[0037] The following is a partial content of the OTU sequence file OTUs.fas: Table 1 shows a partial content of otus.table.txt.

[0038] The local BLAST tool (BLASTN) was used to compare the data with a phytoplankton barcode database. The top 20 best-matching records (top 20 hits) for each OTU query sequence were retained. The final result was generated as an otus.blast.result.csv file. The BLAST comparison results are shown below (first 10 records) (Table 2): Table 2: Partial content of otus.blast.result.csv BLAST comparison results

[0039] Data source for the phytoplankton barcode database: The database is constructed based on sequence data downloaded from the NCBI Nucleotide (nt) database. For phytoplankton barcode analysis, a subset of sequences highly correlated with the target marker gene (18S rRNA) was prioritized for download and screening. FASTA format sequences related to 18S rRNA from major phytoplankton groups were obtained using the NCBI website or the Entrez batch download tool. Subsequently, a local BLAST database was constructed using the makeblastdb command.

[0040] Based on the local BLAST alignment results (otus.blast.result.csv), the optimal taxonomic annotation information (the alignment result with the highest position score) is selected for each OTU and added to the OTU abundance table (otus.table.txt), finally generating the abundance table otus.table.tax.txt with taxonomic annotations.

[0041] The file is in tab-separated TXT format. The first column is the OTU id, followed by the read abundance count for each sample, and the last column (or multiple columns) is the hierarchical taxonomic information (Kingdom; Phylum; Class; Order; Family; Genus; Species) (Table 3).

[0042] Table 3 contains partial content from otus.table.tax.txt

[0043] Using the method provided in this application, the otus.table.tax.txt table is deredundant based on the results of BLAST alignment annotations. This method optimizes and merges the common redundancy phenomenon (i.e., different OTU ids correspond to the same taxonomic annotation) in OTU tables generated by traditional 97% similarity clustering, further simplifying the OTU abundance table, reducing data dimensionality, and improving the efficiency and reliability of downstream statistical analysis.

[0044] The input files are the OTU abundance table otus.table.tax.txt and the BLAST alignment data otus.blast.result.csv. After redundancy removal, the output file is otus.table.taxOTUs_combine.csv.

[0045] The original data otus.table.tax.txt contained 1957 sequences, while the deduplicated result otus.table.taxOTUs_combine.csv contained 904 sequences, a compression rate of approximately 53.8% (calculated as (1 - 904 / 1957) × 100%). This indicates that more than half of the original sequences ultimately point to the same taxonomic information. Furthermore, the deduplication operation primarily affects the fine taxonomic level without altering the overall macroscopic biological composition of the sample (Table 4).

[0046] Table 4 Comparison before and after removing redundant OTUs

[0047] Existing amplicon sequencing data post-processing methods (such as sequence similarity-based clustering) often generate a large number of redundant OTUs due to minor sequence differences, leading to overestimation of species richness. This application constructs a comprehensive alignment signature based on BLAST alignment results and merges OTUs based on signature consistency, thus simplifying the dataset without losing taxonomic information. This redundancy removal strategy based on comprehensive alignment signatures demonstrates higher consistency and biological interpretability compared to existing simple clustering methods.

[0048] This application employs coverage and identity percentage as screening parameters during OTU merging, which can filter out low-quality or non-specific alignment results and retain highly reliable alignment information. This BLAST-based alignment quality screening improves annotation accuracy and the reliability of community diversity analysis.

[0049] The method presented in this application exhibits good compatibility and can be adapted to existing amplicon sequencing data analysis workflows. It introduces a layer of redundancy removal analysis based on alignment signatures on top of the existing OTU clustering and annotation workflow. Therefore, this application can supplement existing amplicon data processing workflows, seamlessly integrating with common steps such as data preprocessing, quality control, dechimeric removal, cluster analysis, and species annotation, improving the overall simplicity and biological interpretability of the analysis results without requiring major restructuring or replacement of existing workflows. The parameters N (number of returned matching entries) and R (coverage threshold) in this application can be adjusted and optimized according to different experimental data characteristics, fragment lengths, alignment database sizes, and the biodiversity of the research subjects. For example, when the sequence to be annotated contains a large number of similar sequences, N can be appropriately increased to improve the resolution of the comprehensive alignment signature; in cases where alignment coverage requirements are more stringent, the threshold of R can be increased to further filter low-coverage alignment results.

[0050] Example 3: The raw sequencing data contained barcode DNA sequences of zooplankton. The raw sequencing data obtained from next-generation sequencing was preprocessed.

[0051] Raw data preprocessing: The data from the second-generation sequencing machine are preprocessed, and a mapping file is created (providing the correspondence between barcode sequences and sample IDs).

[0052] Paired-end sequencing processing (if applicable): Use the fastq_mergepairs command in vsearch (v2.30.2) to merge the forward and reverse FASTQ files.

[0053] Sequence splitting and quality filtering: The QIIME1 split_libraries.py script is used to assign sequences to different samples; at the same time, quality filtering is performed, with parameter e2 (bases with a quality score <2 are considered errors).

[0054] OTU Clustering and Dechirping (using vsearch v2.30.2): Redundancy removal, sorting by abundance, OTU clustering and dechirping, generating the OTU representative sequence file OTUs.fas, as shown in Figure 3.

[0055] Abundance table generation (using vsearch v2.30.2): Generate the standard OTU abundance table otus.table.txt (Table 5).

[0056] Table 5 contains partial content from otus.table.txt

[0057] The local BLAST tool (BLASTN) was used to compare the data with a phytoplankton barcode database. The top 20 best-matching records (top 20 hits) for each OTU query sequence were retained. The final result was generated as an otus.blast.result.csv file (Table 6). The BLAST comparison results are shown below (first 10 records): Table 6: Partial content of the BLAST comparison results otus.blast.result.csv

[0058] Data source for the zooplankton barcode database: The database is constructed based on sequence data downloaded from the NCBI Nucleotide (nt) database. For zooplankton barcode analysis, a subset of sequences highly correlated with the target marker gene (COI gene) was prioritized for download and screening. FASTA format sequences related to the COI gene of the major zooplankton groups were obtained using the NCBI website or the Entrez batch download tool. Subsequently, a local BLAST database was constructed using the makeblastdb command.

[0059] Based on the local BLAST alignment results (otus.blast.result.csv), the optimal taxonomic annotation information (the alignment result with the highest position score) is selected for each OTU and added to the OTU abundance table (otus.table.txt), finally generating the abundance table otus.table.tax.txt with taxonomic annotations (Table 7).

[0060] Table 7 contains partial content from otus.table.tax.txt

[0061] Using the method provided in this application, redundancy is removed from the otus.table.tax.txt table based on the BLAST alignment annotation results. The input files are the OTU abundance table otus.table.tax.txt and the BLAST alignment data otus.blast.result.csv. After redundancy removal, the output file is otus.table.taxOTUs_combine.csv.

[0062] The original data otus.table.tax.txt contains 2344 sequences, and the deduplicated result otus.table.taxOTUs_combine.csv contains 1064 sequences, with a compression rate of approximately 54.41% (calculated as (1 -1064 / 2344) × 100%) (Table 8).

[0063] Table 8 Comparison before and after removing redundant OTUs

[0064] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform the method of this application.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A method for identifying and removing redundant OTUs, characterized in that, include: The first OTU set is compared with the species database using BLAST to obtain matching records for each OTU_i. These matching records constitute the first matching record set. Each matching record includes at least: a unique identifier of the reference sequence, a percentage of similarity, and a coverage rate. Matching records with coverage rates below a preset coverage threshold are deleted from the first matching record set, resulting in the second matching record set. The matching records in the second matching record set are sorted according to the unique identifier of the reference sequence, resulting in the third matching record set. The matching records in the third matching record set are concatenated according to a preset format to obtain the alignment signature of the OTU_i. The signatures of OTU_i are deduplicated and merged to obtain the second OTU set; wherein, the first OTU set is the OTU set before redundancy removal, and the second OTU set is the OTU set after redundancy removal.

2. The method for identifying and removing redundant OTUs according to claim 1, characterized in that, Also includes: If the number of matching records is less than the preset threshold, all matching records are returned as the first set of matching records.

3. The method for identifying and removing redundant OTUs according to claim 2, characterized in that: The blast alignment uses the blastn algorithm.

4. The method for identifying and removing redundant OTUs according to claim 3, characterized in that: The species database is a blast database constructed from selected species barcode DNA sequence files; the species barcode DNA sequences include COI, 16S, or 18S sequences.

5. The method for identifying and removing redundant OTUs according to claim 4, characterized in that: The species type contained in the species barcode DNA sequence file is fish, benthic animals, phytoplankton, or zooplankton.

6. The method for identifying and removing redundant OTUs according to claim 4, characterized in that: The formula for calculating coverage is: Coverage = (End position of query sequence alignment - Start position of query sequence alignment + 1) / Total length of query sequence × 100%.

7. The method for identifying and removing redundant OTUs according to any one of claims 1 to 6, characterized in that: The preset coverage threshold ranges from 70% to 80%.

8. The method for identifying and removing redundant OTUs according to claim 1, characterized in that: The matching records in the third matching record set are concatenated according to a preset format to obtain the matching signature of OTU_i. The preset format is: unique identifier of reference sequence | consistency percentage. The sorted matching records are concatenated with semicolons as separators.

9. The method for identifying and removing redundant OTUs according to claim 8, characterized in that: The signatures of OTU_i are deduplicated and merged to obtain a second set of OTUs, which includes: merging OTUs with completely identical signatures into a representative OTU; and using the original OTU sequence with the highest consistency percentage among the OTUs with completely identical signatures as the representative sequence of the representative OTU, thus obtaining the second set of OTUs.

10. A system for identifying and removing redundant OTUs, characterized in that, include: The alignment module aligns each sequence OTU_i in the OTU set with the species database using BLAST to obtain N matching records for sequence OTU_i. Each matching record contains at least: a unique identifier of the reference sequence, a percentage of similarity, and a coverage rate. The filtering module filters the matching records obtained by the alignment module, deleting those with a coverage rate below a threshold, resulting in a filtered set of matching records. The sorting module sorts the filtered matching records based on the unique identifier of the reference sequence, resulting in a sorted set of matching records. The signature generation module is used to concatenate the sorted matching records obtained by the sorting module according to a preset format to form the matching signature of OTU_i; the deduplication and merging module is used to deduplicatize and merge the matching signatures formed by the signature generation module to obtain the merged OTU set.

Images