Method for filtering low-quality sequences in nucleotide multi-sequence alignment matrix

By developing revComplement, alignmentLength, anyShortseq, degap and maskSegment functions in the multi-sequence alignment matrix, the problem of identifying and filtering reverse complementary sequences, short sequences and missing segments in the prior art is solved, and more efficient comparisons are achieved Matrix quality filtering improves the reliability and analysis accuracy of the comparison matrix.

CN118969091AInactive Publication Date: 2024-11-15GUANGXI INST OF BOTANY THE CHINESE ACAD OF SCI
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410997131.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing multi-sequence comparison matrix quality filtering software cannot effectively identify and process reverse complementary sequences, short sequences and short matrices, and the comparison error recognition and filtering effect is poor, making it difficult to set thresholds flexibly, and the calculation volume is large, so it is impossible to separate. Process missing sections at the beginning and end of the matrix.

Method used

A five-function method based on R language was developed, including revComplement, alignmentLength, anyShortseq, degap and maskSegment functions, used to correct reverse complementary sequences, filter matrices that meet length thresholds, delete short sequences and missing segments, and mask Highly mutated fragments, the quality of the alignment matrix is ​​improved through sliding window method and grouping-recombination method.

Benefits of technology

It significantly improves the quality of the multi-sequence alignment matrix, can more effectively identify and process comparison errors, improves the reliability of the alignment matrix and the correctness of data analysis, and is simple and flexible in operation. Users can freely set thresholds according to their needs. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118969091A_ABST
    Figure CN118969091A_ABST
Patent Text Reader

Abstract

The invention discloses a method for filtering a low-quality sequence in a nucleotide multi-sequence alignment matrix, which comprises the following steps of: writing five functions based on R language, namely a revComplement function, an alignentLength function, an anyShartseq function, a degap function and a maskSegment function; the revComplement function is used for correcting a reverse complementary sequence in the matrix; the align Length function is used for screening a matrix which accords with a length threshold value; the anyShotseq function is used for deleting a short sequence existing in the matrix; the gap function is used for deleting missing columns reaching a preset threshold value in the matrix; the maskSegment function is used for shielding fragments with height variation and unreliable comparison in the matrix.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical fields of phylogenetic genomics, bioinformatics and comparative genomics, and is used for quality filtering of homologous sequence comparison matrices obtained by DNA or RNA sequencing screening. Background Art

[0002] Multiple sequence alignment (MSA) is a basic step in the study of evolutionary biology and bioinformatics. It can be used for comparative genomic and evolutionary studies such as constructing phylogenetic trees, analyzing selection pressures (KaKs), and analyzing gene families. The correct alignment of homologous sequences, that is, the determination of positional homology, has always been a challenging task, because the heuristic algorithms used by existing alignment methods usually cannot obtain the optimal solution, or there are multiple optimal solutions, and the actual alignment may be suboptimal under the algorithm used. In addition, many factors, such as deep differences, insertions, inversions, non-homologous recombination, incorrect assembly or annotation of sequences, and large amounts of data, may increase the difficulty of correct alignment and cause alignment errors, although some alignment software may be better than others in some cases. Therefore, alignment problems are more or less inevitable, especially in the era of genomics, when thousands of alignment matrices cannot be manually checked and corrected one by one. There are often alignment errors or questionable segments, missing (gap) segments, too short sequence lengths, and short lengths of the entire matrix, and sometimes reverse complementary sequences are introduced in homologous multiple sequence alignment matrices. Existing MSA quality filtering software such as Gblock, trimAl, BMGE, HmmCleaner, TAPER, Spruceup and Guidance2 usually determine the correctness of MSA alignment based on sequence similarity or multiple alignment stability. Among them, Gblocks, trimAl and HmmCleaner identify excessively variable columns, rows or entire sequences in MSA as unreliable parts based on sequence variation (sometimes including the frequency of missing gaps), and the variation of these parts statistically falls into the range of random alignment or outliers. Columns, rows or entire sequences with excessively high variation are filtered out, while the remaining parts with lower variation are retained for downstream analysis. The other is to measure the alignment stability / consistency of a group of sequences under different alignment settings, for example, by setting different starting trees, alignment orders or subsets, head-to-tail (HoT) directions of sequences, or deletion / replacement penalty parameters, and perform multiple alignments to statistically analyze the stability values ​​of each column, each residue pair, or the entire matrix (such as the total score of residue pairs SPC, the total score of columns TC) of a given reference matrix (reference MSA; or base matrix base MSA) generated by multiple alignments, and thereby give different weight values ​​to different columns in subsequent analysis, or directly filter out low stability columns or fragments for subsequent analysis.

[0003] Although the above software can handle alignment errors or questionable segments and missing (gap) segments to a certain extent, they currently have problems such as poor processing effect, failure to recognize reverse complementary sequences, failure to recognize short sequences and short matrices, failure to separately process the gap segments at the beginning and end of the matrix and in the middle, and usually failure to flexibly set thresholds. Software based on column filtering (block-filtering), such as Gblock, trimAL, BMGE, etc., will delete the sequence sites with correct alignment in the column and usually cannot identify the alignment errors with a very low proportion (very few sequences); software based on segment filtering (segment-filtering), such as HmmCleaner, has four unclear parameters that jointly control the stringency, which is difficult to predict and set. TAPER software has difficulty in identifying short, long and high-frequency incorrect alignments, and the sensitivity and specificity of Spruceup software are affected by the background of alignment errors; software based on column, segment or sequence filtering or weighting (segment-, block-, or sequence filtering or weighting), such as Guidance2, how to set appropriate alignment conditions (different starting trees, replacement and insertion and deletion parameters, etc.), sensitivity and specificity, etc. need to be tested and explored, and the amount of calculation is usually large (time-consuming). Moreover, there is still a lack of program software for correcting reverse complementary sequences, deleting short sequences and short matrices. Therefore, it is necessary to develop more comprehensive programs to solve different alignment problems, especially filtering highly variable segments. Summary of the invention

[0004] To solve the above technical problems, the present invention proposes a method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix, which significantly improves the data filtering quality, obtains a more reliable alignment matrix, provides warnings and references for a large number of related studies, and provides a new method for ensuring the correctness of the basic steps of nucleotide data analysis.

[0005] To achieve the above-mentioned purpose, the present invention provides a method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix, comprising: obtaining a nucleotide multiple sequence alignment matrix, screening the nucleotide multiple sequence alignment matrix to find five types of low-quality sequences, namely, sequences that require reverse complementary sequences, matrices that meet a length threshold, short sequences that exist in the matrix, columns that are missing in the matrix, and sequences that have highly variable alignments and are unreliable; and filtering these five low-quality sequences.

[0006] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention comprises five functions written in R language, namely, revComplement function, alignmentLength function, anyShortseq function, degap function, and maskSegment function;

[0007] The revComplement function is used to correct the reverse complementary sequence in the matrix;

[0008] The alignmentLength function is used to screen matrices that meet the length threshold;

[0009] The anyShortseq function is used to delete short sequences existing in the matrix;

[0010] The degap function is used to delete the columns in the matrix whose missing columns reach a preset threshold;

[0011] The maskSegment function is used to mask highly variable and unreliable segments in the matrix.

[0012] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention, and the method for correcting reverse complementary sequences in a matrix using the revComplement function, comprises: performing a similarity search on a sequence in each matrix and all other sequences to obtain a similarity search result file;

[0013] According to the strand=Plus / Minus judgment moment in the similarity search result file, it is judged that the direction is inconsistent, and the matrix with reverse complementary sequence is found;

[0014] By judging the number of sequences with strand = Plus / Minus and the number of sequences with strand = Plus / Plus in the matrix, the group of sequences with a smaller number will be reverse complemented in the matrix, and the matrix will be re-output and placed in a new folder. The matrix without reverse complementary sequences will be selected and output in another new folder.

[0015] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention, and the method for screening a matrix that meets a length threshold by using the alignmentLength function, comprises: directly judging the total length of the matrix, when the total length of the matrix is ​​greater than or equal to the set threshold length, outputting the matrix in a new folder; and selecting to output a matrix whose total length is less than the set threshold length in a new folder.

[0016] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention, and the method for deleting short sequences existing in the matrix using the anyShortseq function, comprises:

[0017] Extract each sequence name and corresponding base sequence in the matrix, determine the base length of the sequence and compare it with the set threshold;

[0018] When the sequence base length is less than the set threshold length, the sequence in the matrix is ​​deleted and the matrix is ​​output in a new folder. The matrix with a base length greater than or equal to the set threshold is selected and output in a new folder;

[0019] The user sets a threshold value according to the characteristics and needs of his own data. The threshold value is set in two ways: base length or the ratio of base length to the total length of the sequence. These two setting methods are in an or relationship. If one of the situations occurs, it will be deleted.

[0020] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention, and the method for deleting missing segments in the matrix using the degap function include:

[0021] Set parameter thresholds for missing sites at both ends and in the middle of the matrix.

[0022] Process the missing sites at both ends, determine the sites with a front end smaller than the threshold and delete them according to the threshold, and determine the sites with a end smaller than the threshold and delete them;

[0023] Process the missing sites of the entire matrix, determine the sites that are smaller than the threshold according to the set threshold and delete them;

[0024] The matrix of deleted deletion sites is output in a new folder, and the matrix of undeleted deletion sites is output in another new folder.

[0025] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention, and the method for shielding highly variable and unreliable alignment segments in the matrix using the maskSegment function, comprises:

[0026] Calculate the correlation between the random similarity probability and the random permutation similarity threshold;

[0027] Extract each sequence name and base sequence in the aligned matrix to form a complete aligned sequence;

[0028] Dividing the alignment sequence into windows according to a set window size and step size;

[0029] Calculate the pairwise similarity score and compare it with the random permutation similarity threshold, group and merge, and classify the qualified fragments into specific groups;

[0030] The segments in each group are compared with a preset random arrangement similarity threshold to determine whether they need to be marked as segments that need to be blocked.

[0031] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention and the method for calculating the random similarity probability P are:

[0032]

[0033] Where n represents the fragment length; k represents the number of substitution or variation sites.

[0034] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention and the method for calculating the random arrangement similarity RS are:

[0035]

[0036] Where n represents the fragment length; k represents the number of substitution or variation sites.

[0037] The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix provided by the present invention and the method for calculating the pairwise similarity score PS are as follows:

[0038]

[0039] Where n is the fragment length; k is the number of substitution or variation sites; x is i and i Each pair of segments in the sliding window represents two different sequences.

[0040] Technical effect of the present invention: The present invention discloses a method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix. Based on various alignment problems existing in the multiple sequence alignment matrix found in the study, by comparing the advantages and disadvantages of the existing filtering methods, and using actual data to verify the filtering effect of the existing software, it is found that the processing effect is not good or the threshold that can be set is limited, and there is still a lack of program software for correcting reverse complementary sequences, deleting short sequences and short matrices. Therefore, the present invention develops a set of processing functions to respectively handle various alignment problems, wherein the core function maskSegment is to filter the wrong alignment or doubtful fragments in the matrix by a sliding window method according to the degree of sequence variation, using a grouping-recombining method, which is different from the existing software method. Through the same data, different alignment error forms (high-frequency and low-frequency error alignment, long fragment and segment segment error alignment), the processing effects of the present invention and the existing method on the wrong alignment or doubtful fragments are compared, and the results show that: in the case of high-frequency and low-frequency error alignment, long fragment and segment segment error alignment, the filtering effect of the maskSegment function is better. In addition, the existing matrix optimization software does not include the functions of processing the reverse complementary sequence in the matrix (or unaligned sequence data), identifying the short matrix, and the processing method for gaps is not comprehensive enough, and the method for deleting short sequences is not simple and direct enough. The present invention has developed functions to specifically and effectively handle these problems, and the operation is simple and controllable, and the specific threshold can be freely set by the user according to the needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 It is a schematic diagram of various alignment problems existing in the existing multiple sequence alignment matrix of the present invention;

[0043] Figure 2 A schematic diagram of high-frequency errors in the defects of the existing multiple sequence alignment matrix filtering method of the present invention;

[0044] Figure 3 A schematic diagram of low-frequency errors in the defects of the existing multiple sequence alignment matrix filtering method of the present invention;

[0045] Figure 4 A schematic diagram of long fragment errors in the defects of the existing multiple sequence alignment matrix filtering method of the present invention;

[0046] Figure 5 A schematic diagram of short segment errors in the defects of the existing multiple sequence alignment matrix filtering method of the present invention;

[0047] Figure 6 This is a flowchart of the maskSegment function according to an embodiment of the present invention.

[0048] Figure 7 A simplified example of the maskSegment processing process in an embodiment of the present invention;

[0049] Figure 8 Schematic diagram of the revComplement function correcting the reverse complementary sequence in an embodiment of the present invention;

[0050] Fig. 9 Schematic diagram of the anyShortseq function in an embodiment of the present invention deleting sequences that do not meet the length requirement;

[0051] Fig.10 Schematic diagram of the degap function in an embodiment of the present invention deleting sites with gaps at both ends and in the middle that are lower than a set threshold;

[0052] Fig.11 Schematic diagram showing that the maskSegment function according to an embodiment of the present invention has a better filtering effect on high-frequency errors;

[0053] Fig.12 Schematic diagram showing that the maskSegment function according to an embodiment of the present invention has a better filtering effect on low-frequency errors;

[0054] Fig.13Schematic diagram showing that the maskSegment function of an embodiment of the present invention has a better filtering effect on long segment errors;

[0055] Fig.14 Schematic diagram showing that the maskSegment function of an embodiment of the present invention has a better filtering effect on short segment errors;

[0056] Fig.15 A schematic flow chart of a method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0058] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0059] like Figure 1-15 As shown, this embodiment provides a method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix, including: obtaining a nucleotide multiple sequence alignment matrix, screening the nucleotide multiple sequence alignment matrix to find five low-quality sequences, including sequences that require reverse complementation, matrices that meet a length threshold, short sequences that exist in the matrix, columns that are missing in the matrix and reach a preset threshold, and sequences with high variation and unreliable alignment; and filtering these five low-quality sequences.

[0060] Five functions written in R language were input into the software to filter low-quality sequences in the nucleotide multiple sequence alignment matrix; the five functions were revComplement function, alignmentLength function, anyShortseq function, degap function, and maskSegment function;

[0061] The revComplement function is used to correct the reverse complementary sequence in the matrix;

[0062] alignmentLength function, used to filter matrices that meet the length threshold;

[0063] anyShortseq function is used to delete short sequences in the matrix;

[0064] The degap function is used to delete columns in the matrix whose missing columns reach a preset threshold;

[0065] The maskSegment function is used to mask highly variable and unreliable segments in the matrix.

[0066] Furthermore, the revComplement function searches for similarity between a sequence in each matrix and all other sequences. According to the strand=Plus / Minus in the similarity search result file, it can be judged that the direction is inconsistent, and the matrix with reverse complementary sequences is found. Then, by judging the number of strand=Plus / Minus and the number of strand=Plus / Plus sequences, the group of sequences with fewer numbers will be reverse complemented in the matrix, and then the matrix will be re-output and placed in a new folder. The matrix without reverse complementary sequences can be output in another new folder or not.

[0067] Furthermore, the alignmentLength function directly determines the total length of the matrix. When the total length of the matrix is ​​greater than or equal to the set threshold (Len) length, the matrix will be output in a new folder. When the total length of the matrix is ​​less than the set threshold (Len) length, the matrix can be output in a new folder or not.

[0068] Furthermore, the anyShortseq function first extracts each sequence name and corresponding sequence in the matrix, and then determines the base (ATCG) length of the sequence and compares it with the set threshold (Len or LenP). When the base length of the sequence is less than the set threshold length, the sequence in the matrix will be deleted and the matrix will be output in a new folder. The matrix with a base length greater than or equal to the set threshold can be output in a new folder or not. Users can adjust the two parameters of Len and LenP according to their own data characteristics and needs. The two parameters are in an or relationship. If it is one of the cases, it will be deleted.

[0069] Furthermore, the degap function sets parameter threshold processing for the gap sites at both ends and in the middle, and the specific threshold can be freely set by the user according to the needs. Because according to a lot of experience, the sites at the beginning and end of the matrix (sequence) are usually more prone to sequencing and alignment errors (introducing wrong gaps). The "degap" function can choose to process the gaps at both ends first (set check_flank_gaps = T), determine the sites (columns) with a front end smaller than the threshold according to the threshold (p_flankgap) and delete them, and then determine the sites (columns) with a terminal smaller than the threshold and delete them. If the gaps at both ends are not considered separately (set check_flank_gaps = F), the "degap" function will perform the same processing on the gaps of the entire matrix, determine the sites with a threshold smaller than the threshold according to the set threshold (P) and delete them. The matrix containing the gap sites deleted will be output in a new folder, and the matrix without the gaps deleted can be output in a new folder or not.

[0070] The maskSegment function is the core function of this new method. Its basic principle is to identify and remove fuzzy or highly variable fragments, which have low similarity with the largest group of fragments identified as well-aligned after the grouping-recombination process. The calculation of random similarity probability uses the simplest model, that is, the DNA sequence is composed of four nucleotides randomly, without any GC bias, and does not use any base substitution model. The simplest model may not match the real data more or less, but it still performs well in most real cases and has the advantage of low computational load. Under the simplest assumption, the probability of random arrangement similarity can be formulated as Where n is the fragment length and k is the number of substitution or variant sites. In the default sliding window width of 15 bp (n = 15), the random permutation similarity (RS) is calculated at given probability values ​​(P = 0.05, 0.01, 0.001, 0.0001, i.e., the parameter prob in the maskSegment function and its four recommended values) as follows:

[0071] The pairwise similarity score (PS) is then calculated and compared with the corresponding RS at a specified probability (stringency) to evaluate whether they should be combined into the same group. The PS calculation formula is Where xi and yi represent each pair of fragments in the sliding window, and xi and yi are compared and counted only when they are both certain nucleotides (degenerate bases and gap sites are skipped). If the PS value is lower than RS under a given probability threshold condition, the fragments in the sliding window will initially be classified into different temporary groups, which means that they fall statistically significantly in the randomly arranged space. Subsequently, after all sequences have been classified in the first round of grouping, a regrouping process is performed between the temporary groups. If the PS value of any intermediate sequence in one temporary group is higher than the RS value corresponding to any sequence in the second temporary group, the two temporary groups are recombined. The characteristic of this grouping-regrouping process is that it not only avoids unnecessary pairwise comparisons, but also ensures that the optimal solution is obtained. After the grouping-regrouping process, the fragments except those in the largest group are finally blocked.

[0072] Furthermore, the maskSegment function uses a sliding window to check whether all sequences in the matrix have suspicious alignments in each sliding window. The advantage of the sliding window test is that it can identify alignment problems in only some segments of the sequences in the matrix, and will not be masked by the good overall alignment effect. For each sliding window with a width of k (setting the parameter window_width = k), by calculating the similarity scores of the two sequences (segments), and comparing them with the random similarity reference values ​​that can be achieved by two random sequences of the same length in the sliding window, it is determined whether the alignment between the two sequences is suspicious. In order to reduce the operation speed and find the optimal solution at the same time, a unique grouping-regrouping method is created, which is completely different from the methods of all known filtering software. If all sequences in the sliding window are compared in pairs, the amount of calculation will reach the square of the number of sequences N, and it is proportional to the length of the sequence L, that is, O(N2L). In order to improve the calculation rate without sacrificing accuracy (obtaining the optimal solution), the first sequence in the data is compared with all subsequent sequences for similarity and clustered. If the similarity between the first sequence and several subsequent sequences is higher than the random similarity, they will be clustered into one group. This avoids the pairwise comparison between the sequences after the first sequence in the same group, greatly saving the calculation time. With this method, in the case of more conservative sequences (i.e., when the similarity between the first sequence and the remaining sequences is higher than the random similarity threshold), all sequences are directly clustered into one group (no low-quality alignment, no need for shielding), and the amount of calculation in this case is only O(NL). If all the fragments in the sliding window are grouped by this clustering method and divided into more than two groups, it is still impossible to directly determine whether the groups are low-quality alignments or different sources, because the first round of clustering process does not compare the similarity between the sequences after the first sequence of each group, and the similarity between them may be higher than the random similarity, and can also be judged as a correct homologous alignment. If any two sequences between the two groups can be judged as correct homology alignments, then it is obvious that the two groups of sequences are also correct homology alignments with each other, and should be regrouped and merged. Obviously, this original method can not only achieve the (necessary) "exhaustive search" to find the optimal solution, but also greatly save the amount of calculation and improve the calculation speed. Existing matrix optimization software all use heuristic search or limit the maximum number of searches (for example, by setting the starting tree, comparing the similarity between the consensus sequences of the starting tree branches, etc.), which improves the running speed by reducing the number of alignments and sacrificing accuracy.

[0073] Discovery and statistics of various alignment problems in multiple sequence alignment matrices

[0074] Based on the homologous multiple sequence alignment matrix obtained from actual sequencing data, we checked the alignment problems in the matrix and found that there were mainly the following five problems (such as Figure 1 As shown in the figure): (1) the presence of reverse complementary sequences, (2) the length of the entire matrix is ​​very short, (3) the length of the sequence is too short, (4) the sequence is missing (gap), (5) the alignment is wrong or the segment is questionable.

[0075] Search for existing software to handle the above problems and compare the processing effects of existing software.

[0076] Search for software that handles matrix alignment problems and compare their methods, principles and flaws

[0077] Existing filtering software (Table 1) was extensively collected through websites such as Google, Baidu, github, and bioconductor, and their filtering methods (Table 1) and defects (Table 2) were compared.

[0078] Table 1

[0079]

[0080]

[0081] Table 2

[0082]

[0083] Comparison of the effectiveness of existing software in handling different types of alignment errors

[0084] The processing effects of existing mainstream filtering software HmmCleaner, Spruceup, TAPER, Guidance2, HoT, BMGE, Gblocks, trimAl, and FasParser2 were tested using sequence matrices of high-frequency alignment errors, low-frequency alignment errors, long-fragment alignment errors, and short-fragment alignment errors. Spruceup used three different filtering rigorities, trimAl set four different filtering rigorities and one automatic mode, and the other software used default parameter settings. The results showed that existing filtering software could not identify or was difficult to fully identify high-frequency alignment errors ( Figure 2 ), low frequency comparison errors ( Figure 3 ), long fragment alignment errors ( Figure 4 ) and short segment alignment errors ( Figure 5 In addition, existing matrix filtering software does not include the functions of processing reverse complementary sequences in matrices (or unaligned sequence data) and identifying short matrices, and the processing method for gaps is not comprehensive enough, and the method for deleting short sequences is not simple and direct enough.

[0085] Develop a program to comprehensively and effectively filter various alignment errors in multiple sequence alignment matrices.

[0086] In order to solve various alignment problems in multiple sequence alignment, a set of functions was developed using R language. At the full-length sequence level, the "revComplement" function identifies and adjusts the reverse complementary sequences contained in the matrix; the "alignmentLength" function sorts according to its length and screens the matrix that meets the length; and the "anyShortseq" function deletes short sequences in the alignment matrix. At the module (column) level, the "degap" function deletes the gap sites (gap) at both ends and embedded inside, respectively, with different levels of stringency. More importantly, at the fragment level, the "maskSegment" function masks fuzzy alignments or highly variable fragments, based on a newly developed grouping-regrouping algorithm, acting on the similarity / divergence of paired fragments in each sliding window.

[0087] Install the software that the program depends on

[0088] Install R (>=4.1.2) and RStudio. If you use a lower R version (maybe <3.5.3), some of these 5 functions may not work properly and give wrong results. Install and import in R: magrittr package, Biostring package, utils package.

[0089] Write functions to handle the above five comparison problems respectively.

[0090] Writing the implementation of the "revComplement" function

[0091] (1) Obtain BLAST result file: Extract a sequence from the matrix as a query, use the matrix as a database, perform similarity search using local BLAST software, and obtain a BLAST result file.

[0092] (2) Extracting species name and reverse complementary direction information: Use the grep function to extract the sequence name and the corresponding direction relationship between the sequence and the query sequence (stand = Plus / Plus or Plus / Minus) from the BLAST results and store them in corresponding variables.

[0093] (3) Process the sequence according to the reverse complementary direction information: perform correction operations based on the directional relationship between the sequence name and the query sequence. If a reverse complementary (Plus / Minus) sequence is detected, a folder containing the corrected sequence will be created, and corresponding operations will be performed according to different situations: If there are more "Plus / Plus" sequences than "Plus / Minus" sequences, then the reverse complementary sequence (Plus / Minus) will be corrected one by one in the original matrix, and then saved as a new FASTA format file and stored in a new folder. If there are more "Plus / Minus" sequences than "Plus / Plus" sequences, then the forward (Plus / Plus) sequence will be corrected one by one in the original matrix, and then saved as a new FASTA format file and stored in a new folder. As for the specific reverse complementary operation, it is processed by calling the reverseComplement function in the function. This function usually complements A with T and C with G according to the characteristics of the DNA sequence, thereby obtaining a reverse complementary sequence.

[0094] (4) Decide whether to keep the original data that does not need to be processed: If there is no matrix of reverse complementary sequences, decide whether to output the original matrix to a new folder according to the setting parameter none_revcomplement_data. If the parameter none_revcomplement_data is set to TRUE, a folder named "none_revcomplement_data" is created and the original matrix is ​​copied to the folder using the file.copy function.

[0095] Writing the implementation of the "alignmentLength" function

[0096] (1) Extracting sequence information: By reading in the aligned matrix (all sequences are of equal length), the total length of only one sequence is determined to be the total length of the entire matrix.

[0097] (2) Compare the total length of the matrix with the specified threshold (parameter Len) and output the result: If the matrix length is greater than or equal to the specified threshold, a folder named "long_alignments" is created and the matrix is ​​copied to the folder. If the matrix length is less than the specified threshold, if the parameter short_alignments is set to TRUE, a folder named "short_alignments" is created and the matrix is ​​copied to the folder.

[0098] Writing the implementation of the "anyShortseq" function

[0099] (1) Extract sequence information: Use the regular expression grep function to extract all sequence names and base sequences from the input alignment matrix.

[0100] (2) Calculate the length of each sequence: Traverse each sequence in the alignment matrix and calculate its length. In this process, characters that do not belong to the nucleic acid sequence (such as N, -) are excluded and the effective nucleic acid sequence length is calculated.

[0101] (3) Determine based on the set length threshold: By comparing the length of each sequence with the set length threshold (Len or LenP, the two are in an or relationship, and if it is one of the cases, it will be deleted. Len refers to the sequence whose base (ATGC) length in the screening and deletion matrix is ​​less than the set threshold, and LenP refers to the sequence whose base (ATGC) ratio in the screening and deletion matrix is ​​less than the threshold), determine whether each sequence meets the expected length requirement. If the sequence length is less than the threshold (Len or LenP), the sequence name and sequence length information are recorded.

[0102] (4) Output processing results: Output the information containing sequences that do not meet the length requirements to the short_seq_information folder, delete these sequences that do not meet the length requirements in the matrix, and save the processed matrix to the rm_shortseq_data folder. For matrices without sequences that need to be removed, if the parameter none_shortseq_data is set to TRUE, the matrix will also be copied to the none_shortseq_data folder.

[0103] Writing the implementation of the "degap" function

[0104] (1) Extracting alignment information: First, the function extracts the name and sequence of each DNA alignment sequence from the input DNA alignment object. The grep function is used here to identify the lines with the ">" mark as the sequence name, and then the substr and paste functions are used to extract the sequence itself and store them in the all_seq variable.

[0105] (2) Processing end gaps: By checking and deleting the two end regions (i.e., the parts containing continuous gaps), the gaps at both ends are removed according to the set length threshold. Here, the gregexpr function is used to identify the regions containing continuous gaps, and then processed according to the set parameters.

[0106] (3) Judge based on the set gap ratio: For each position in the entire alignment matrix, the gap ratio is calculated, and then based on the set length threshold, it is judged whether the gap ratio at each position meets the expected requirements.

[0107] (4) Deleting gap positions: If the gap ratio of certain positions meets the deletion criteria, these gap positions will be deleted. The specific deletion process uses the strsplit function to split the sequence into single bases, and then removes the corresponding gap positions based on the calculated positions to be deleted.

[0108] (5) Output processing results: Finally, the processed alignment data is saved in the degaps folder. If the none_gappy_data parameter is set to TRUE, the alignment data without gap positions that need to be removed will also be copied to the none_gappy_data folder.

[0109] Write the implementation process of the "maskSegment" function and show the processing example

[0110] (1) Calculate the correlation between the random similarity probability and the similarity threshold (RS): First, set the window size to n (window_width), initialize the probability P to 0, and k (indicating the number of substitution or variation sites) to 0. Then, through the while loop, use the probability mass function of the binomial distribution to calculate the probability A in each iteration and add it to P. After each round of iteration, determine whether the cumulative probability P exceeds or equals the given probability (prob). When the cumulative probability P exceeds or equals the given probability for the first time, calculate the random similarity RS value based on the current k and n, and jump out of the loop.

[0111] (2) Extract sequence information from the aligned matrix: Use the grep function to extract the sequence name from the input data. Here, the regular expression "^>" is used to match the line where the sequence name is located. Use the paste function to connect multiple lines of text for each sequence to form a complete aligned sequence.

[0112] (3) Divide the window: Use the strsplit function to divide the sequence into windows according to the specified window size (window_width) and step size (window_step). In this step, conditional judgment is used to distinguish different types (DNA or codon) for window division.

[0113] (4) Calculate the pairwise similarity score (PS) and compare it with the RS value, then group and merge: Use a for loop (or repeat loop) to traverse and process the sequence fragments in each window. Use the substring function to extract subsequences from the sequence, that is, the fragments in the window. Calculate the similarity score (PS) between the sequences. Then compare the PS value with the RS value. If it is greater than or equal to the RS value, these sequences will be classified into the same group and marked as fragments that need to be transferred to a new group. In this way, in each loop, the fragments that meet the conditions are classified into a specific group based on the comparison result of the calculated similarity score (PS) and the RS value.

[0114] (5) Shielding unreliable fragments: For each fragment in a group, the similarity score (PS) is compared with the preset similarity threshold (RS) to determine whether it needs to be marked as a fragment to be shielded. If the similarity score of a fragment does not meet the requirements, that is, the score is too low, it may be marked as a fragment to be shielded. The nucleotides contained in the segment to be shielded are replaced with "N".

[0115] (6) Output processing results: Use the dir.create function to create the mask_alignment, mask_information, and unfiltered_data folders to store related information and unprocessed data. Use the file.copy function to copy the unprocessed data to the unfiltered_data folder.

[0116] In order to better understand the principle and processing of the "maskSegment" function, a flowchart of the "maskSegment" function is written (such as Figure 6 As shown), four sets of simplified examples of the "maskSegment" function processing process are also made, such as Figure 7 shown.

[0117] The filtering effect of the set of functions of the present invention is tested.

[0118] Verify the processing effect of the "revComplement" function:

[0119] Inputting the matrix and blast result file as data to the "revComplement" function will correct the sequence containing reverse complement, such as Figure 8 shown.

[0120] Check the effect of the "alignmentLength" function:

[0121] Input the matrix as data to the "alignmentLength" function. As long as the Len parameter is set, the matrix greater than or equal to the threshold will be stored in a new folder long_alignments.

[0122] Check the processing effect of the "anyShortseq" function:

[0123] Input the matrix as data to the "anyShortseq" function and set the Len parameter, such as Fig. 9 As shown, when len=100 is set, base sequences shorter than 100 bp in the matrix are removed.

[0124] Check the effect of the "degap" function:

[0125] Input the matrix as data to the "degap" function. You can set the p parameter to uniformly process the entire matrix sequence. In addition, because the gaps at both ends are more prone to sequencing errors, you can also choose to use different stringency processing for the sequences at both ends. Then set check_flank_gaps = T, and then set the p_flankgap parameter, such as Fig.10 As shown, p=0.5 is set for the middle sequence, and a higher stringency is adopted for both ends, check_flank_gaps=T, p_flankgap=0.1 is set, and the corresponding gap columns (represented as - in the matrix) that meet the conditions are removed.

[0126] Verify the processing effect of the core function "maskSegment":

[0127] The same data (sequence matrix of high-frequency alignment errors, low-frequency alignment errors, long-fragment alignment errors, and short-fragment alignment errors) were used to test the filtering effect of the "maskSegment" function. Parameter settings: The prob that controls the filtering stringency was set to 0.05, 0.01, 0.001, and 0.0001, respectively, the sliding window (window_width) size was 15bp, and the step size (window_step) was 3bp. The results showed that ( Figure 11-Figure 14 ), the filtering effect of the "maskSegment" function is better in cases of high-frequency and low-frequency mismatches, long fragments and short fragments mismatches, etc.

[0128] The present invention is based on various alignment problems existing in the multiple sequence alignment matrix found in the research. By comparing the advantages and disadvantages of the existing filtering methods, and using actual data to test the filtering effect of the existing software, it is found that the processing effect is not good or the threshold that can be set is limited, and there is still a lack of program software for correcting reverse complementary sequences, deleting short sequences and short matrices. Therefore, the present invention has developed a set of processing functions to respectively process various alignment problems, wherein the core function "maskSegment" is to filter the wrong alignment or doubtful fragments in the matrix by a sliding window method according to the degree of sequence variation, using a grouping-recombining method, which is different from the existing software method. Through the same data, different alignment error forms (high-frequency and low-frequency error alignment, long fragment and short fragment error alignment), the processing effects of the new method and the existing method on the wrong alignment or doubtful fragments are compared, and the results show that: in the cases of high-frequency and low-frequency error alignment, long fragment and short fragment error alignment, etc., the filtering effect of the maskSegment function is better. In addition, the existing matrix optimization software does not include the functions of processing the reverse complementary sequence in the matrix (or unaligned sequence data), identifying the short matrix, and the processing method for gaps is not comprehensive enough, and the method for deleting short sequences is not simple and direct enough. The present invention has developed functions to specifically and effectively handle these problems, and the operation is simple and controllable, and the specific threshold can be freely set by the user according to the needs.

[0129] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix, characterized in that: include: A nucleotide multiple sequence alignment matrix is ​​obtained, and five low-quality sequences are screened out from the nucleotide multiple sequence alignment matrix, including sequences that require reverse complementary sequences, matrices that meet the length threshold, short sequences that exist in the matrix, missing columns in the matrix, and sequences with high variation and unreliable alignment; these five low-quality sequences are filtered.

2. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 1, characterized in that: The five functions written in R language are revComplement function, alignmentLength function, anyShortseq function, degap function, and maskSegment function; The revComplement function is used to correct the reverse complementary sequence in the matrix; The alignmentLength function is used to screen matrices that meet the length threshold; The anyShortseq function is used to delete short sequences existing in the matrix; The degap function is used to delete the columns in the matrix whose missing columns reach a preset threshold; The maskSegment function is used to mask highly variable and unreliable segments in the matrix.

3. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 2, characterized in that: The method for correcting the reverse complementary sequence in the matrix by using the revComplement function comprises: performing a similarity search on a sequence in each matrix and all other sequences to obtain a similarity search result file; According to the strand=Plus / Minus judgment moment in the similarity search result file, it is judged that the direction is inconsistent, and the matrix with reverse complementary sequence is found; By judging the number of sequences with strand = Plus / Minus and the number of sequences with strand = Plus / Plus in the matrix, the group of sequences with a smaller number will be reverse complemented in the matrix, and the matrix will be re-output and placed in a new folder. The matrix without reverse complementary sequences will be selected and output in another new folder.

4. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 2, characterized in that: The method of using the alignmentLength function to screen matrices that meet the length threshold includes: directly judging the total length of the matrix, when the total length of the matrix is ​​greater than or equal to the set threshold length, outputting the matrix in a new folder; and selecting to output the matrix whose total length is less than the set threshold length in a new folder.

5. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 2, characterized in that: Methods for deleting short sequences in a matrix using the anyShortseq function include: Extract each sequence name and corresponding base sequence in the matrix, determine the base length of the sequence and compare it with the set threshold; When the sequence base length is less than the set threshold length, the sequence in the matrix is ​​deleted and the matrix is ​​output in a new folder. The matrix with a base length greater than or equal to the set threshold is selected and output in a new folder; The user sets a threshold value according to the characteristics and needs of his own data. The threshold value is set in two ways: base length or the ratio of base length to the total length of the sequence. These two setting methods are in an or relationship. If one of the situations occurs, it will be deleted.

6. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 2, characterized in that: The method of deleting the missing section in the matrix by using the degap function includes: The parameter threshold processing is set for the missing sites at both ends and in the middle of the matrix. Process the missing sites at both ends, determine the sites with a front end smaller than the threshold and delete them according to the threshold, and determine the sites with a end smaller than the threshold and delete them; Process the missing sites of the entire matrix, determine the sites that are smaller than the threshold according to the set threshold and delete them; The matrix of deleted deletion sites is output in a new folder, and the matrix of undeleted deletion sites is output in another new folder.

7. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 2, characterized in that: Methods for using the maskSegment function to mask highly variable and unreliable segments in a matrix include: Calculate the correlation between the random similarity probability and the random permutation similarity threshold; Extract each sequence name and base sequence in the aligned matrix to form a complete aligned sequence; Dividing the alignment sequence into windows according to a set window size and step size; Calculate the pairwise similarity score and compare it with the random permutation similarity threshold, group and merge, and classify the qualified fragments into specific groups; The segments in each group are compared with a preset random arrangement similarity threshold to determine whether they need to be marked as segments that need to be blocked.

8. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 7, characterized in that: The method for calculating the random similarity probability P is: Where n represents the fragment length; k represents the number of substitution or variation sites.

9. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 7, characterized in that: The method for calculating the random permutation similarity RS is: Where n represents the fragment length; k represents the number of substitution or variation sites.

10. The method for filtering low-quality sequences in a nucleotide multiple sequence alignment matrix according to claim 7, characterized in that: The method for calculating the pairwise similarity score PS is: Where n is the fragment length; k is the number of substitution or variation sites; x is i and i Each pair of segments in the sliding window represents two different sequences.

Citation Information

Patent Citations

  • Automatic method for analyzing constitution and diversity of bacterial community of 16SrRNA gene

    CN106815492A

  • Methods, devices, and media for detecting single nucleotide variations and insertions / deletions

    CN114649055A

  • Method and system for predicting capacity of small open reading window coding polypeptide in non-coding RNA (Ribonucleic Acid)

    CN118038995A