Non-parameter deduplication method and device for UMI sequencing data of cfDNA, electronic equipment and computer readable storage medium

Abstract

The invention discloses a parameter-free duplicate removal method and device for UMI sequencing data of cfDNA, electronic equipment and a computer readable storage medium, and relates to the technical field of biomedical research and clinical diagnosis. Through the parameter-free deduplication method provided by the invention, the overall analysis process is optimized, multiple comparison and data format conversion are avoided, the overall analysis efficiency is improved, and the deduplication analysis speed is greatly improved. Therefore, key biomarkers contained in the cfDNA can be mined, and more reliable data support is provided for early diagnosis, treatment monitoring and prognosis evaluation of diseases.

Description

Technical Field

[0001] The present invention relates to the field of biomedical research and clinical diagnostic technology, and more specifically, to a parameter-free deduplication method, device, electronic device, and computer-readable storage medium for UMI sequencing data of cfDNA. Background Art

[0002] In biomedical research and clinical diagnostics, UMI (Unique Molecular Identifier) ​​sequencing data for cfDNA (cfDNA) has demonstrated unparalleled importance. cfDNA (cell-free DNA) is widely present in various bodily fluids, including blood and cerebrospinal fluid. Although present in extremely low quantities, it contains a wealth of critical information about the body's health and disease. UMI sequencing technology, by attaching a unique identifier to each DNA molecule, can accurately identify and distinguish sequencing results from different raw molecules. This is a significant advantage in analyzing cfDNA. On the one hand, it effectively overcomes the problem of sequencing errors caused by the scarcity of cfDNA and its susceptibility to external interference, accurately identifying true genetic mutations from massive amounts of data, rather than misidentifying sequencing noise as key variants. On the other hand, UMI sequencing can accurately count cfDNA for quantitative analysis, providing extremely reliable data support for both detecting gene copy number variations and monitoring the dynamic fluctuations of cfDNA levels during disease progression.

[0003] Current mainstream bioinformatics workflows are extremely complex for deduplication analysis of cfDNA UMI sequencing data. First, FASTQ files are converted to BAM files, UMIs are extracted, and then converted back to FASTQ format for trimming. Next, BWAMEM is used for alignment, merging the alignment results, grouping by UMI, calling for double-strand consensus reads, and further alignment, sorting, merging BAM files, adding read group information, and finally running the calibration script. The entire process encompasses 14 key steps, including data conversion, processing, alignment, and merging.

[0004] In cutting-edge fields such as precision medicine and early cancer screening, in-depth research on circulating free DNA (cfDNA) is becoming increasingly critical, and accurate and rapid deduplication analysis strategies for cfDNA UMI (Unique Molecular Identifier) ​​sequencing data are a key component. Mainstream analysis methods for processing cfDNA UMI sequencing data typically require multiple alignments and data format conversions, making the overall analysis time-consuming.

[0005] In view of this, the present invention is proposed. Summary of the Invention

[0006] The present invention aims to provide a method, device, electronic device, and computer-readable storage medium for parameter-free deduplication of UMI sequencing data of cfDNA to solve the above-mentioned technical problems.

[0007] The present invention is achieved in that:

[0008] In a first aspect, the present invention provides a method for deduplication of UMI sequencing data of cfDNA without parameters, comprising the following steps:

[0009] (1) Preliminary grouping of sequencing sequences based on a preset dictionary tree;

[0010] (2) Regrouping based on sequence alignment and clustering algorithm;

[0011] (3) Perform consensus sequence analysis based on algorithms;

[0012] (4) Obtain UMI standard FASTQ file;

[0013] The preliminary grouping in step (1) includes the following steps: for each sequencing sequence in the fastq file, according to the established UMI design rule with a fixed length of n, extract the UMI sequence and obtain the insert fragment; each sequencing sequence refers to a cfDNA sequence containing a variable length UMI;

[0014] Then, a precise search is performed on all UMI sequences in the loaded dictionary tree. Through the dictionary tree traversal mechanism, each character of each UMI sequence is compared to locate the matching UMI node. When a matching UMI node is found, the insert information and insert quality value corresponding to the UMI sequence, as well as the UMI sequence and UMI sequence quality value, are added to the group data structure associated with the matching UMI node.

[0015] The regrouping in step (2) includes: intra-group sequence alignment and splitting of the preliminary group; wherein the intra-group sequence alignment includes: intra-group sequence alignment for a number of inserted fragments within each group, and obtaining information on the number of differential bases;

[0016] The splitting process includes the following steps:

[0017] Using the number of base differences as a key indicator, a clustering algorithm is used to evaluate and adjust each group. The clustering algorithm calculates the similarity or distance between sequences and, based on a pre-set threshold for the number of base differences, splits the sequences within the group so that the differences between sequences in each subgroup are less than 3 bases.

[0018] The specific rules for evaluation and adjustment are as follows: if the number of base differences between any two sequences in a group is less than 3, the group is maintained; conversely, if the number of base differences between any two sequences in the group is at least 3, the group is split into multiple subgroups;

[0019] The consensus sequence analysis in step (3) includes the following steps: first, setting the prior probabilities for the four DNA bases A, T, C and G to 0.3, 0.3, 0.2 and 0.2 respectively; second, calculating the likelihood probability based on the sequence quality value information and variation coverage; then, combining the prior probability with the likelihood probability to calculate the posterior probability of the base at each position in the DNA sequence within the group; selecting the base with the largest posterior probability at each position and combining them to generate a consensus sequence.

[0020] In a second aspect, the present invention also provides a parameter-free deduplication device for UMI sequencing data of cfDNA, which includes: a dictionary tree construction module, a preliminary grouping module, a secondary grouping module and a consistency sequence analysis module;

[0021] The dictionary tree construction module is used to: construct a dictionary tree for variable-length UMI sequences;

[0022] The preliminary grouping module is used to: perform preliminary grouping of sequencing sequences based on a preset dictionary tree;

[0023] The regrouping module is used to: regroup based on sequence alignment and clustering algorithm;

[0024] The consensus sequence analysis module is used to perform consensus sequence analysis based on algorithms.

[0025] In a third aspect, the present invention also provides an electronic device comprising: a processor and a memory; the processor and the memory are connected, wherein the memory is used to store a computer program, and the processor is used to call the computer program to execute the above-mentioned parameter-free deduplication method for cfDNA UMI sequencing data.

[0026] In a fourth aspect, the present invention also provides a computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned parameter-free deduplication method for UMI sequencing data of cfDNA.

[0027] The present invention has the following beneficial effects:

[0028] The present invention is based on a parameter-free deduplication method that does not require comparison with a reference genome. First, a dictionary tree of UMI sequences is constructed, followed by preliminary grouping based on the dictionary tree, and then sequence grouping based on sequence similarity, and then a consensus sequence is constructed for the sequences within the group. The parameter-free deduplication method provided by the present invention optimizes the overall analysis process, avoids multiple comparisons and data format conversions, improves overall analysis efficiency, and significantly increases the speed of deduplication analysis. This helps to explore key biomarkers contained in cfDNA and provide more reliable data support for early diagnosis, treatment monitoring, and prognosis assessment of diseases.

[0029] The present invention combines the UMI dictionary tree and alignment algorithm to achieve accurate grouping of cfDNA UMI sequence repeats, providing a basis for consistent sequence analysis.

[0030] The present invention uses an algorithm to construct a consensus sequence for repeated sequences within a group based on sequence base information and quality values, thereby improving data integrity and saving time for overall analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 Flowchart of the parameter-free deduplication method for cfDNA UMI sequencing data. DETAILED DESCRIPTION

[0033] Reference will now be made in detail to embodiments of the present invention, one or more examples of which are described below. Each example is provided to illustrate, not to limit, the present invention. Indeed, it will be apparent to those skilled in the art that various modifications and variations may be made to the present invention without departing from the scope or spirit of the invention. For example, features illustrated or described as part of one embodiment may be used in another embodiment to produce further embodiments.

[0034] In a first aspect, the present invention provides a method for deduplication of UMI sequencing data of cfDNA without parameters, comprising the following steps:

[0035] (1) Preliminary grouping of sequencing sequences based on a preset dictionary tree;

[0036] (2) Regrouping based on sequence alignment and clustering algorithm;

[0037] (3) Perform consensus sequence analysis based on algorithms;

[0038] (4) Obtain UMI standard FASTQ file;

[0039] The preliminary grouping in step (1) includes the following steps: for each sequencing sequence in the fastq file, according to the established UMI design rule with a fixed length of n, extract the UMI sequence and obtain the insert fragment; each sequencing sequence refers to a cfDNA sequence containing a variable length UMI;

[0040] Then, a precise search is performed on all UMI sequences in the loaded dictionary tree. Through the dictionary tree traversal mechanism, each character of each UMI sequence is compared to locate the matching UMI node. When a matching UMI node is found, the insert information and insert quality value corresponding to the UMI sequence, as well as the UMI sequence and UMI sequence quality value, are added to the group data structure associated with the matching UMI node.

[0041] The regrouping in step (2) includes: intra-group sequence alignment and splitting of the preliminary group; wherein the intra-group sequence alignment includes: intra-group sequence alignment for a number of inserted fragments within each group, and obtaining information on the number of differential bases;

[0042] The splitting process includes the following steps:

[0043] Using the number of base differences as a key indicator, a clustering algorithm is used to evaluate and adjust each group. The clustering algorithm calculates the similarity or distance between sequences and, based on a pre-set threshold for the number of base differences, splits the sequences within the group so that the differences between sequences in each subgroup are less than 3 bases.

[0044] The specific rules for evaluation and adjustment are as follows: if the number of base differences between any two sequences in a group is less than 3, the group is maintained; conversely, if the number of base differences between any two sequences in the group is at least 3, the group is split into multiple subgroups;

[0045] The consensus sequence analysis in step (3) includes the following steps: first, setting the prior probabilities for the four DNA bases A, T, C and G to 0.3, 0.3, 0.2 and 0.2 respectively; second, calculating the likelihood probability based on the sequence quality value information and variation coverage; then, combining the prior probability with the likelihood probability to calculate the posterior probability of the base at each position in the DNA sequence within the group; selecting the base with the largest posterior probability at each position and combining them to generate a consensus sequence.

[0046] The formula for calculating the posterior probability is as follows:

[0047]

[0048] Where Q represents the sequence quality information, V represents the variation coverage, B∈{A,T,C,G}, P(B) represents the prior probability of a certain base appearing, P(Q,V|B) is the likelihood probability, which represents the probability of observing a specific sequence quality information Q and variation coverage V when the known base is a certain base, P(Q,V|B)*P(B) represents the joint probability of a certain base (A,T,C,G) appearing at this position, ∑ B′∈{A，T，C，G}P(Q, V|B′)P(B′) is the sum of the joint probabilities of all possible bases.

[0049] This method efficiently groups each sequencing sequence based on its UMI sequence. After initial grouping using a constructed dictionary tree, it uses sequence alignment and clustering algorithms to further group sequences so that sequences within a group or subgroup meet a preset threshold for base difference. This algorithm-based consensus sequence analysis allows for rapid identification of consensus sequences.

[0050] The parameter-free deduplication method provided by this invention optimizes the overall analysis process, avoids multiple comparisons and data format conversions, improves overall analysis efficiency, and significantly increases the speed of deduplication analysis. This helps to discover key biomarkers contained in cfDNA, providing more reliable data support for early disease diagnosis, treatment monitoring, and prognosis assessment.

[0051] The present invention combines the UMI dictionary tree and alignment algorithm to achieve accurate grouping of cfDNA UMI sequence repeats, providing a basis for consistent sequence analysis.

[0052] The present invention uses an algorithm to construct a consensus sequence for repeated sequences within a group based on sequence base information and quality values, thereby improving data integrity and saving time for overall analysis.

[0053] In a preferred embodiment of the present invention, step (1) includes the construction of a dictionary tree, which includes the following steps:

[0054] Trie preprocessing, trie construction for variable-length UMI sequences, and trie output. The core concept of the trie is to use the common prefixes of strings to reduce storage space and improve query efficiency. It organizes multiple strings into a tree based on their prefix relationships. Each node in the tree represents a character, and the characters along the path from the root node to a specific node are concatenated to form a string.

[0055] In a preferred embodiment of the present invention, the dictionary tree preprocessing includes:

[0056] The operation is performed on each determined UMI length n in turn. First, given that there are 4 bases at each position in the DNA sequence, all possible UMI sequences of that length are generated, totaling 4n different combinations of UMI sequences.

[0057] In a preferred embodiment of the present invention, a method for constructing a dictionary tree for a variable-length UMI sequence includes the following steps:

[0058] Initialize an empty dictionary tree structure, where each node in the tree represents a base; insert all previously generated UMI sequences into the dictionary tree in sequence according to the dictionary tree construction rules.

[0059] In a preferred embodiment of the present invention, the method for sequentially inserting all previously generated UMI sequences into the dictionary tree according to the dictionary tree construction rules is as follows:

[0060] The insertion process starts from the root node and proceeds according to the order of bases in the UMI sequence. If the child node corresponding to the current base does not exist, a new node is created. If the child node already exists, the next base is inserted downward along this node until the entire UMI sequence is completely inserted into the dictionary tree.

[0061] In a preferred embodiment of the present invention, a specific tag is added to the leaf node corresponding to each UMI sequence.

[0062] Finally, the constructed dictionary tree is saved as a binary file to a local storage medium. During the saving process, it is necessary to ensure that the stored information can accurately restore the dictionary tree structure and the relationship between nodes, so as to provide guarantee for subsequent direct loading and use.

[0063] The sequence alignment in step (2) is performed by, for example, using the classic Needleman-Wunsch algorithm to perform pairwise alignment.

[0064] In a preferred embodiment of the present invention, the clustering algorithm is selected from at least one of a mean shift clustering algorithm and a hierarchical clustering algorithm;

[0065] Algorithms for consensus sequence analysis are Bayesian, neighbor-joining, or maximum parsimony.

[0066] The Bayesian algorithm is preferred. The present invention has grouped the high-throughput sequencing data first, and the data in a group is relatively small. Therefore, using a Bayesian-based algorithm is more accurate, especially when the amount of repetitive sequence data in a group is small or there is uncertainty.

[0067] The fixed length n is 6-12, for example, the fixed length is 6, 7, 8, 9, 10, 11 or 12.

[0068] In a second aspect, the present invention also provides a parameter-free deduplication device for UMI sequencing data of cfDNA, which includes: a dictionary tree construction module, a preliminary grouping module, a secondary grouping module and a consistency sequence analysis module;

[0069] The dictionary tree construction module is used to: construct a dictionary tree for variable-length UMI sequences;

[0070] The preliminary grouping module is used to: perform preliminary grouping of sequencing sequences based on a preset dictionary tree;

[0071] The regrouping module is used to: regroup based on sequence alignment and clustering algorithm;

[0072] The consensus sequence analysis module is used to perform consensus sequence analysis based on algorithms.

[0073] In a preferred embodiment of the present invention, the parameter-free deduplication device further includes a FASTQ file output module;

[0074] In a preferred embodiment of the present invention, preliminary grouping includes: for each sequencing sequence in the fastq file, extracting the UMI sequence and obtaining the insert fragment according to the established UMI design rules with a fixed length of n; each sequencing sequence refers to a cfDNA sequence containing variable-length UMIs; then, performing an accurate search for all UMI sequences in the loaded dictionary tree in sequence, comparing each character of each UMI sequence through the dictionary tree traversal mechanism, and locating the matching UMI node; when a matching UMI node is found, adding the insert fragment information and insert fragment quality value, UMI sequence, and UMI sequence quality value corresponding to the UMI sequence to the grouping data structure associated with the matching UMI node;

[0075] The regrouping includes: intra-group sequence alignment and splitting of the preliminary group; wherein the intra-group sequence alignment includes: intra-group sequence alignment for several inserted fragments within each group, and obtaining the number of differential bases;

[0076] The splitting process includes the following steps:

[0077] Using the number of differential bases as the key indicator, each group is evaluated and adjusted through a clustering algorithm. The clustering algorithm calculates the similarity or distance between sequences and splits the sequences within the group based on a pre-set threshold for the number of differential bases, so that the difference between the sequences in each sub-group is less than 3 bases.

[0078] In the above device, two adjacent modules are communicatively connected.

[0079] In a third aspect, the present invention also provides an electronic device comprising: a processor and a memory; the processor and the memory are connected, wherein the memory is used to store a computer program, and the processor is used to call the computer program to execute the above-mentioned parameter-free deduplication method for cfDNA UMI sequencing data.

[0080] Specifically, the electronic device may include a memory, a processor, a bus, and a communication interface, wherein the memory, processor, and communication interface are electrically connected to each other directly or indirectly to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more buses or signal lines. The processor may process information and / or data related to target identification to perform one or more functions described in this application.

[0081] The memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0082] A processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU) or a network processor (NP). It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0083] In a fourth aspect, the present invention also provides a computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned parameter-free deduplication method for UMI sequencing data of cfDNA.

[0084] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely below. Where specific conditions are not specified in the embodiments, conventional conditions or conditions recommended by the manufacturer are used. Where the manufacturer of the reagents or instruments is not specified, they are all conventional products that can be purchased commercially.

[0085] The features and performance of the present invention are further described in detail below with reference to the embodiments.

[0086] Example 1

[0087] This embodiment provides a method for deduplication of cfDNA UMI sequencing data without parameters. Figure 1 As shown, the specific steps include:

[0088] 1. Dictionary tree construction

[0089] This invention processes cfDNA sequence data containing variable-length UMIs. To efficiently group millions of sequences by UMI, a trie is generated for preprocessing. The UMI length n typically ranges from 6 to 12 bp, and a separate trie is generated for each specific value of n. The core concept of the trie is to use the common prefixes of strings to reduce storage space and improve query efficiency. It organizes multiple strings into a tree based on their prefix relationships. Each node in the tree represents a character, and the characters along the path from the root node to a specific node are concatenated to form a string. Tries are commonly used in bioinformatics scenarios such as sequence pattern matching, sequence storage and retrieval, and primer design.

[0090] During the tree preprocessing phase, operations are performed sequentially for each determined UMI length n. First, given that there are four bases at each position in a DNA sequence, all possible UMI sequences of that length are generated, totaling 4n different combinations. This step provides a complete sequence foundation for the subsequent construction of an accurate tree.

[0091] Then, the dictionary tree is constructed. An empty dictionary tree structure is initialized, and each node in the tree represents a base (A, T, C, G). All previously generated UMI sequences are inserted into the dictionary tree in sequence according to the dictionary tree construction rules. That is, the insertion process starts from the root node and strictly follows the order of the bases in the UMI sequence; if the child node corresponding to the current base does not exist, a new node is created; if the child node already exists, the next base is inserted downward along this node until the entire UMI sequence is completely inserted into the dictionary tree. To facilitate subsequent rapid search and positioning, specific tags can be added to the leaf nodes corresponding to each UMI sequence.

[0092] Finally, the constructed trie is saved as a binary file to local storage. During the saving process, it is important to ensure that the stored information accurately restores the trie structure and the relationships between nodes, ensuring that it can be directly loaded and used later. The seven sets of tries generated in this way can provide efficient and accurate support for subsequent UMI grouping and related analysis of large-scale sequence data.

[0093] 2. Sequence Grouping

[0094] (1) Preliminary grouping

[0095] For each sequencing sequence in the fastq file, the UMI sequence is accurately extracted according to the established UMI design rules of fixed length n, and the insert fragment is simultaneously obtained. During the sequence extraction process, the corresponding quality value information must also be separated to obtain independent and accurate quality value information for the UMI sequence and insert.

[0096] In the present invention, a previously saved trie structure corresponding to a specific length n is read from a local storage medium. This trie, as an efficient data retrieval tool, has been pre-built and stored. Its construction process may be based on specific algorithms and rules designed to optimize the efficiency of querying UMI sequences. Subsequently, a precise search operation is performed sequentially in the loaded trie for all UMI sequences decomposed through the previous data processing steps. During the search process, the trie's unique node traversal mechanism compares each character of the UMI sequence one by one to locate the matching UMI node. Once a matching node is successfully found, the insert information corresponding to the UMI, including its associated sequence and its quality value, is immediately added to the grouping data structure associated with the node.

[0097] (2) Group again

[0098] To further perform intra-group sequence alignment and splitting on the sequencing data that has been grouped by UMI, we first perform a sequence alignment operation, using the classic Needleman-Wunsch algorithm to perform pairwise alignments on the sequences to be tested within each group.

[0099] After sequence alignment is complete and the number of differential bases is obtained, the group splitting phase begins. In this phase, the number of differential bases is used as a key indicator to evaluate and adjust the grouping. If the number of differential bases between any two test sequences in a group is less than three, based on the cfDNA sequencing repetition rate and sequencing error rate, it can be inferred that these sequences are likely derived from the same original molecule. Therefore, the group remains in its current state and no additional processing is required. Conversely, if the number of differential bases between any two test sequences in a group reaches three or more, the group must be split into multiple subgroups to ensure data homogeneity and accuracy of the analysis results.

[0100] The specific splitting process is achieved with the help of a hierarchical clustering algorithm. The hierarchical clustering algorithm calculates the similarity or distance between sequences and splits the sequences according to a pre-set threshold of the number of base differences, ultimately reducing the difference between the test sequences in each subgroup to less than 3 bases.

[0101] 3. Consensus Sequence Analysis

[0102] If a group contains only one sequence, it is considered a consensus sequence. If not, the following algorithm is used to generate a consensus sequence.

[0103] First, based on sequencing data characteristics and experience, prior probabilities were set for the four DNA bases A, T, C, and G at 0.3, 0.3, 0.2, and 0.2, respectively. This setting comprehensively considers the distribution of genomic bases and sequencing errors. Second, likelihood probabilities were calculated based on the quality value information and variant coverage of the sequenced sequences. The quality value reflects the accuracy of the sequenced bases. Using a specific mathematical model and algorithm, the probability of observing a base given the true base is calculated, reflecting the likelihood of the base occurring under different sequencing quality conditions. This allows for a precise calculation of the probability of actually observing a specific base given the true base state. Then, applying Bayes' theorem, the prior probabilities are combined with the likelihood probabilities to calculate the posterior probability of the base at each position in the DNA sequence within the group. This integration of prior knowledge and new sequencing evidence provides a reliable basis for base determination. Finally, the consensus bases are determined. At each position, the bases with the highest posterior probability are selected and combined to generate a consensus sequence, reducing the interference of sequencing errors and improving sequence accuracy.

[0104] The formula for calculating the posterior probability is as follows:

[0105]

[0106] Where Q represents the sequence quality information, V represents the variation coverage, B∈{A,T,C,G}, P(B) represents the prior probability of a certain base appearing, P(Q,V|B) is the likelihood probability, which represents the probability of observing a specific sequence quality information Q and variation coverage V when the known base is a certain base, P(Q,V|B)*P(B) represents the joint probability of a certain base (A,T,C,G) appearing at this position, ∑ B′∈{A，T，C，G} P(Q, V|B′)P(B′) is the sum of the joint probabilities of all possible bases.

[0107] 4. Output UMI standard FASTQ file

[0108] After a series of processing, the consensus sequences for all groups are output to form a standard FASTQ file for subsequent analysis and storage. The second row contains the consensus sequence, and the fourth row contains the highest quality value of the corresponding base. In other words, the type of information contained in the output file is the same as the information before input.

[0109] Experimental Example 1

[0110] Using the SRR15081493, SRR15081482, and SRR15081494 data of the SRA database, the accuracy of the final variant identification was used to measure the difference between the results of the non-parameter deduplication method provided by the present invention and the mainstream method, which was used as an indicator of accuracy.

[0111] Solution 1: This is the implementation scheme provided in Example 1 of the present invention, which subsequently uses bwamem for alignment, samtools for sorting, and the GATKMutect2 process for variant identification.

[0112] Solution 2: Refer to the umi_analyze.sh of the mainstream solution (https: / / github.com / marcus-hoy-hansen / Vimalathas2025-cfDNApanel / ) to analyze the umi data, and finally use the GATKMutect2 pipeline for variant identification.

[0113] result:

[0114] Accuracy: Scheme 1 (Example 1 of the present invention) and Scheme 2 both identified 452, 279, and 204 variants in the SRR15081493, SRR15081482, and SRR15081494 data, respectively, with no difference.

[0115] Speed: Solution 1 (Example 1 of the present invention) takes 63 minutes, and Solution 2 takes 117 minutes.

[0116] Comprehensive comparison shows that the parameter-free deduplication method for UMI sequencing data provided by the present invention has significantly better deduplication efficiency than the conventional UMI data deduplication method and has high accuracy.

[0117] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A parameter-free deduplication method for UMI sequencing data of cfDNA, characterized in that: It includes the following steps: (1) Preliminary grouping of sequencing sequences based on a preset dictionary tree; (2) Regrouping based on sequence alignment and clustering algorithm; (3) Perform consensus sequence analysis based on algorithms; (4) Obtain UMI standard FASTQ file; The preliminary grouping in step (1) includes the following steps: for each sequencing sequence in the fastq file, extracting the UMI sequence according to the established UMI design rule with a fixed length of n, and obtaining the inserted fragment; each sequencing sequence refers to a cfDNA sequence containing a variable-length UMI; Then, a precise search is performed on all UMI sequences in the loaded dictionary tree. Through the dictionary tree traversal mechanism, each character of each UMI sequence is compared to locate the matching UMI node. When a matching UMI node is found, the insert information and insert quality value corresponding to the UMI sequence, as well as the UMI sequence and UMI sequence quality value, are added to the group data structure associated with the matching UMI node. The regrouping in step (2) includes: intra-group sequence alignment and splitting of the preliminary grouping; wherein the intra-group sequence alignment includes: intra-group sequence alignment for a number of inserted fragments within each grouping, and obtaining information on the number of differential bases; The splitting comprises the following steps: Using the number of base differences as a key indicator, each group is evaluated and adjusted using a clustering algorithm. The clustering algorithm calculates the similarity or distance between sequences and, based on a pre-set threshold for the number of base differences, splits the sequences within the group so that the differences between the sequences in each subgroup are less than 3 bases. The specific rules for the evaluation and adjustment are as follows: if the number of base differences between any two sequences in a group is less than 3, the group is maintained; conversely, if the number of base differences between any two sequences in the group is at least 3, the group is split into multiple subgroups; The consensus sequence analysis in step (3) includes the following steps: first, setting prior probabilities for the four DNA bases A, T, C, and G to 0.3, 0.3, 0.2, and 0.2, respectively; second, calculating the likelihood probability based on the sequence quality value information and variation coverage; then, combining the prior probability with the likelihood probability to calculate the posterior probability of the base at each position in the DNA sequence within the group; selecting the base with the largest posterior probability at each position and combining them to generate a consensus sequence.

2. The method for deduplication of UMI sequencing data of cfDNA according to claim 1, characterized in that: The step (1) also includes the construction of a dictionary tree, and the construction of the dictionary tree includes the following steps: Trie preprocessing, trie construction for variable-length UMI sequences, and trie output.

3. The method for deduplication of cfDNA UMI sequencing data without parameters according to claim 2, characterized in that: The dictionary tree preprocessing includes: The operation is performed on each determined UMI length n in turn. First, given that there are 4 bases at each position in the DNA sequence, all possible UMI sequences of that length are generated, totaling 4n different combinations of UMI sequences.

4. The method for deduplication of cfDNA UMI sequencing data without parameters according to claim 3, characterized in that: The method for constructing a dictionary tree for a variable-length UMI sequence includes the following steps: Initialize an empty dictionary tree structure, where each node in the tree represents a base; insert all previously generated UMI sequences into the dictionary tree in sequence according to the dictionary tree construction rules.

5. The method for deduplication of cfDNA UMI sequencing data without parameters according to claim 4, characterized in that: The method of inserting all previously generated UMI sequences into the dictionary tree in sequence according to the dictionary tree construction rules is as follows: The insertion process starts from the root node and proceeds according to the order of bases in the UMI sequence. If the child node corresponding to the current base does not exist, a new node is created. If the child node already exists, the next base is inserted downward along this node until the entire UMI sequence is completely inserted into the dictionary tree.

6. The method for deduplication of cfDNA UMI sequencing data without parameters according to claim 5, characterized in that: Add a specific tag to the leaf node corresponding to each UMI sequence.

7. The method for deduplication of UMI sequencing data of cfDNA according to claim 1, characterized in that: The clustering algorithm is selected from at least one of a mean shift clustering algorithm and a hierarchical clustering algorithm; The algorithms used for consensus sequence analysis are Bayesian, neighbor-joining, or maximum parsimony; The fixed length n is 6-12.

8. A parameter-free deduplication device for UMI sequencing data of cfDNA, characterized in that: It includes: dictionary Tree construction module, preliminary grouping module, regrouping module and consensus sequence analysis module; The dictionary tree construction module is used to: construct a dictionary tree for a variable-length UMI sequence; The preliminary grouping module is used to: perform preliminary grouping of sequencing sequences based on a preset dictionary tree; The regrouping module is used for: regrouping based on sequence alignment and clustering algorithm; The consensus sequence analysis module is used to: perform consensus sequence analysis based on an algorithm; Preferably, the parameter-free deduplication device further includes a FASTQ file output module; Preferably, the preliminary grouping includes: for each sequencing sequence in the fastq file, extracting a UMI sequence and obtaining an insert according to a predetermined UMI design rule with a fixed length of n; each sequencing sequence refers to a cfDNA sequence containing a variable-length UMI; then, performing an accurate search for all UMI sequences in the loaded dictionary tree in sequence, comparing each character of each UMI sequence through a dictionary tree traversal mechanism, and locating a matching UMI node; when a matching UMI node is found, adding the insert information and insert quality value, UMI sequence, and UMI sequence quality value corresponding to the UMI sequence to a grouping data structure associated with the matching UMI node; The regrouping includes: aligning and splitting the sequences within the preliminary group; wherein the intra-group sequence alignment includes: aligning the sequences within a number of inserted fragments within each group, and obtaining information on the number of differential bases; The splitting comprises the following steps: Using the number of differential bases as the key indicator, each group is evaluated and adjusted using a clustering algorithm; the clustering algorithm calculates the similarity or distance between sequences and, based on a pre-set threshold for the number of differential bases, splits the sequences within the group so that the difference between the sequences in each subgroup is less than 3 bases.

9. An electronic device, characterized in that: It includes: processor and memory; The processor is connected to the memory, wherein: The memory is used to store a computer program, and the processor is used to call the computer program to execute the parameter-free deduplication method for cfDNA UMI sequencing data according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The method includes instructions that, when executed on a computer, cause the computer to perform the parameter-free deduplication method for UMI sequencing data of cfDNA according to any one of claims 1 to 7.

Images