Master-slave collaborative de novo assembly method and system based on sequence feature matrix

Abstract

This invention discloses a master-slave collaborative second-generation de novo assembly method and system based on sequence feature matrices. The method includes the following steps: S1: Gene analysis of sequencing files to obtain custom IDs and base sequences; S2: Management of base sequence data via a floppy disk array (RAID); S3: Calculation of the front and back feature matrices for each sequencing read and generation of an ID docking table; S4: Assembly tasks performed in a multi-machine parallel manner; S5: Outputting the assembly results of the second-generation sequencing data after patching and gap filling. This invention significantly improves the speed of bioinformatics analysis workflows for genome assembly analysis, helping bioinformatics analysts obtain accurate analysis results faster and in less time.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a method and system for second- and fourth-generation genome fusion assembly based on numerical feature expression. Background Technology

[0002] Since Roche launched its first next-generation sequencer, the Roche 454, in 2005, life sciences officially entered the era of high-throughput sequencing. The introduction of the Illumina sequencing platform significantly reduced the price of next-generation sequencing, enabling its widespread adoption across various research fields in life sciences. To this day, next-generation short-read sequencing technology still holds an absolute dominant position in the global sequencing market.

[0003] While second-generation sequencing technology has greatly improved sequencing throughput, the length of the single sequence it obtains is very short, often only 50 to 300 bp, which is extremely small relative to the entire genome. Furthermore, because the sequencing coverage is deeper, the number of sequences that need to be processed in a genome project increases rapidly.

[0004] Currently, the main methods for assembling second-generation sequencing data include Overlap-Layout-Consensus (OLC) and De Bruijn Graph (DBG) algorithms. OLC is suitable for assembling long reads, but its data structures require significant memory, resulting in slow speed and a high error rate. DBG, on the other hand, is faster and more accurate than OLC, but it requires large amounts of memory for computation because it splits each read into K-mer strings of a specified length and stores them all in memory for sequence assembly. Furthermore, pathfinding after constructing the De Bruijn Graph also consumes considerable computational resources. Most current genome assembly algorithms are improvements and optimizations based on DBG. SPAdes is suitable for assembling small genomes, while SOAPdenovo can assemble large genomes and is fast, but also has a high error rate. The extremely high memory consumption and slow single-machine assembly speed are the biggest limitations of various mainstream DBG assembly algorithms. Summary of the Invention

[0005] The purpose of this invention is to provide a second- and fourth-generation genome fusion assembly method and system based on numerical feature expression, so as to solve the technical problems of poor accuracy and low efficiency of existing genome assembly methods.

[0006] The objective of this invention is achieved through the following technical solution: a master-slave collaborative second-generation de novo assembly method based on sequence feature matrices, comprising the following steps:

[0007] S1: Perform gene analysis on the sequencing file to obtain custom numbers and base sequences;

[0008] S2: Management of base sequence data is achieved through floppy disk array RAID;

[0009] S3: Calculate the front and back feature matrices of each sequencing read and generate an ID docking table;

[0010] S4: Assembly tasks are performed using a multi-machine parallel approach;

[0011] S5: Output the assembly results of the second-generation sequencing data after replying and filling in the gaps.

[0012] Furthermore, the sequencing file includes a fastq file. First, the fastq file needs to be analyzed to obtain a custom number and base sequence.

[0013] Furthermore, step S2 specifically involves the edge host loading the base sequence data into virtual memory and issuing a feature matrix calculation task to the slave device.

[0014] Furthermore, step S3 includes the following sub-steps:

[0015] S31: Generate two one-dimensional numerical feature matrices for each sequencing read to describe the feature values ​​of a sequencing read.

[0016] S32: After obtaining the feature matrix corresponding to all sequencing reads, generate the ID docking table using DNN, compressed sensing, or numerical fast sorting methods.

[0017] Furthermore, the numerical quick sorting method is as follows: first, the preceding and following feature matrices are quickly sorted according to their values; then, according to a set error threshold, the preceding feature matrix is ​​searched for a following feature matrix that meets the requirement, or the following feature matrix is ​​searched for a preceding feature matrix that meets the requirement; and an ID matching table is generated according to the number of the matched preceding and following feature matrices.

[0018] Furthermore, step S4 includes the following sub-steps:

[0019] S41: Preliminary assembly, assembling the small contig and SNP information table in a highly asynchronous manner;

[0020] S42: Graph theory assembly, constructing the de Brené graph assembly contig and SNP information table.

[0021] Furthermore, step S41 specifically involves: after obtaining the ID matching table, submitting the assembly calculation task in a highly asynchronous manner, assembling the small contig and SNP information table, the edge host receiving the data and distributing the task to each server slave, each server slave claiming the matching calculation task in the current ID matching table and returning the assembly result to the edge host.

[0022] Further, step S42 specifically involves: the edge host constructing a de Blain graph based on the obtained overlap relationship, splitting the overall graph into subgraphs containing information, distributing the subgraphs to each server slave, the server slave performing pathfinding based on the subgraph depth, and reading the small contig results assembled in the previous step stored in the edge host, assembling them into longer contig segments and SNP information tables, and after the calculation is completed, sending the assembly results back to the edge host for integration.

[0023] Furthermore, step S5 specifically involves dividing the assembly results obtained from graph theory into two files, long and short sequences, based on whether they exceed the longest second-generation sequencing length. First, sliding matching is performed one by one between the long sequences to find the overlapping relationships between the long fragments and then assemble them into longer fragments. Then, the remaining short sequences are regarded as operation sequences and matched one by one with the long sequences to find the anchoring position of the short sequences and obtain the final assembly result.

[0024] A master-slave collaborative second-generation de novo assembly system based on sequence feature matrices includes a gene analysis module, a management module, a computing module, a multi-machine assembly module, and an output module. The gene analysis module performs gene analysis on sequencing files to obtain custom numbers and base sequences. The management module manages the base sequence data using a floppy disk array (RAID). The computing module calculates the front and back feature matrices of each sequencing read and generates an ID docking table. The multi-machine assembly module performs assembly tasks in a multi-machine parallel manner. The output module outputs the assembly results of the second-generation sequencing data after patching and filling gaps.

[0025] The beneficial effects of this invention are as follows: This invention achieves a linear performance improvement in the assembly algorithm. Compared to previous statistical algorithms, this method does not split reads into multiple K-mer segments and construct a De BruijnGraph directed graph through sequence overlap. Instead, it uses unsupervised machine learning and other methods to quantify the sequencing bases, transforming the overlapping order of reads into a numerical quicksort order. This not only improves the speed of read arrangement but also, to some extent, avoids the generation of gap positions between K-mers during assembly, reducing the number of repeated read alignments and significantly improving computational speed while maintaining accuracy. During the assembly process, SNP variant information is simultaneously detected using a centralized server cluster architecture with master-slave collaborative computation. In summary, this method is a second-generation assembly scheme that integrates high accuracy, low computational requirements, and high efficiency, significantly improving the speed of bioinformatics analysis in genome assembly and helping bioinformatics analysts obtain accurate analysis results faster and in less time.

[0026] Leveraging the rapid development of gene sequencing technology and bioinformatics, this method provides strong support for low-cost, rapid, and efficient genome K-mer analysis, truly realizing the storage, retrieval, and analysis of biological information using computer science and technology. Furthermore, bioinformatics is one of the core fields of 21st-century natural science, and this method, as a key technology in DNA-seq analysis within bioinformatics, possesses significant technological advantages and market value in genomics, making it suitable for practical applications and further technological advancements in gene analysis.

[0027] The core breakthroughs of this invention include at least the following: 1. Completely solving the problem of excessive memory usage in bioinformatics assembly. 2. Achieving complete traversal assembly, greatly reducing the risk of assembly errors. 3. Utilizing server cluster computing to achieve a linear performance improvement in assembly results. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0029] Figure 1 This is a flowchart of the present invention;

[0030] Figure 2 Schematic diagram of front-end feature matrix extraction;

[0031] Figure 3 Schematic diagram of backend feature matrix extraction principle;

[0032] Figure 4 This is a schematic diagram of the front-end and back-end feature matrix mapping;

[0033] Figure 5 This is a system block diagram of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0035] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0036] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0037] Example 1:

[0038] See Figure 1 The master-slave collaborative second-generation de novo assembly method based on sequence feature matrices includes the following steps:

[0039] S1: Perform gene analysis on the sequencing file to obtain custom numbers and base sequences;

[0040] S2: Management of base sequence data is achieved through floppy disk array RAID;

[0041] S3: Calculate the front and back feature matrices of each sequencing read and generate an ID docking table;

[0042] S4: Assembly tasks are performed using a multi-machine parallel approach;

[0043] S5: Output the assembly results of the second-generation sequencing data after replying and filling in the gaps.

[0044] In this embodiment, the sequencing file includes a fastq file. After obtaining the sequencing reads, the fastq file first needs to be analyzed to obtain a new number and extract the corresponding base sequence.

[0045] In this embodiment, step S2 specifically involves the edge host loading the base sequence data into virtual memory and issuing a feature matrix calculation task to the slave device.

[0046] In this embodiment, step S3 includes the following sub-steps:

[0047] S31: Generate two one-dimensional numerical feature matrices for each sequencing read to describe the feature values ​​of a sequencing read.

[0048] S32: After obtaining the feature matrix corresponding to all sequencing reads, generate the ID docking table using DNN, compressed sensing, or numerical fast sorting methods.

[0049] Furthermore, the numerical quick sorting method is as follows: first, the preceding and following feature matrices are quickly sorted according to their values; then, according to a set error threshold, the preceding feature matrix is ​​searched for a following feature matrix that meets the requirement, or the following feature matrix is ​​searched for a preceding feature matrix that meets the requirement; and an ID matching table is generated according to the number of the matched preceding and following feature matrices.

[0050] Using base position as the x-axis, the four bases A, T, C, and G are assigned 16384, 32767, -16384, and -32767 on the y-axis, respectively (this mapping can be expanded accordingly to ensure the centroid value fills the int16 value space). Two one-dimensional numerical feature value (FP32) matrices are generated for each sequencing read to describe its feature values. Specifically, when generating the front and back feature matrices, n bp are taken sequentially from the 5' end to the 3' end (the back feature matrix is ​​the opposite). These n read strings are then subjected to unsupervised machine learning K-means clustering, and the resulting cluster centroid value y-axis (int16) is used as the feature value of that K-mer substring. The principle of front feature matrix extraction is as follows... Figure 2 As shown in Figure 3, the principle of post-feature matrix extraction is as follows. For example, if the base sequence of a read is: ATTCTGAAATGACAACATGATGAAATTCT, then its correspondence on the coordinate axis is as follows. Figure 4 As shown. The ordinate of the cluster centroid of this read segment is calculated to be 10922.66667. Its int16 format, i.e., 10923, is taken as the feature value of this segment.

[0051] It is important to note that when generating the feature matrix, the base sequences corresponding to subsequent feature values ​​need to include the preceding base sequences. The generated front-end feature matrix is ​​[Kmeans1 Kmeans2 Kmeans3 Kmeans4 Kmeans5 Kmeans6……], and the generated back-end feature matrix is ​​[Kmeans6' Kmeans5' Kmeans4' Kmeans3' Kmeans2' Kmeans1'……]. Different sequencers have different sequencing lengths, therefore, different maximum sequencing lengths are set for different sequencers. If the sequence read length is less than the maximum length, the feature values ​​excluding the sequence read length are all 0. For example, if a sequence read length is 154bp, and feature values ​​are obtained by taking 30bp increments forward, then starting from the 7th feature value, all subsequent feature values ​​are 0. As shown in Table 1:

[0052] Table 1. Feature values ​​of sequencing reads with a length of 154 bp.

[0053]

[0054]

[0055] After obtaining the feature matrices corresponding to all sequencing reads, if there is an overlap between two sequences, their feature matrices will reflect similar or identical feature values. Specifically, when generating the ID matching table, the preceding and following feature matrices are first quick-sorted according to their values ​​to obtain the approximate order of all feature values. For different overlapping correspondences, the preceding feature matrices of the sequencing reads are used to calculate possible following feature matrices within a certain threshold range, and sequence IDs that meet the threshold range are searched in the quick-sorted following feature matrices, thus generating the preceding and following correspondences between sequences. Alternatively, the following feature matrices of the sequencing reads can be used to calculate possible preceding feature matrices within a certain range, and sequence IDs that meet the threshold range are searched in the quick-sorted preceding feature matrices, thus generating the preceding and following correspondences between sequences.

[0056] For example, the post-feature matrix of read 1 is [Kmeans1 Kmeans2 Kmeans3 Kmeans4 Kmeans5 Kmeans6], and the pre-feature matrix of read 2 is [Kmeans7 Kmeans8 Kmeans9 Kmeans10 Kmeans11 Kmeans12]. The possible overlap between the two (assuming eigenvalues ​​are calculated in 30bp increments) is as follows: If there is 30 or fewer bases overlapping, Kmeans6 and Kmeans7 will have high similarity or be completely identical; Kmeans5 and Kmeans8 will have low similarity. As the base sequence corresponding to the eigenvalue extends forward and backward, the similarity of the eigenvalues ​​will gradually decrease until they are completely dissimilar. If there are 60 or fewer overlapping bases, Kmeans5 and Kmeans8 will have high similarity or be completely identical. Kmeans6 and Kmeans7 will have lower similarity, and Kmeans4 and Kmeans9 will have even lower similarity. As the base sequence corresponding to the eigenvalue extends forward and backward, the similarity of the eigenvalues ​​will decrease step by step until they are completely dissimilar. Other corresponding cases will follow the same pattern.

[0057] In practical applications, thresholds for different matching scenarios are determined based on the changes in feature values ​​at each bit interval. Within this range, the reads corresponding to the subsequent feature matrices that meet the conditions are searched to generate an ID matching table. Besides using numerical quicksort to generate the ID matching table, a dictionary-learning compressed sensing method can also be used. This method uses the preceding and following feature matrices of reads with overlapping matching relationships as a training set, predicts the subsequent feature matrices using the preceding feature matrices, and then quickly sorts the subsequent feature matrices before searching for similar feature matrices to generate the ID matching table.

[0058] In this embodiment, the multi-machine parallel approach described in step S4 can be broadly divided into three steps, the purpose of which is to obtain the de novo assembly results and SNP information of the second-generation sequencing data: First, the edge host issues the corresponding assembly tasks from the ID docking table to the slaves, and the slaves assemble small contigs and output the SNP mutation information from the first step; Second, the edge integrates the thread overlap information of the small contigs and constructs a de Brené graph accordingly, distributing the split subgraphs to the slaves for pathfinding assembly, and obtaining the second SNP mutation information; Third, according to the longest sequencing length of the second-generation sequencing, the graph-theorized assembly results are divided into two files, long and short sequences, and the long or short sequences are used as operation sequences for further extension to obtain the final assembly result. Specifically, this includes the following sub-steps:

[0059] S41: Preliminary assembly, assembling the small contig and SNP information table in a highly asynchronous manner;

[0060] S42: Graph theory assembly, constructing the de Brené graph assembly contig and SNP information table.

[0061] Furthermore, in step S41, a highly asynchronous approach is adopted. After obtaining a relatively accurate preliminary ID matching table, the user submits the assembly computation task. The edge server cluster host receives the data and distributes the task to each server slave. Each server slave claims the matching computation task in the current ID matching table and returns the assembly result to the host. Specifically, after the preliminary ID matching table is generated, the PC submits the assembly computation task, the edge host receives the data, establishes a B+ tree index of ID and specific base sequence, and treats the PC as one of the slaves in the server cluster, issuing computation tasks to the edge server cluster slaves.

[0062] When computation begins, the available thread resources in the server cluster are first counted and assigned a unique identifier for this task. When the first thread performs computation and matches read 1 with other docking base sequences, a 1-1 tag is added before the first row (1-3 9 18 66 105) of the ID docking table stored on the host as the thread identifier, indicating the first computation of the first thread. Sequences are then extracted according to their read IDs using a B+ tree index for one-by-one matching. If reads 3 and 66 have more than 80% base similarity to read 1, with only a small number of SNP variations, then reads 1, 3, and 66 are considered to have overlapping regions and can be merged into one. Reads 9, 18, and 105 with read 1 have less than 80% similarity in their overlapping regions and are discarded. When sequence number 3 matches sequence number 1, the search continues in the ID matching table for sequences 3 and 66. If a third thread has already started calculating the thread identifier count (3-1) before the ID matching table, the calculation stops, and the calculated thread identifier count {1-1: 3-1, 66} is output. This small contig sequence is identified as contig1-1. During assembly, the ID matching layer continues to calculate the ID matching table after expanding the error threshold and supplements it to the preliminary ID matching table stored in the edge server cluster host.

[0063] After the reads of sequencing fragment 1 are calculated, K-mer check bits are generated for contig1-1. That is, a 5-bit K-mer is generated every 50 bp for contig1-1, and the result is converted into a hash table for indexing. For example, tatgc-atttc-ttagc-tttgc: 1-55, 2-88, 4-99, 8-107 means that the four contigs 1-55, 2-88, 4-99, and 8-107 all contain the above four 5-bit K-mer sequences.

[0064] After the ID matching table for read number 1 is assembled, when checking the next batch of ID matching tables, if a thread has already started calculating for that read, when reading the reads in the next ID matching table, first generate the K-mer checksum for that read using the same method. Then, search the K-mer index table of the contig for an already assembled contig containing the same K-mer. If found, add the corresponding contig tag to the beginning of the ID matching table for that read, and merge and assemble the read with the corresponding contig again. After completion, update the information in the K-mer checksum index table of the contig. If the read matches the end of the contig but exceeds the length of the contig, or if no checksum matching the read is found in the K-mer index table, then create a new thread to start assembling the read.

[0065] Once the ID matching table for read #1 is assembled, when checking the next batch of ID matching tables, if no reads have been started for calculation by other threads at the same time, it is determined which of these reads has the longest assembly and merging result, and that ID matching table row is selected to continue the next assembly step.

[0066] Further, in step S42, after obtaining the preliminary assembly results and thread identification overlap information, the edge host constructs a de Brené graph based on the obtained overlap relationship, and splits the overall graph into subgraphs containing information. The subgraphs are distributed to each slave device, which uses the subgraph depth for pathfinding and reads the small contig results from the previous assembly stored in the host, assembling them into longer contig segments. After the computation is complete, the assembly results are sent back to the host for integration. Throughout this process, the host can also undertake computational tasks.

[0067] In this embodiment, step S5 specifically involves the following: After graph theory computation, the resulting contigs contain base sequences of varying lengths, including overlap information that may be ignored by the ID docking table or new overlap relationships generated after sequence extension. In this step, the assembly results obtained from graph theory are divided into two files—long and short sequences—based on whether they exceed the longest second-generation sequencing length. First, sliding matching is performed between the long sequences one by one to find overlap relationships between long fragments and assemble them into longer fragments. Then, the remaining short sequences are treated as operation sequences and matched one by one with the long sequences to find the anchoring positions of the short sequences, resulting in the final assembly result.

[0068] In addition to completing de novo gene sequence assembly, this invention can also detect gene SNPs (Single Nucleotide Polymorphisms). During multi-threaded contig construction, by extracting and matching sequencing reads one by one, SNP site information can be output. When assembling small contigs in multi-threaded assembly, the SNP site information is output for the first time, including the small contig's identifier, full length, SNP variant base site, the base in the output contig at that variant site, and the base replaced by the variant. During graph theory assembly and posting, the SNP variant information needs to be adjusted accordingly when extracting small contigs to assemble contigs.

[0069] This invention enables de novo assembly of second-generation sequencing data from a server cluster. To improve computational speed and fully realize single-task multi-threaded implementation, this invention adopts a cloud-edge collaborative large-scale computer system architecture. The system comprises one user PC, one edge host, n edge slaves, and a cloud server. In the entire computing system, the PC submits tasks and communicates with the edge host. After the PC submits an assembly task to the edge host, the host treats the PC as one of the edge cluster slaves and distributes feature matrix calculation tasks to the slaves. When the slaves complete their calculations, the host receives and integrates the results. If deep learning neural networks, compressed sensing, or other inference and prediction methods are chosen to generate the ID matching table, the edge host transmits feature values ​​to the cloud data server. The cloud server then issues inference or prediction tasks to the cloud inference server in a dual-machine hot standby configuration. If numerical quicksort is chosen to generate the ID matching table, the edge host completes this process, loading the reads data into virtual memory and generating a B+ tree index of IDs and base sequences. After obtaining the ID matching table, the edge host issues specific assembly tasks to the slaves and receives the assembly results.

[0070] See Figure 5Based on the same inventive concept, this invention also provides a master-slave collaborative second-generation de novo assembly system based on sequence feature matrices to implement the aforementioned master-slave collaborative second-generation de novo assembly method based on sequence feature matrices. This system includes a gene analysis module, a management module, a computing module, a multi-machine assembly module, and an output module. The gene analysis module is used to perform gene analysis on sequencing files to obtain custom numbers and base sequences. The management module manages the base sequence data through a floppy disk array (RAID). The computing module calculates the front and back feature matrices of each sequencing read and generates an ID docking table. The multi-machine assembly module performs the assembly task in a multi-machine parallel manner. The output module outputs the assembly results of the second-generation sequencing data after patching and filling gaps.

[0071] This invention has at least the following technical effects:

[0072] This invention achieves a linear performance improvement in the assembly algorithm. Compared to previous statistical algorithms, this method does not split reads into multiple K-mer segments and construct a De Bruijn Graph through sequence overlap. Instead, it uses unsupervised machine learning and other methods to quantify the sequencing bases, transforming the overlapping order of reads into a numerical quicksort order. This not only improves the speed of read arrangement but also avoids the generation of gap positions between K-mers that may occur during assembly, reducing the number of repeated read alignments. This significantly improves computational speed while maintaining accuracy. During assembly, SNP variant information is simultaneously detected using a centralized server cluster architecture with master-slave collaborative computation. In summary, this method is a second-generation assembly scheme that integrates high accuracy, low computational requirements, and high efficiency, significantly improving the speed of bioinformatics analysis in genome assembly and helping bioinformatics analysts obtain accurate analysis results faster and in less time.

[0073] Leveraging the rapid development of gene sequencing technology and bioinformatics, this method provides strong support for low-cost, rapid, and efficient genome K-mer analysis, truly realizing the storage, retrieval, and analysis of biological information using computer science and technology. Furthermore, bioinformatics is one of the core fields of 21st-century natural science, and this method, as a key technology in DNA-seq analysis within bioinformatics, possesses significant technological advantages and market value in genomics, making it suitable for practical applications and further technological advancements in gene analysis.

[0074] The core breakthroughs of this invention include at least the following: 1. Completely solving the problem of excessive memory usage in bioinformatics assembly. 2. Achieving complete traversal assembly, greatly reducing the risk of assembly errors. 3. Utilizing server cluster computing to achieve a linear performance improvement in assembly results.

[0075] It should be noted that, for the sake of simplicity, the foregoing embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0076] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A master-slave collaborative second-generation de novo assembly method based on sequence feature matrices, characterized in that, Includes the following steps: S1: Perform gene analysis on the sequencing file to obtain custom numbers and base sequences; S2: Management of base sequence data is achieved through floppy disk array RAID; S3: Calculate the front and back feature matrices for each sequencing read and generate an ID docking table; step S3 includes the following sub-steps: S31: Generate two one-dimensional numerical feature matrices for each sequencing read to describe the feature values ​​of a sequencing read. S32: After obtaining the feature matrix corresponding to all sequencing reads, use DNN, compressed sensing or numerical fast sorting methods to generate an ID docking table; The numerical quick sorting method is as follows: First, the preceding and following feature matrices are quickly sorted according to their values. Based on the set error threshold, the preceding feature matrix is ​​searched for a following feature matrix that meets the requirements, or the following feature matrix is ​​searched for a preceding feature matrix that meets the requirements. Based on the numbers of the matched preceding and following feature matrices, an ID matching table is generated. S4: The assembly task is performed using a multi-machine parallel approach; step S4 includes the following sub-steps: S41: Preliminary assembly, assembling the small contig and SNP information table in a highly asynchronous manner; S42: Graph theory assembly, constructing de Blaine graph assembly of contigs and SNP information tables; S5: Output the assembly results of the second-generation sequencing data after replying and patching the holes; Step S5 is as follows: According to whether it exceeds the longest second-generation sequencing length, the assembly results obtained by graph theory are divided into two files, long and short sequences. First, the long sequences are slid-matched one by one to find the overlapping relationship between the long fragments and then assemble them into longer fragments. Then, the remaining short sequences are regarded as operation sequences and matched with the long sequences one by one to find the anchor position of the short sequences and obtain the final assembly results.

2. The master-slave collaborative second-generation de novo assembly method based on sequence feature matrices as described in claim 1, characterized in that, The sequencing files include fastq files. First, the fastq files need to be analyzed to obtain custom numbers and base sequences.

3. The master-slave collaborative second-generation de novo assembly method based on sequence feature matrices as described in claim 1, characterized in that, Step S2 specifically involves the edge host loading the base sequence data into virtual memory and issuing a feature matrix calculation task to the slave.

4. The master-slave collaborative second-generation de novo assembly method based on sequence feature matrices as described in claim 1, characterized in that, Step S41 is as follows: After obtaining the ID matching table, the assembly calculation task is submitted in a highly asynchronous manner to assemble the small contig and SNP information table. The edge host receives the data and distributes the task to each server slave. Each server slave claims the matching calculation task in the current ID matching table and returns the assembly result to the edge host.

5. The master-slave collaborative second-generation de novo assembly method based on sequence feature matrices as described in claim 1, characterized in that, Step S42 is as follows: The edge host constructs a de Blain graph based on the obtained overlap relationship, and splits the overall graph into subgraphs containing information. The subgraphs are distributed to each server slave. The server slave uses the depth of the subgraph to find the path, and reads the small contig result of the previous step of assembly stored in the edge host. It assembles the small contig into a longer contig and SNP information table. After the calculation is completed, the assembly result is sent back to the edge host for integration.

6. A master-slave collaborative second-generation de novo assembly system based on sequence feature matrices, used to implement the master-slave collaborative second-generation de novo assembly method based on sequence feature matrices as described in any one of claims 1 to 5, characterized in that, The system includes a gene analysis module, a management module, a computing module, a multi-machine assembly module, and an output module. The gene analysis module performs gene analysis on sequencing files to obtain custom numbers and base sequences. The management module manages the base sequence data using a floppy disk array (RAID). The computing module calculates the front and back feature matrices of each sequencing read and generates an ID docking table. The multi-machine assembly module performs assembly tasks in a multi-machine parallel manner. The output module outputs the assembly results of the second-generation sequencing data after patching and filling gaps.

Images