A high-performance gene alignment method and system for autonomous and controllable heterogeneous many-core clusters

Abstract

The present invention relates to a high-performance gene alignment method and system for autonomous and controllable heterogeneous many-core clusters. The present invention proposes a three-stage gene alignment process of Seeding-Scoring-and-Extension. By introducing an independent Scoring stage and using an improved matching scoring algorithm, it is possible to timely and efficiently filter low-quality initial matching positions at runtime, reduce the amount of redundant calculations in the subsequent alignment process and ensure its sufficient parallelism. The present invention alleviates the contradiction between memory consumption and alignment performance by introducing distributed computing technology in gene alignment, and proposes a matrix-based NW matching scoring algorithm and a tensor-based SW algorithm, the core calculations of which can be efficiently implemented using a variety of fine-grained parallel means of the platform. The present invention comprehensively utilizes the distributed parallel capability, main-coprocessor asynchronous parallel capability, processor internal task parallel capability and fine-grained data parallel capability of the platform, giving full play to the computing performance of the platform while having high platform portability.

Description

Technical Field

[0001] The present invention belongs to the field of computer application technology, and belongs to the field of combining high-performance computing technology with bioinformatics. Specifically, it relates to a high-performance gene alignment method and system for autonomous and controllable heterogeneous many-core clusters. Background Art

[0002] Gene alignment aligns reads (gene fragments) with the species' RefSeq (reference genome) to obtain two pieces of information: 1) the top-k best matching positions of any read on the RefSeq—these positions identify the RefSeq subsequences that are most similar to the given read within the RefSeq, with the highest similarity score; and 2) the details of the match between the read and the RefSeq subsequence at the best matching position (including base matches, insertions, deletions, and substitutions). This information is typically presented as a CIGAR (Compact Idiosyncratic Gapped Alignment Report) string (referred to as CIGAR information). Gene alignment can be considered one of the most fundamental operations in bioinformatics analysis. Based on the alignment results, a variety of further bioinformatics analysis processes, such as deduplication, biological homology analysis, gene annotation, and variant calling, can be carried out. Advances in gene sequencing technology have made it possible to sequence a large number of species, generating a massive amount of reads that need to be analyzed and applied. However, as a key component of the genetic analysis process, currently used genetic alignment software, due to its early development, cannot fully utilize the powerful computing power provided by modern computer architectures. Its low throughput (the number of reads aligned per unit time) restricts the implementation of large-scale bioinformatics analysis. For example, using the well-known genetic alignment software BWA on a mainstream Intel server, it takes approximately five hours to complete a whole-genome alignment of a human individual. Therefore, it is necessary to implement higher-performance genetic alignment software on today's advanced computing platforms to support the rapid development of the bioinformatics analysis field.

[0003] In recent years, with the development of high-performance computing technology in my country, a variety of powerful heterogeneous computer architectures have emerged. Typical examples include the Sugon CPU + DCU architecture, the Shenwei SW26010 many-core architecture, and the Huawei Kunpeng + Ascend architecture. Generally speaking, a heterogeneous computer node consists of a host (a multi-core CPU) and a device (an accelerator) with different architectures. The host is typically a conventional multi-core CPU, while the device can be equipped with different coprocessors, such as the Sugon DCU or Huawei Ascend coprocessors. The host and device can share a uniformly addressed memory space or have independently addressed memory spaces, interconnected by high-speed data buses (such as PCI-e). Multiple heterogeneous nodes are interconnected via a network to form a heterogeneous cluster. Such clusters are a common platform in my country's high-performance computing field and are widely used in fields such as large-scale artificial intelligence, numerical simulation, and scientific and engineering computing. While heterogeneous clusters theoretically offer enormous computing power, their high programming complexity also poses significant challenges to program development. Generally speaking, to fully leverage the computing power of the platform, an application needs to exploit at least four levels of parallelism on the platform, including:

[0004] 1. Distributed parallelism: the parallel processing capability among nodes in the cluster;

[0005] 2. End-to-end parallelism: Asynchronous parallel processing capability between the host and device ends within the node;

[0006] 3. Task parallelism: Thread-based multi-task parallel processing capabilities on both the host and device sides;

[0007] 4. Fine-grained parallelism: Fine-grained data parallel (vectorization) processing capabilities within the CPU core and coprocessors.

[0008] As a crucial component of the bioinformatics analysis process, efficient gene alignment can significantly facilitate large-scale bioinformatics analysis. However, classic gene alignment software was not designed and developed specifically for today's advanced computer platforms. Furthermore, compared to mainstream international platforms, my country's domestically developed computer platforms often possess unique architectural features and underlying software stacks. These factors contribute to the significant engineering complexity of localizing and optimizing existing gene alignment software. By customizing and deeply optimizing key gene alignment algorithms for my country's mainstream heterogeneous high-performance computer clusters, and developing independently controllable high-performance gene alignment software, this approach can significantly improve gene alignment throughput, supporting the analysis of massive amounts of genetic data. Furthermore, it can facilitate the localization and independent control of bioinformatics analysis processes in both software and hardware.

[0009] Most of today's mainstream gene alignment software uses a two-stage alignment process called Seeding-and-Extension. The alignment processes of multiple reads and RefSeq are independent of each other and have natural parallelism. For a given read, in the Seeding stage, the alignment algorithm selects a substring as the Seed (seed substring) based on certain rules, and finds all the positions where the Seed appears in the RefSeq in an exact matching manner to form the initial matching position set of the read. In the Extension stage, the algorithm compares the RefSeq fragment anchored by each initial matching position with the non-Seed fragment of the given read to determine the optimal matching position of the read and generate CIGAR information.

[0010] In the seeding phase, to quickly match the exact seed position on the RefSeq, the alignment software preprocesses the RefSeq, typically converting it into a hash table or a BWT (Burrows-Wheeler Transform) + FM-Index index structure. The hash table-based seeding phase predetermines a fixed seed length, l, and constructs a hash table using all l-length substrings of the RefSeq as keys and their positions on the RefSeq as values. For a given seed, all its matching positions on the RefSeq can be retrieved by searching the hash table. Because seeds may contain genetic variation, l should not be too large, as this can easily lead to seeding failures. A smaller seed length, l, not only significantly increases the hash table's memory usage but also results in a large number of seed matching positions, resulting in a significant amount of inefficient computation during the extension phase. BWT+FM-Index employs a different approach to implementing the seeding phase. After the BWT transformation, the occurrences of all substrings with the same suffix in the RefSeq are clustered and stored in a continuous suffix array (SA), flexibly supporting variable-length seeds. The flexible selection of l allows for more accurate initial matching positions in the Seeding phase, reducing inefficient computations in the Extension phase.

[0011] The extension phase performs a fuzzy comparison between two strings of arbitrary lengths n and m, generating a similarity score and CIGAR information. This comparison typically uses the Edit-Distance (ED) algorithm, the Needleman-Wunsch (NW) algorithm, or the Smith-Waterman (SW) algorithm. These algorithms, based on a scoring matrix, use dynamic programming to generate a similarity score for the input strings with a time complexity of O(mn). By backtracking the scoring matrix based on certain rules, the algorithms can also generate CIGAR information describing the matching. Compared to the ED algorithm, the SW and NW algorithms allow for more flexible setting of scoring weights, making them more commonly used in gene alignment workflows. Without loss of generality, the following focuses on the NW and SW algorithms.

[0012] Assume that the two sequences involved in the comparison are A=a1a2…a n (length n) and B = b1b2…b m (length is m). The NW algorithm consists of a scoring matrix H (n+1 rows, m+1 columns) and scoring rules to calculate the matching scores and matching CIGAR information of sequences A and B. The calculation formula of the scoring matrix H is as follows:

[0013] H k0 =-k(0≤k≤n)

[0014] H 0l =-l(0≤l≤m)

[0015]

[0016] where s(a i ,b j ) means a i ,b j The comparison score is 1 if the same, otherwise -1; W1 is a constant, defined as a gap penalty of length 1. By recursively calculating the above formula, the value of each element in the score matrix H can be calculated. i,j Finally, take H n,m is the matching score of A and B. During the scoring process, NW can calculate each sub-diagonal of the matrix H one by one, and the calculation of each element value on the same sub-diagonal is independent of each other. Figure 1 As shown in the calculation steps of the value "α", the evaluation of each element in the subdiagonal i only takes the corresponding characters in A and B and the values ​​of the corresponding elements in the subdiagonals i-1 and i-2 as input. n,m Start backtracking until H 0,0 , get the CIGAR information describing the global comparison of A and B, such as Figure 2 As shown. Generally, let H i,j ≠H0,0 For any point in the backtracking path, the NW algorithm backtracking rule selects H i-1,j 、H i,j-1 and H i-1,j-1 The maximum value is used as the next backtracking point.

[0017] The calculation process of the SW algorithm is highly similar to that of the NW algorithm, but negative numbers are not allowed during the calculation process. Therefore, the calculation rule for the score matrix H becomes:

[0018] H k0 =H ol =0(0≤k≤n,0≤l≤m)

[0019]

[0020] In addition, the SW algorithm uses the same backtracking rules as the NW algorithm, but the backtracking starting point is the largest element H in H max The location, the end point is the first appearance of H i,j (i=0or j=0or H i,j = 0). Through SW backtracking, the CIGAR information describing the longest matching substring in sequences A and B can be obtained.

[0021] While previous gene alignment software mostly relies on a two-stage seeding-and-extension process, it's worth noting that the seeding phase typically lacks the ability to assess the quality of each read's initial alignment position, potentially leading to a large number of low-quality intermediate results that need to be filtered out during the extension phase. This approach not only increases the amount of inefficient computation in the extension phase, but also severely hinders the exploitation of fine-grained parallelism.

[0022] Early gene alignment software, such as BWA, Bowtie, RaserS3, and Hobbes3, were all developed for purely CPU-based homogeneous nodes. Due to the enormous length of RefSeq sequences for complex organisms, the index structures they create consume a significant amount of memory. Limited by the memory capacity of a single computer, these alignment software stored RefSeq and related index data in a compressed format and performed online decompression on demand during the alignment process, consuming significant computing resources and time. Furthermore, even on traditional multi-core CPUs, these early alignment software programs failed to fully exploit the inherent task and fine-grained data parallelism within the CPU. With the introduction of distributed computing and big data technologies, distributed alignment software for CPU clusters, such as SparkBWA, SEAL, and parSRA, emerged. These distributed alignment software implements cross-node partitioning of alignment tasks, enabling coordinated alignment across multiple compute nodes, achieving distributed parallelism. However, these existing distributed solutions do not leverage distributed technology to mitigate the large index data size within each compute node, and online decompression of compressed index data still incurs significant performance overhead. In addition, although most of the commonly used gene comparison software mentioned above can be directly compiled and run on the host side of today's domestic computing platforms, they do not yet directly support the device side composed of different domestic coprocessors.

[0023] Over the past decade, academics have attempted to port typical open-source gene alignment software to many-core architectures. Chen S. et al. parallelized and accelerated BWA on Intel's multi-core Xeon architecture and the many-core accelerator card (MIC). Sima V.M. et al. implemented the Seeding phase of BWA on GPGPU and FPGA architectures, while retaining the Extension phase on the CPU. Using software pipelining, they achieved overlapping execution of these two phases.

[0024] In recent years, domestic researchers have also proposed several efficient gene alignment software, among which Minimap2 is a representative example. Minimap2 uses a hash table as the index structure for RefSeq, making it more suitable for aligning long reads (>10k bases). However, its disadvantage is that the index structure consumes a lot of memory, making it difficult to implement on devices with smaller memory capacities. In addition, for currently common short reads (~100-1000 bases), Minimap2's accuracy is significantly lower than alignment algorithms based on the BWT+FM-Index index structure, such as BWA. S-Aligner and FMapper adopt the design principles of Minimap2 and carry out in-depth performance optimization for the domestically produced many-core processor SW26010. They utilize the SW26010 cluster to achieve distributed parallelization of the gene alignment process, and implement intra-core task parallelism and intra-core vectorized fine-grained parallelism for the SW26010's lightweight core group, achieving significant acceleration. However, S-Aligner and FMapper have the following shortcomings: First, because the host and device sides of the SW26010 platform share a unified addressing space, their technical solutions are difficult to directly extend to other host-device architectures with independent addressing spaces interconnected by a data bus; Second, in S-Aligner and FMapper, the host-side multi-core CPU only participates in data communication and task coordination, so in the actual comparison process, there is a lack of utilization of the asynchronous parallelism between the host and device sides; Third, due to the memory capacity of each SW26010 core group (7.7GB), the online decompression process of the compressed hash table also limits the further improvement of the operating efficiency of S-Aligner and FMapper.

[0025] In summary, previous gene alignment schemes have the following four problems:

[0026] First, the problem of over-coupling of computational processes: In the previous two-stage Seeding-and-Extension process, the Extension stage not only generates matching scores and CIGAR information, but also needs to dynamically identify and filter low-quality matching results. This results in a large amount of redundant computation and hinders the exploration of parallelism.

[0027] Second, the conflict between excessive memory consumption and performance: Due to the large amount of index structure data and the limited memory space, the data compression and online decompression strategies widely used in existing solutions reduce memory usage but also lead to significant performance loss.

[0028] Third, existing solutions have yet to propose a unified execution framework for diverse heterogeneous architectures, enabling the host and device to participate in the entire gene matching process as equals. Furthermore, load balancing between the host and device, and between the multiple execution components within each, has not been fully addressed.

[0029] Fourth, fine-grained parallel technologies, such as vectorized instructions and lightweight threads, have not been fully applied in the entire process of gene matching. The efficient application of fine-grained parallel technologies is the key to fully leveraging the computing capabilities of domestic heterogeneous many-core platforms. Summary of the Invention

[0030] The present invention proposes a new high-performance gene alignment method for autonomous and controllable heterogeneous many-core clusters to make up for the shortcomings of previous solutions.

[0031] The technical solution adopted in the present invention is as follows:

[0032] A high-performance gene alignment method for an autonomous and controllable heterogeneous many-core cluster includes the following steps:

[0033] Expand the Seeding-and-Extension two-stage gene alignment process into a Seeding-Scoring-and-Extension three-stage gene alignment process, in which the Scoring stage is independent of the Extension stage;

[0034] In the Scoring stage, the matching score of the gene fragment read is calculated at the initial matching position of the reference genome RefSeq, and the low-quality initial matching positions are filtered out in a timely manner based on the matching score, minimizing the redundant calculation amount in the subsequent Extension stage alignment process and ensuring sufficient parallelism in the Extension stage.

[0035] Furthermore, on a heterogeneous cluster containing n nodes, a distributed gene alignment circular pipeline is used to perform gene alignment. The distributed gene alignment circular pipeline includes: on the cluster of n nodes, the Read set, RefSeq and related index data with a total memory capacity of m are evenly divided into n parts, and placed in the memory of each node accordingly, so that the memory usage of each node is only m / n; then, each node uses a circular rotation to transmit the Read subset or RefSeq substring to form an alignment pipeline to complete the pairwise alignment of the Read subset and the RefSeq substring.

[0036] Furthermore, the gene comparison calculation framework used in a heterogeneous node includes: dividing the Read set into two subsets, ReadH and ReadD, with the entire comparison process completed by the Host and Device ends respectively; the Host and Device ends use basic linear data structures and follow the same software interface protocol to ensure multi-platform adaptability; using dynamic throughput sampling means to dynamically adjust the task division ratio to ensure load balancing between the Host and Device ends and maximize node throughput.

[0037] Furthermore, in the Scoring and Extension stages, by dynamically clustering matching tasks with the same computational form and scale, matrixed and tensored computational structures are formed. By combining the processes of the NW algorithm and the SW algorithm, a NW batch scoring algorithm based on the subdiagonal matrix and a SW batch matching algorithm based on 3D tensors are designed, respectively, so that both algorithms can be fully accelerated by applying fine-grained parallel technology.

[0038] A high-performance gene alignment system for autonomous and controllable heterogeneous many-core clusters using the above method includes a heterogeneous cluster that expands the two-stage Seeding-and-Extension gene alignment process into a three-stage Seeding-Scoring-and-Extension gene alignment process, wherein the Scoring stage is independent of the Extension stage. In the Scoring stage, the matching scores of gene fragment reads are calculated for the initial matching positions of the reference genome RefSeq, and low-quality initial matching positions are promptly filtered out based on the matching scores, thereby minimizing redundant computational workload in the subsequent Extension stage alignment process and ensuring sufficient parallelism in the Extension stage.

[0039] The beneficial effects of the present invention are embodied in the following aspects:

[0040] 1. This paper proposes a three-stage gene alignment process: Seeding-Scoring-and-Extension. By introducing a separate scoring stage and using an improved match-scoring algorithm, the present solution can promptly and efficiently filter low-quality initial matches during runtime, reducing redundant computational overhead in the subsequent alignment process and ensuring sufficient parallelism.

[0041] 2. The present invention alleviates the contradiction between memory consumption and comparison performance by introducing distributed computing technology in gene comparison. The present invention adopts two-dimensional data partitioning and cyclic comparison pipeline for distributed clusters to significantly reduce the memory usage pressure of each computing node: Assuming that the memory usage of RefSeq and its index structure is m RS, the total memory consumption of the Read sample to be compared is m R , using p nodes to complete the comparison, the memory consumption of each node is only (m RS +m R ) / p, significantly lower than the memory consumption per node of previous solutions m RS +m R The present invention is more friendly to coprocessors with smaller memory capacity and can effectively avoid performance loss caused by online decompression of compressed data.

[0042] 3. To fully utilize the platform's fine-grained parallel processing capabilities, the present invention proposes a matrix-based NW matching scoring algorithm and a tensor-based SW algorithm. Both have highly structured computational structures, and their core computations can be efficiently implemented using the platform's various fine-grained parallel techniques. Assuming the platform vector width is 1 (or using 1 lightweight thread), the application of fine-grained parallelism can theoretically increase the matching throughput by 1 times.

[0043] 4. This invention implements high-performance gene alignment software for a typical domestically produced heterogeneous cluster, leveraging the platform's distributed parallelism, master-coprocessor asynchronous parallelism, intra-processor task parallelism, and fine-grained data parallelism to fully tap the target platform's computational potential. This solution proposes a unified gene alignment software framework to shield platform differences, enabling both the host and device to participate in the alignment process as equals, fully leveraging the platform's computational performance while maintaining high platform portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 :NW schematic diagram of scoring matrix calculation along the sub-diagonal direction.

[0045] Figure 2 : Example diagram of the backtracking process of the NW algorithm.

[0046] Figure 3 : Schematic diagram of the two-dimensional data partitioning and cyclic comparison pipeline on four computing nodes.

[0047] Figure 4 : Execution flow chart of process Align-Proc.

[0048] Figure 5 : The cyclic alignment process of four DH_Aligner processes on the CPU+DCU cluster, where (a)-(d) respectively represent the correspondence between the Read part and the process and the data exchange process in the program from the 0th to the 3rd loop iteration. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to specific embodiments and drawings.

[0050] 1. Main contents of the present invention

[0051] The present invention mainly includes the following four aspects:

[0052] First, a three-stage gene alignment process based on Seeding-Scoring-and-Extension: Seeding is the seed position selection stage, Scoring is the initial matching position scoring stage, and Extension is the extended matching stage. The present invention expands the previous two-stage Seeding-and-Extension alignment process, making the scoring of each initial matching position of the Read independent of the Extension stage. The scoring process uses a simplified and highly parallel NW algorithm to promptly filter out low-quality matching positions, minimize the waste of computing resources in the Extension stage, and ensure that sufficient fine-grained parallelism in the Extension stage can be exploited.

[0053] Second, the present invention proposes a distributed gene alignment loop pipeline for domestic heterogeneous clusters: on a cluster with a total of n nodes, the Read set and RefSeq are divided into n subsets and substrings respectively, and placed in the memory of each node accordingly. Afterwards, the Read subsets or RefSeq substrings are transferred between the computing nodes in a round-robin manner to complete the pairwise alignment of the Read subsets and RefSeq substrings. The introduction of the alignment pipeline can, on the one hand, solve the problem of insufficient memory by simply increasing the number of devices. On the other hand, it can also effectively avoid the loss of computing performance caused by excessive data compression.

[0054] Third, the present invention proposes a gene comparison calculation framework for typical domestic heterogeneous many-core nodes: In a heterogeneous node, the Read set is further divided into Read H and Read D The two subsets are compared in their entirety, with the host and device completing the entire comparison process, respectively. Both sides utilize a basic linear data structure and adhere to the same software interface protocol, ensuring the solution's multi-platform adaptability. Furthermore, because the comparison throughput rates of the host and device often differ, this invention employs dynamic throughput sampling to dynamically adjust the task allocation ratio to ensure load balancing between the host and device.

[0055] Fourth, a core gene alignment algorithm based on fine-grained parallel technology: In the scoring and extension stages, by dynamically clustering matching tasks with the same computational form and scale, a matrixed and tensored computational structure is formed. Combining the NW algorithm and SW algorithm processes, a matrixed NW scoring algorithm and a 3D tensored SW algorithm are designed, respectively, enabling the above algorithms to be fully accelerated by applying fine-grained parallel technology.

[0056] 2. Specific technical solutions of the present invention

[0057] 2.1 Seeding-Scoring-and-Extension Three-Stage Gene Alignment Process

[0058] In the three-stage gene alignment process, the Seeding stage remains consistent with the original solution. For any initial matching position p of a given Read (denoted as r), the solution of the present invention uses a simplified and improved NW algorithm (i.e., an NW scoring algorithm based on a subdiagonal matrix) in the Scoring stage to calculate its matching score s. Among them, the algorithm needs to make full use of the vectorized computing capabilities of the platform to ensure the efficiency of the Scoring stage (see Algorithm 1 and Algorithm 2 in Section 2.4 for details). Afterwards, the solution of the present invention sorts all the initial matching positions of r based on the score s to obtain the Top-k (k is a user-specified parameter) optimal matching position of r. Finally, in the Extension stage, the solution of the present invention uses a complete SW algorithm to analyze the detailed matching information of Read and RefSeq (see Algorithm 3, Algorithm 4, and Algorithm 5 in Section 2.4 for details).

[0059] 2.2 Distributed Gene Alignment Circular Pipeline

[0060] Suppose there are n computing nodes set p∈{p0,p1…p n-1}, Read set R and reference genome RS. The present invention divides R and RS into R according to the number of nodes n. i ∈{R0,R1,…,R n-1} and RS i ∈{RS0,RS1,…,RS n-1}, forming a two-dimensional segmentation of the entire input data. In the initial state, each node p i Corresponding loading R i and RS i For any R i , in order to determine its optimal match on the RS as a whole, R i Need to be with all RS j(0≤j<n), perform pairwise comparison one by one. Therefore, the present invention sets up a closed-loop comparison pipeline among each computing node to implement the above process. Generally, for a loop comparison pipeline composed of n nodes, all R can be completed through n iterations. i In the k-th (0 ≤ k < n) iteration, any node p i is responsible for completing the local comparison between RS i and R j to form a locally optimal match. After that, node p i transmits R j to the p i+1 node, and so on until all R i and PS j complete pairwise comparison, as Figure 3 shown. Figure 3 In Figure 3 , g_offset represents the global position offset of the RefSeq segmentation.

[0061] 2.3 Gene Alignment Computing Framework for Typical Domestic Heterogeneous Multi-Core Nodes

[0062] Assume that each gene alignment process manages a multi-core CPU and a coprocessor, and the two can run independently in an asynchronous manner. At this parallel level, the present invention aims to achieve the following two goals:

[0063] 1) Given the throughput difference between the multi-core CPU and the coprocessor, the present invention attempts to let the CPU and the coprocessor jointly participate in the local comparison of the Read set in an equal role and ensure load balance between the two;

[0064] 2) Propose a unified comparison framework for different domestic main-coprocessor architectures to improve the platform compatibility of the present invention.

[0065] To achieve the above goals, the processing flow of each comparison process is as Figure 4 shown:

[0066] As Figure 4As shown, the main thread of an alignment process "Align-Proc" is responsible for reading the RefSeq segment, BWT, SA and the Read set belonging to the Align-Proc from the external memory before the alignment loop starts. Afterwards, the main thread derives two child threads "H-Thread" and "D-Thread" to respectively manage the actual alignment tasks on the Host and Device sides, and ensure the asynchrony of the two sides. In each alignment iteration, Align-Proc divides the current Read set to be aligned in a certain ratio, and delivers it to the Host and Device sides for parallel processing respectively, and the two sides jointly complete the Seeding-Scoring-and-Extension alignment process of the Read set. Among them, H-Thread uses a nested parallel mechanism to open several OpenMP threads on the multi-core CPU side to complete the alignment process, such as Figure 4 As shown in "Align_OMP_Backend" in the figure; the Device side can implement the alignment process for different coprocessors, such as Figure 4 As shown in "Align_DEV_Backend" in the diagram, after the alignment is completed at both ends, the H-Thread collects and summarizes the results. It then sends the locally aligned set of reads to the neighboring process and receives the next batch of reads sent by the neighboring process. At this point, Align-Proc completes one iteration of the alignment loop. After all iterations are complete, Align-Proc writes the matching positions of all reads and the CIGAR information back to external memory.

[0067] Although Align_OMP_Backend and Align_DEV_Backend operate on different architectures, they implement a unified function interface, shielding the upper-layer Align_Proc-based distributed gene alignment framework from different platform peculiarities. This ensures the platform portability of the present invention to a certain extent. Furthermore, for both Read and RefSeq and their index structures, the present invention uses one-dimensional arrays as their storage structure, ensuring the multi-platform adaptability of data storage and transmission to the greatest extent possible.

[0068] 2.4 Core Alignment Algorithm for Fine-Grained Parallelism

[0069] a. NW scoring algorithm based on the secondary diagonal matrix

[0070] Assume that for a Read with c initial matching positions, the length of its non-Seed region substring A is lenA, and let m = lenA + 1. Take a substring of the reference gene sequence at each matching position to form a substring set B k,k∈{0,1,...,c-1}, where the length of each substring is lenB. Let n=lenB+1,m≤n. According to the NW algorithm, A and B k The comparison will generate c scoring matrices H k ,k∈{0,1,...,c-1}. And for any score matrix H k , the NW algorithm can be used along each diagonal w i ,i∈{0,1,...,m+n-2} are compared and scored in turn, and w i The calculation depends on w i-1 With w i-2 Therefore, the batch processing NW algorithm for fine-grained parallelism constructs the set of sub-diagonal matrices AD j ,j∈{0,1,2}, where AD j Used to store H k The secondary diagonal w i 、w i-1 and w i-2 .AD j Stored in memory in column-first storage mode, each row dimension is min(m,n), H k The length of the longest secondary diagonal, each column dimension is c, storing A and B k The local comparison results are shown in Algorithm 1:

[0071] Algorithm 1: Batch NW Algorithm

[0072] S01 algorithm input: Read string A, length l A , the number of strings to be compared in RefSeq c, the array of strings to be compared B (two-dimensional pointer array), the length of the string to be compared l B .

[0073] S02 algorithm output: score array of comparison scores between A and each string to be compared.

[0074] S03 defines two variables m and n, whose initial values ​​are m←l A +1, n←l B +1, the number of rows and columns of the scoring matrix respectively.

[0075] ← is the assignment symbol (the same below).

[0076] S04 defines three matrices AD0, AD1, and AD2 and stores them in memory in column - first order. The initial value of each element of the matrices is 0. The maximum dimension of the three matrices is c rows and d←min(m,n) columns, where min(a,b) is a function that takes the minimum value of a and b. During the execution of the algorithm, since the number of columns of AD0, AD1, and AD2 changes during the execution, d0, d1, and d2 are used to represent the dynamic number of columns of each matrix respectively.

[0077] S05 changes the memory layout of the values of the strings to be compared: defines a two - dimensional string array b with dimension (c, l B ), and stores it in column - first order. Copies the elements of the string array B to be compared into b, so that the elements with the same subscript in each string to be compared in B are stored in a column of b, making them continuously arranged in memory.

[0078] S06 For the i - th anti - diagonal (0≤i < n) of the scoring matrix, perform the following steps:

[0079] S061 d2←min(m,i + 1), which is the number of columns of AD2 and is the length of the i - th anti - diagonal, that is, the number of columns of AD2.

[0080] S062 Define a logical variable flag: if i < m, then the value of flag is 1, otherwise it is 0.

[0081] S063 Initialize the 0 - th and d2 - 1 columns of matrix AD2, where the element in the k - th row is set to the initial value of –i.

[0082] S064 For the j - th column of matrix AD2, 1≤j < d2 - flag columns:

[0083] S0641 Define subscripts i1←j - 1, j1←i - j - 1.

[0084] S0642 Call algorithm 2 with the parameter list (AD 0…2 , d 0…2 , c, A[i1], b, j1, j - 1, j, j - 1, j), where

[0085] The operator "[]" is used to get array elements (the same below).

[0086] S065 Transform the AD matrices: AD0←AD1, AD1←AD2, and clear AD2 (note: in the implementation, change the pointers instead of copying the memory). S07 For the i - th anti - diagonal (n≤i < m + n) of the scoring matrix, perform the following steps:

[0087] S071 d2←m-(i - n)–1 is the length of the i - th anti - diagonal, that is, the number of columns of AD2.

[0088] S072 defines a logical variable flag: if i is not equal to n, its value is 1, otherwise it is 0.

[0089] S073 For the j-th column (0≤j <d2):

[0090] S0731 defines subscripts i1←i-n+j, j1←nj-2.

[0091] S0732 is a parameter list (AD 0…2 , d 0…2 ,c,A[i1],b,j1,j+flag,j+1,j,j) call Algorithm 2.

[0092] S075 transforms the AD matrix: AD0←AD1, AD1←AD2, and clears AD2 (Note: in this implementation, the matrix pointer is changed instead of the memory copy).

[0093] S08 copies the first column of AD1 into the score array to obtain the NW matching score for each character string to be matched.

[0094] Algorithm 2: NW algorithm core calculation process

[0095] S01 algorithm input: subdiagonal matrices AD0, AD1, AD2 and their corresponding columns d0, d1, d2, the number of characters to be compared c, the comparison character a, and the comparison character array b. Subscript: subscript j of array b. 00 , AD0 subscript j 01 , AD1 subscript j 10 and j 11 , AD2 subscript j 20 .

[0096] The output of the S02 algorithm is the j20th column of AD2, where each element is the comparison score between each character in array b and character a.

[0097] S03 defines pointer f 10 ←&AD1[0+j 10 *c], where the operator & is used to obtain the memory address (the same below).

[0098] S04 defines pointer f 11 ←&AD1[0+j 11 *c],f 01 ←&AD0[0+j 01 *c], define pointer f 20 ←&AD2[0+j 20 *c],

[0099] Define pointer b 00 ←&b[0+j 00 *c].

[0100] For each element i (0 ≤ i < c), S05 performs the core calculation steps of the NW algorithm:

[0101] S051 If b 00 [i] is equal to a, then t0 ← 1; otherwise t0 ← -1.

[0102] S052 t1 ← f 10 [i] - 1, t2 ← f 11 [i] - 1, t3 ← f 01 [i] + t0.

[0103] S053 f 20 [i] ← max(t1, t2, t3), where max(a, b, c) is the element that takes the maximum value among a, b, and c.

[0104] In Algorithm 1, since the alignment scores between A and all B k are independent of each other, each column of the matrix AD j is continuously stored in memory, enabling the calculations in Algorithm 2 to naturally support fine-grained data parallelism. On the CPU architecture, the core calculations (S051 - S053) of Algorithm 2 can be efficiently implemented using vector instructions such as AVX and AVX2. On the DCU / GPGPU architecture, its lightweight thread parallelism can be utilized to process this loop and ensure the efficiency of memory access (such as coalesced memory access). Additionally, by unrolling the loops in lines S064 and S073 of Algorithm 1, the instruction scheduling range can be further expanded to assist the processor in improving the throughput of instruction execution.

[0105] b. SW algorithm based on 3D tensor

[0106] After being screened by the NW scoring algorithm, all initial matches of the Reads have high quality. Therefore, in this invention, the SW algorithm can calculate the scoring matrix in the most regular way and avoid wasting computing resources. Now, the SW algorithm based on 3D tensor is introduced. Without loss of generality, assume that the alignment process has the entire reference genome RS (with length l RS ), the Read set R contains n Reads, R = {r0, r1, …, r n-1}, the lengths of the corresponding Reads are L R = {l0, l1, …, l n-1}, and the set of the top-k initial matching positions of each Read forms the set P k = {p0, p1, …, p n-1}. Now, the 3D tensor SW algorithm is used to calculate the CIGAR information of the corresponding matching positions of each Read on RS, as shown in Algorithm 3.

[0107] Algorithm 3: Tensor-based batch SW algorithm

[0108] S01 algorithm input: reference genome RS, RS length L RS , Read number n, Read array R, each Read length array L R , the matching position array P of each Read on RS.

[0109] S02 algorithm output: CIAGR information comparing each read with the reference genome fragment.

[0110] S03 defines minLen as L R Minimum value of the elements in .

[0111] S04 defines maxLen as L R The maximum value of the elements.

[0112] S05 For length L (minLen≤L≤maxLen):

[0113] S051 Find all Reads of length L and their matching positions in R, the number of which is n L , respectively stored in array R L and P L (Note: R L Characters with the same subscript in each Read are stored consecutively in memory).

[0114] S052 According to P L Each matching position in RS intercepts a substring of length L and stores it in array RS L In the reference genome segment, the target is compared (Note: RS L The same subscript characters in each fragment to be compared are stored consecutively in memory).

[0115] S053 applies for a three-dimensional floating point tensor S, whose dimensions are (n L , L+1, L+1), byte tensor D, whose dimension is (n L , L+1, L+1), ensuring that the first dimension of S and D is stored continuously.

[0116] S054 applies for a one-dimensional integer array maxI, whose dimension is n L , maxJ, whose dimension is n L , and initialize each element to 0.

[0117] S055 applies for a one-dimensional floating point array score, whose dimension is n L , and initialize each element to 0.

[0118] S056 calls Algorithm 4 with the parameter list (n L , L + 1, R L , RS L , S, D, maxI, maxJ, score).

[0119] S057 calls Algorithm 5 with the parameter list (n L , L, S, D, maxI, maxJ, CIAGAR).

[0120] Algorithm 4: Core calculation process of the tensor SW algorithm

[0121] Define the direction types FROM_UPPER = 0, FROM_LEFT = 1, FROM_DIAG = 2.

[0122] Input of S01 algorithm: The number of strings to be compared n, the SW tensor dimension L, the Read array R, and the reference genome fragment array RS.

[0123] Output of S02 algorithm: The three-dimensional scoring tensor S, the three-dimensional direction tensor D, and the one-dimensional arrays maxI, maxJ, and score representing the row and column indices of the maximum scoring position and the score.

[0124] S03 Set the part of the tensor S[0…n - 1][0…L - 1][0] to 0.

[0125] S04 Set the part of the tensor S[0…n - 1][0][0…L - 1] to 0.

[0126] S05 Set the array score[0…n - 1] to 0.

[0127] S06 For all i (0 ≤ i < L), j (0 ≤ j < L):

[0128] S061 Perform the core calculation of the SW algorithm for all k (0 ≤ k < L):​​​​​​​​​​​​​​​

[0134] S0616max2←max(max1,t0).

[0135] S0617 If max1>t0, then d2←d1, otherwise d2←FROM_DIAG.

[0136] S0618S[k][i][j]←max2.

[0137] S0619D[k][i][j]←d2.

[0138] S0620 updates the subscripts maxI[k], maxJ[k] and the maximum equal score score[k] of the current maximum score.

[0139] Algorithm 5: Backtracking process of CIGAR string generation

[0140] Define table[3] = {"I", "D", "M"}

[0141] S01 algorithm input: number of strings to be compared n, string length L, batch SW algorithm score tensor S, batch SW algorithm direction tensor D, maximum score row subscript array maxI, maximum score column subscript array maxJ.

[0142] S02 algorithm output: CIGAR comparison results for each read.

[0143] S03 For the kth Read (0≤k <n):

[0144] S031i←maxI[k],j←maxJ[k]

[0145] S032cigar[k]←str(Li)+“S”+cigar[k], where the function str(a) converts the value a into a string, the content in “” is a string literal constant, and + is a string concatenation operator.

[0146] S033 When i≥0 and j≥0 and S[k][i][j]>0, perform the following backtracking steps:

[0147] S0331 Current direction d←D[k][i][j].

[0148] S0332 initializes variables i1←i, j1←j, cnt←0.

[0149] s0333 When D[k][i1][j1] is equal to d and i1 ≥ 0 and j1 ≥ 0, execute:

[0150] S03331cnt←cnt+1.

[0151] S03332 If d is equal to FROM_DIAG, then i1←i1-1, j1←j1-1.

[0152] S03333 Otherwise if d is equal to FROM_UPPER, then i1←i1-1.

[0153] S03334 Otherwise j1←j1-1.

[0154] S0334i←i1.

[0155] S0335j←j1.

[0156] S0336cigar[k]←str(cnt)+table[d]+cigar[k].

[0157] S034 If i>1, execute cigar[k]←str(ii)+“S”+cigar[k].

[0158] Algorithm 3 shows the tensor-based SW algorithm. Among them, Algorithm 3 (S051-S053) first clusters based on the length of the reads. Reads with the same length are clustered together to form a subset "R L ", and develop a regular tensor computing structure for it. Among them, R L Each Read in the corresponding "RS L The substrings are rearranged so that the characters with the same subscript in each base string are stored continuously in memory. Algorithm 4 is the core calculation of the SW algorithm, in which each iteration of the loop shown in S061 is independent of each other and can be easily accelerated by various vectorization methods. L After completing the SW core calculation, each Read executes the backtracking process independently (Algorithm 5), and each loop (the main body is S03331-S03334) can be directly parallelized. 3. Specific Examples

[0160] The technical solution of the present invention is implemented on a heterogeneous cluster composed of Sugon multi-core CPU + DCU, named DH_Aligner. The cluster contains 4 multi-core CPUs (Host side) and 4 DCU accelerator cards (Device side). Each multi-core CPU contains 8 CPU cores, supports AVX2 vector instruction set and OpenMP thread model. Each DCU accelerator card has 32 multi-stream processors and supports HIP lightweight thread model. Multi-core CPUs correspond to DCU cards one by one, and each pair of CPUs and DCUs are connected by a PCI-e 3.0 bus with a bandwidth of 16GB / s (full duplex). DH_Aligner is implemented using 4 alignment processes (denoted as P i,i∈{0,1,2,3}), respectively controlling a pair of multi-core CPUs and DCUs, and the implementation is as follows:

[0161] Given a reference genome RS, first divide it into RS0, RS1, RS2 and RS3 in sequence. i,i∈{0,1,2,3} , record its starting offset off in RS i With length len i Afterwards, RS i Use the open source software glufsort to perform BWT transformation and get bwt i and suffix array sa i Finally, RS i 、off i 、len i 、bwt i with sa i Stored to the comparison process P i The data is stored in the CPU memory and DCU device memory for subsequent use.

[0162] For all Reads R, divide them into subsets R0, R1, R2 and R3, and place them in P i Each comparison process can store R i,i∈{0,1,2,3} It is further divided and transmitted to the DCU device memory for processing.

[0163] Maintain RS j Resides in the comparison process P j In the Message Passing Interface (MPI), any R i Based on the cyclic alignment pipeline, the alignment cluster composed of the above 4 nodes can complete all R i With RS j,j∈{0,1,2,3} Pairwise comparisons of Figure 5 shown.

[0164] In iteration step k, k∈{0,1,2,3}, R i In process P j Get the local alignment result align i,j , which includes R i The local optimal matching position and matching score of each Read in . Summarize all alignment i,k , we can get the global optimal match of all reads. Then, DH_Aligner can generate CIGAR information at the optimal matching position for each read according to application needs.

[0165] In process P iDuring the internal comparison process, two OpenMP threads are used to control the CPU comparison process and the DCU comparison process respectively, to ensure the asynchronous and parallel execution of both ends. i When performing the alignment for the first time, R i Evenly distribute it to the CPU and DCU ends, and calculate the throughput of both ends based on the calculation time and the number of reads: Assume that the CPU throughput is T cpu , the DCU throughput is T DCU In the subsequent comparison, the CPU side is

[0166]

[0167] R for proportional processing j,j≠i , while the DCU end is

[0168]

[0169] R for proportional processing j,j≠i , ensuring dynamic load balancing between the CPU and DCU, maximizing node throughput. For the transmission of reads between the two end memories, the present invention uses the DCU end basic software stack ROCm to implement it.

[0170] For the comparison process on the CPU side, the present invention implements multi-threaded parallel processing based on the nested parallel mechanism of OpenMP. Since a separate thread is used to control the DCU, the CPU side only uses 7 threads to complete the Read comparison. By setting the "GOMP_Thread_Affinity" environment variable and the "numactl-CPU-bind" command, it is ensured that the 8 threads used on the CPU side correspond one-to-one to the CPU physical cores. Each CPU thread completes the Seeding-Scoring-and-Extension process of multiple Reads in turn: in the Seeding stage, each CPU thread obtains the initial matching position of each Read by indexing the bwt and sa in the memory; in the Scoring stage, each thread uses the batch NW algorithm based on AVX2 vector instructions to complete the similarity scoring for all initial matching positions of the given Read, and select the Top-k optimal matching positions; in the Extension stage, the CPU thread uses the tensor SW algorithm based on the AVX2 computing core to complete the generation of CIGAR information describing the Top-k optimal matching positions.

[0171] For the alignment process on the DCU side, DH_Aligner uses thread blocks and lightweight threads to complete the Seeding-Scoring-and-Extension process. Each thread block contains several lightweight threads that process several Reads in parallel. In the Seeding phase, the thread block is regarded as a whole, and all its threads synchronously index the bwt and sa in the DCU memory to obtain all the initial matching positions of the Read. In the Scoring phase, for each initial matching position of a given Read, a lightweight thread is used to independently calculate its similarity score to implement the fine-grained parallel batch processing NW algorithm on the DCU side. Similarly, in the Extension phase, the DCU side uses a tensor SW algorithm based on lightweight threads to complete the generation of CIGAR information. Finally, the DCU side results are returned to the system main memory.

[0172] The solution of the present invention is instantiated on the existing CPU+DCU architecture. The software, named "DH_Aligner," meets all the features of the aforementioned solution. In a 4-CPU+4-DCU environment, DH_Aligner achieves a throughput of 500K reads / s for a read set with an average length of 150 bases. This throughput is 1-2 orders of magnitude higher than many previous gene alignment solutions.

[0173] Another embodiment of the present invention provides a high-performance gene alignment system for an autonomous and controllable heterogeneous many-core cluster, which includes a heterogeneous cluster. The heterogeneous cluster expands the two-stage Seeding-and-Extension gene alignment process into a three-stage Seeding-Scoring-and-Extension gene alignment process, wherein the Scoring stage is independent of the Extension stage; in the Scoring stage, the matching score of the gene fragment read is calculated at the initial matching position of the reference genome RefSeq, and low-quality initial matching positions are promptly filtered out based on the matching score, thereby minimizing the redundant calculation amount in the alignment process of the subsequent Extension stage and ensuring sufficient parallelism of the Extension stage.

[0174] The specific implementation method of gene alignment in the above system can be found in the above description of the method of the present invention. For example: the heterogeneous cluster adopts a distributed gene alignment loop pipeline to perform gene alignment, and the distributed gene alignment loop pipeline includes: on a cluster of n nodes, the Read set, RefSeq and related index data with a total memory capacity of m are evenly divided into n parts, and placed in the memory of each node accordingly, so that the memory usage of each node is only m / n; after that, each node uses a circular rotation to transfer the Read subset or RefSeq substring to form an alignment pipeline to complete the pairwise alignment of the Read subset and the RefSeq substring. In a heterogeneous node of the heterogeneous cluster, the Read set is divided into Read H and Read D The two subsets, respectively, are matched by the host and device, completing the entire comparison process. Both sides utilize basic linear data structures and adhere to the same software interface protocol to ensure multi-platform adaptability. Dynamic throughput sampling is employed to dynamically adjust the task allocation ratio to ensure load balancing between the host and device, maximizing node throughput. During the scoring and extension phases, the heterogeneous cluster dynamically clusters matching tasks of the same computational form and scale, forming matrixed and tensored computational structures. Combining the processes of the NW and SW algorithms, a NW batch scoring algorithm based on off-diagonal matrices and a SW batch matching algorithm based on 3D tensors are designed, respectively. This allows both algorithms to be fully accelerated using fine-grained parallelism.

[0175] The specific embodiments of the present invention disclosed above are intended to facilitate understanding and implementation of the present invention. Those skilled in the art will appreciate that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the embodiments disclosed in this specification; the scope of protection of the present invention shall be determined by the scope defined in the claims.

Claims

1. A high-performance gene alignment method for autonomous and controllable heterogeneous many-core clusters, characterized by: The following steps are involved: Expand the Seeding-and-Extension two-stage gene alignment process into a Seeding-Scoring-and-Extension three-stage gene alignment process, in which the Scoring stage is independent of the Extension stage; In the Scoring phase, the matching score of the gene fragment read is calculated for the initial matching position of the reference genome RefSeq, and low-quality initial matching positions are promptly filtered out based on the matching score, minimizing the redundant calculation amount in the subsequent Extension phase alignment process and ensuring sufficient parallelism in the Extension phase; In the scoring and extension phases, matching tasks with the same computational form and scale are dynamically clustered to form matrixed and tensored computational structures. Combining the processes of the NW and SW algorithms, we design a NW batch scoring algorithm based on off-diagonal matrices and a SW batch matching algorithm based on 3D tensors, respectively. This allows both algorithms to be fully accelerated using fine-grained parallel technology. Assume that for a Read with c initial matching positions, the length of its non-Seed region substring A is lenA, let m = lenA + 1; take a substring of the reference gene sequence at each matching position to form a substring set B k ,k∈{0,1,...,c-1}, where the length of each substring is lenB, let n=lenB+1,m≤n; according to the NW algorithm, A and B k The comparison of the corresponding c score matrices H k ,k∈{0,1,...,c-1}; for any score matrix H k , NW along each of its diagonals w i ,i∈{0,1,...,m+n-2} are compared and scored in turn, and w i The calculation depends on w i-1 With w i-2 The NW batch scoring algorithm based on the sub-diagonal matrix constructs a sub-diagonal matrix set AD j ,j∈{0,1,2}, where AD j Used to store H k The secondary diagonal w i 、w i-1 and w i-2 ;AD j Stored in memory in column-first storage mode, each row dimension is min(m,n), H k The length of the longest secondary diagonal, each column dimension is c, storing A and B k Local comparison results.

2. The method according to claim 1, characterized in that On a heterogeneous cluster containing n nodes, a distributed gene alignment loop pipeline is used to perform gene alignment. The distributed gene alignment loop pipeline includes: on the cluster of n nodes, the Read set, RefSeq and related index data with a total memory capacity of m are evenly divided into n parts, and placed in the memory of each node accordingly, so that the memory usage of each node is only m / n; then, each node uses a circular round-robin method to transmit the Read subset or RefSeq substring to form an alignment pipeline to complete the pairwise alignment of the Read subset and the RefSeq substring.

3. The method according to claim 1, characterized in that The gene alignment computation framework used in a heterogeneous node includes: dividing the Read set into Read H and Read D The two subsets are compared in their entirety on the host and device sides, respectively. Both sides use basic linear data structures and follow the same software interface protocol to ensure multi-platform adaptability. Dynamic throughput sampling is employed to dynamically adjust the task allocation ratio to ensure load balancing between the host and device sides and maximize node throughput.

4. The method according to claim 1, wherein The 3D tensor-based SW batch matching algorithm first clusters reads based on their length. Reads of the same length are grouped together to form a subset Rlen, for which a regular tensor calculation structure is created. Each read and its corresponding RS substring in Rlen are rearranged so that characters with the same subscript in each base string are stored consecutively in memory. The core calculation of the SW algorithm is carried out in a loop where each iteration is independent of each other, making it easy to use various vectorization methods for efficient acceleration. After completing the SW core calculation for Rlen, each Read executes the backtracking process independently, and the loop can be directly parallelized.

5. A high-performance gene alignment system for autonomous and controllable heterogeneous many-core clusters using the method according to any one of claims 1 to 4, characterized in that: The system includes heterogeneous clusters, which expand the two-stage Seeding-and-Extension gene alignment process into a three-stage Seeding-Scoring-and-Extension gene alignment process, in which the Scoring stage is independent of the Extension stage. In the Scoring stage, the matching scores of gene fragment reads are calculated for the initial matching positions of the reference genome RefSeq, and low-quality initial matching positions are promptly filtered out based on the matching scores, minimizing the redundant computational workload in the subsequent Extension stage alignment process and ensuring sufficient parallelism in the Extension stage.

6. The system according to claim 5, characterized in that The heterogeneous cluster uses a distributed gene alignment loop pipeline to perform gene alignment. The distributed gene alignment loop pipeline includes: on a cluster of n nodes, the Read set, RefSeq and related index data with a total memory capacity of m are evenly divided into n parts, and placed in the memory of each node accordingly, so that the memory usage of each node is only m / n; then, each node uses a circular round-robin method to transmit the Read subset or RefSeq substring to form an alignment pipeline to complete the pairwise alignment of the Read subset and the RefSeq substring.

7. The system according to claim 5, characterized in that In a heterogeneous node of the heterogeneous cluster, the Read set is divided into Read H and Read D The two subsets are compared in their entirety on the host and device sides, respectively. Both sides use basic linear data structures and follow the same software interface protocol to ensure multi-platform adaptability. Dynamic throughput sampling is employed to dynamically adjust the task allocation ratio to ensure load balancing between the host and device sides and maximize node throughput.

Images