Genome assembly method and device, computer readable storage medium, and processor

Abstract

The present invention discloses a genome assembly method and device, a computer-readable storage medium, and a processor. The method includes: responding to a first command to generate a predetermined database using raw sequencing data; responding to a second command to generate multiple blocks based on multiple files; responding to a third command to obtain target data in multiple blocks to obtain a target file based on the target data; responding to a fourth command to generate an exclusive lock; when it is determined that there is a running file currently, locking other files to process the running file; when it is determined that the processing of the running file is finished, releasing the exclusive lock to process the raw sequencing data based on the target file to obtain a target genome. The present invention solves the technical problem in the related art that when assembling a genome, the assembly efficiency is relatively low due to the limitation of the bandwidth of the computer cluster and the speed of the hard disk.

Description

Technical Field

[0001] The present invention relates to the field of gene sequencing technology, and in particular to a genome assembly method and device, a computer-readable storage medium, and a processor. Background Art

[0002] Pb-assembly is an assembly software developed for the third generation of pacbio data. The previous version was named falcon. The input file of the software is the original pacbio data, and the output file is a genome file in fasta format. The principle of Pb-assembly is to first self-correct the original pacbio data to obtain the consensus sequence after correction, and then assemble the consensus sequence according to the Overlap-Layout-Consensus algorithm (olc algorithm) to obtain the assembled genome file.

[0003] When Pacbio data is self-corrected, the raw data will be stored as a Dazzler database (Dazzler DB). The Dazzler database has 5 files. The raw_reads.db file stores the raw data input file and the information for segmenting the raw data; the .raw_reads.idx file is a binary file, stored as the raw data index file; the .raw_reads.bps file is a binary file, stored as the compressed DNA sequencing sequence; the .raw_reads.dust.anno file is stored as the annotation file for each read interval in the low-complexity interval reads; and the .raw_reads.dust.data file is stored as the low-complexity interval reads information. Because the error correction (cns) step has hundreds or thousands of tasks running in parallel on the cluster, each task will access these 5 files, and the cluster's computing nodes also have dozens of computers. If the data is synchronized to the memory, the computer node to which each task is delivered is not fixed. Each task synchronizes the database once, which will cause the multiple cns tasks to affect each other, resulting in the task not being able to run completely. Therefore, the strategy used by the software is to store the above 5 files on the hard disk, and multiple tasks access the 5 database files under the same path to avoid confusion in the cns step. During the self-correction process of the third-generation data, Pb-assembly will read these 5 files from the hard disk for the consensus step. The larger the genome, the larger the amount of third-generation sequencing data, and the more storage the Dazzler database occupies. For genomes larger than 4G, the process of the cluster and hard disk reading the Dazzler database will be extremely slow due to the bandwidth limitations of the cluster and hard disk. For genomes larger than 4G, the entire pb-assembly process takes about 1 month.

[0004] With regard to the problem of low assembly efficiency in genome assembly in the above-mentioned related technologies due to limitations on computer cluster bandwidth and hard disk speed, no effective solution has been proposed so far. Summary of the invention

[0005] The embodiments of the present invention provide a genome assembly method and device, a computer-readable storage medium, and a processor to at least solve the technical problem in the related art that when assembling a genome, the assembly efficiency is relatively low due to the limitations of the computer cluster bandwidth and the speed of the hard disk.

[0006] According to one aspect of an embodiment of the present invention, a method for assembling a genome is provided, comprising: responding to a first command to generate a predetermined database using raw sequencing data, wherein the predetermined database includes multiple files; responding to a second command to generate multiple blocks based on the multiple files; responding to a third command to obtain target data within the multiple blocks to obtain a target file based on the target data; responding to a fourth command to generate an exclusive lock, wherein the exclusive lock is used to lock other files except a currently running file; when it is determined that the currently running file exists, locking the other files to process the running file; when it is determined that the processing of the running file is finished, releasing the exclusive lock to process the raw sequencing data based on the target file to obtain a target genome.

[0007] Optionally, in response to a third command, target data within the multiple blocks are obtained to obtain a target file based on the target data, including: in response to the third command, obtaining a first repeating sequence and a second repeating sequence for each of the multiple blocks; masking predetermined objects in the first repeating sequence and the second repeating sequence to obtain masked data in each of the multiple blocks; and comparing the masked data to obtain a target file.

[0008] Optionally, when it is determined that the running file currently exists, the other files are locked, so that before the running file is processed, the genome assembly method also includes: judging whether there is a completed file under the predetermined path to obtain a judgment result; when the judgment result indicates that the completed file exists under the predetermined path, outputting data processing completion; when the judgment result indicates that the completed file does not exist under the predetermined path, continuing to obtain the target file.

[0009] Optionally, the exclusive lock includes an exclusive command, and when it is determined that the running file currently exists, the other files are locked, including: using the exclusive command to read the multiple files to the predetermined path respectively; detecting whether the running file exists under the predetermined path to obtain a detection result; when the detection result indicates that the running file exists under the predetermined path, locking the other files.

[0010] Optionally, the genome assembly method further includes: when it is determined that the multiple files have all been read to the predetermined path, establishing the completed file.

[0011] Optionally, when it is determined that the running file processing is finished, the exclusive lock is released to process the original sequencing data based on the target file to obtain a target genome, including: when it is determined that the multiple files have been read to the predetermined path, extracting information from the multiple files based on the target file to obtain a target sequence, wherein the target sequence contains an object that needs to be corrected; performing error correction processing on the target sequence to obtain a corrected target sequence; and assembling the corrected target sequence to obtain the target genome.

[0012] According to another aspect of an embodiment of the present invention, a genome assembly device is also provided, including: a first generation module, for responding to a first command to generate a predetermined database using raw sequencing data, wherein the predetermined database includes multiple files; a second generation module, for responding to the second command, to generate multiple blocks based on the multiple files; a first acquisition module, for responding to a third command, to acquire target data in the multiple blocks, to obtain a target file based on the target data; a third generation module, for responding to a fourth command, to generate an exclusive lock, wherein the exclusive lock is used to lock other files except the currently running file; a locking module, for locking the other files when determining that the currently running file exists, so as to process the running file; a processing module, for releasing the exclusive lock when determining that the processing of the running file is completed, so as to process the raw sequencing data based on the target file to obtain a target genome.

[0013] Optionally, the first acquisition module includes: a first acquisition unit, used to respond to the third command to obtain the first repeating sequence and the second repeating sequence of each of the multiple blocks; a masking unit, used to mask predetermined objects in the first repeating sequence and the second repeating sequence to obtain masked data in each of the multiple blocks; and a comparison unit, used to compare the masked data to obtain a target file.

[0014] Optionally, the genome assembly device also includes: a judgment module, which is used to lock the other files when it is determined that the running file currently exists, so as to judge whether there is a completed file under the predetermined path before processing the running file to obtain a judgment result; an output module, which is used to output data processing completion when the judgment result indicates that the completed file exists under the predetermined path; and a second acquisition module that continues to acquire the target file when the judgment result indicates that the completed file does not exist under the predetermined path.

[0015] Optionally, the exclusive lock includes an exclusive command, and the locking module includes: a reading unit, used to use the exclusive command to read the multiple files to the predetermined path respectively; a detection unit, used to detect whether the running files exist in the predetermined path and obtain a detection result; and a locking unit, used to lock the other files when the detection result indicates that the running files exist in the predetermined path.

[0016] Optionally, the genome assembly device further includes: an establishment unit, configured to establish the completed file when it is determined that the plurality of files have all been read to the predetermined path.

[0017] Optionally, the processing module includes: an extraction unit, which is used to extract information from the multiple files based on the target file to obtain a target sequence when it is determined that the multiple files have been read to the predetermined path, wherein the target sequence contains an object that needs to be corrected; an error correction unit, which is used to perform error correction on the target sequence to obtain a corrected target sequence; and an assembly unit, which is used to assemble the corrected target sequence to obtain the target genome.

[0018] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute any of the genome assembly methods described above.

[0019] According to another aspect of an embodiment of the present invention, a processor is provided, wherein the processor is used to run a computer program, wherein the computer program executes any of the genome assembly methods described above when running.

[0020] In an embodiment of the present invention, in response to a first command, a predetermined database is generated using raw sequencing data, wherein the predetermined database includes multiple files; in response to a second command, multiple blocks are generated based on multiple files; in response to a third command, target data in multiple blocks are obtained to obtain a target file based on the target data; in response to a fourth command, an exclusive lock is generated, wherein the exclusive lock is used to lock other files except the currently running file; when it is determined that there is a currently running file, other files are locked to process the running file; when it is determined that the processing of the running file is finished, the exclusive lock is released to process the raw sequencing data based on the target file to obtain the target genome. The genome assembly method provided by the embodiment of the present invention realizes that by establishing an exclusive lock, only the first task being synchronized is synchronized in a computing node, and other parallel tasks will wait due to the existence of the exclusive lock. After the first synchronization task is completed, the other parallel tasks directly skip the purpose of the synchronization process, thereby achieving the effect of improving the genome assembly speed, thereby solving the technical problem of low assembly efficiency in the related art due to the limitation of computer cluster bandwidth and hard disk speed when assembling the genome. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 is a flow chart of a genome assembly method according to an embodiment of the present invention;

[0023] Figure 2 is a flow chart of an optional genome assembly method according to an embodiment of the present invention;

[0024] Figure 3 Schematic diagram of a genome assembly device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Example 1

[0028] According to an embodiment of the present invention, a method embodiment of a genome assembly method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] Figure 1 is a flow chart of a genome assembly method according to an embodiment of the present invention, such as Figure 1 As shown, the genome assembly method includes the following steps:

[0030] Step S101, in response to a first command, generating a predetermined database using raw sequencing data, wherein the predetermined database includes a plurality of files.

[0031] In this embodiment, the first command may be a fasta2DB command.

[0032] The Dazzler database was established for the original three-generation pacbio data using the above fasta2DB command.

[0033] Optionally, in this embodiment, the multiple files may include but are not limited to: raw_reads.db, .raw_reads.idx, .raw_reads.bps, .raw_reads.dust.anno, .raw_reads.dust.data.

[0034] Among them, the above raw_reads.db file is stored as the raw data input file and the information for segmenting the raw data; the raw_reads.idx file is a binary file, stored as the raw data index file; the .raw_reads.bps file is a binary file, stored as the compressed DNA sequencing sequence; .raw_reads.dust.anno is stored as the annotation file of each reads interval in the low-complexity interval reads; .raw_reads.dust.data is stored as the low-complexity interval reads information.

[0035] Step S102, in response to the second command, generating a plurality of blocks based on a plurality of files.

[0036] Optionally, in this embodiment, the second command may be a DBsplit command, which may be used to generate multiple blocks based on the multiple files.

[0037] Step S103 , in response to the third command, acquiring target data in a plurality of blocks to obtain a target file based on the target data.

[0038] Step S104: In response to the fourth command, an exclusive lock is generated, wherein the exclusive lock is used to lock other files except the currently running file.

[0039] Optionally, in this embodiment, the fourth command may be a flock command, through which an exclusive lock may be established to prevent confusion in synchronization of multi-task execution processes.

[0040] Among them, flock is a file lock command under Linux. When multiple processes may perform operations on the same data, these processes need to ensure that other processes are not also operating to avoid data damage during the operation. Generally speaking, the process using the flock command will use a "lock file", that is, create a file to tell other processes that it is running. If the existence of that file is detected, it is considered that there is a process operating the same data, and other processes with the same operation will not perform the same operation at the same time. The flock command is often used in Linux to manage crontab scheduled tasks to prevent conflicts with the same tasks. There are almost no instances of its application in bioinformatics analysis software.

[0041] Step S105, when it is determined that there is a file currently being run, lock other files to process the file currently being run.

[0042] Step S106, when it is determined that the processing of the running file is finished, the exclusive lock is released to process the original sequencing data based on the target file to obtain the target genome.

[0043] As can be seen from the above, in an embodiment of the present invention, a predetermined database can be generated using raw sequencing data in response to a first command, wherein the predetermined database includes multiple files; then, multiple blocks are generated based on the multiple files in response to a second command; then, target data in the multiple blocks are obtained in response to a third command to obtain a target file based on the target data; and an exclusive lock is generated in response to a fourth command, wherein the exclusive lock is used to lock other files except the currently running file; when it is determined that there is a currently running file, other files are locked to process the running file; and when it is determined that the processing of the running file is finished, the exclusive lock is released to process the raw sequencing data based on the target file to obtain the target genome, thereby achieving the goal of only performing synchronization operations on the first task being synchronized on a computing node by establishing an exclusive lock, and other parallel tasks will wait due to the existence of the exclusive lock. After the first synchronization task is completed, other parallel tasks directly skip the synchronization process, thereby achieving the effect of improving the genome assembly speed.

[0044] Therefore, the genome assembly method provided by the embodiment of the present invention solves the technical problem in the related art that when assembling genomes, the assembly efficiency is relatively low due to the limitations of computer cluster bandwidth and hard disk speed.

[0045] As an optional embodiment, in response to a third command, target data within multiple blocks are obtained to obtain a target file based on the target data, including: in response to the third command, a first repeating sequence and a second repeating sequence are obtained for each of the multiple blocks; predetermined objects in the first repeating sequence and the second repeating sequence are masked to obtain masked data in each of the multiple blocks; and the masked data are compared to obtain the target file.

[0046] In this embodiment, the HPC.TANmask and HPC.REPmask commands can be used to calculate the tandem repeat sequences and general repeat sequences of each block, and the reads regions of these sequences can be masked. The data in each block can be aligned using the daligner program to obtain the aligned las file (ie, the target file).

[0047] The first repeating sequence may be a tandem repeating sequence, and the second repeating sequence may be a general repeating sequence. The repeating tandem sequence here refers to a sequence in which the repeating units are connected end to end and arranged in a string. The general repeating sequence refers to a common repeating sequence other than a non-tandem repeating sequence.

[0048] The above las file is a file that stores the comparison result information.

[0049] As an optional embodiment, when it is determined that there is a running file currently, other files are locked to process the running file before the running file is processed. The genome assembly method also includes: judging whether there is a completed file under the predetermined path to obtain a judgment result; when the judgment result indicates that there is a completed file under the predetermined path, outputting data processing completion; when the judgment result indicates that there is no completed file under the predetermined path, continuing to obtain the target file.

[0050] In this embodiment, it can be determined whether the cp.done file is stored in the computer memory directory / dev / shm; if it exists, "db ok" is output; if it does not exist, the third command is executed in response to obtain the first repeating sequence and the second repeating sequence of each block in the multiple blocks and other operations.

[0051] As an optional embodiment, the exclusive lock includes an exclusive command, and when it is determined that there is a running file currently, other files are locked, including: using the exclusive command to read multiple files to a predetermined path respectively; detecting whether there is a running file in the predetermined path and obtaining a detection result; when the detection result indicates that there is a running file in the predetermined path, other files are locked.

[0052] In this embodiment, an exclusive command (eg, rsync command) may be used to synchronize the raw_reads.db file, the .raw_reads.idx file, the .raw_reads.bps file, the .raw_reads.dust.anno file, and the .raw_reads.dust.data file to the computer memory directory / dev / shm.

[0053] As an optional embodiment, the genome assembly method may further include: when it is determined that multiple files have been read to a predetermined path, creating a completed file.

[0054] In this embodiment, the touch command may be used to create a cp.done file (ie, a completed file) in the computer memory directory / dev / shm.

[0055] As an optional embodiment, when it is determined that the running file processing is finished, the exclusive lock is released to process the original sequencing data based on the target file to obtain the target genome, including: when it is determined that multiple files have been read to the predetermined path, information of the multiple files is extracted based on the target file to obtain a target sequence, wherein the target sequence contains an object that needs to be corrected; error correction is performed on the target sequence to obtain a corrected target sequence; and the corrected target sequence is assembled to obtain a target genome.

[0056] In this embodiment, each block uses consensus_task.py to extract the sequence that needs to be corrected from the database that has been synchronized to the memory according to the aligned las file, and stores it as a fasta file. Then, consensus.py is used to correct the fasta file in each block to obtain the corrected fasta file; and then the corrected fasta files of each block are assembled into one fasta file (i.e., the target genome).

[0057] The consensus_task.py is a script for executing the sequence steps of extracting and correcting errors, and consensus.py is a script for executing the error correction steps.

[0058] The preferred embodiments of the genome assembly method provided by the embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0059] Figure 2 is a flow chart of an optional genome assembly method according to an embodiment of the present invention, such as Figure 2 As shown, first, a Dazzler database can be established, and then the DBsplit command is used to split the Dazzler database established above; the HPC.TANmask and HPC.REPmask commands are used to mask the repeated sequence regions of the split files, and the daligner command is used to align them to obtain the Las file; the flock command is used to establish an exclusive lock in the computer memory directory / dev / shm, and it is determined whether there is a cp.done file in the memory directory / dev / shm. If so, the "db ok"; otherwise, multiple files (raw_reads.db file, .raw_reads.idx file, .raw_reads.bps file, .raw_reads.dust.anno file, .raw_reads.dust.data file) are synchronized to dev / shm; the cp.done file is created through the touch command, and the sequence that needs to be corrected is extracted using the consensus_task.py command, and the sequence that needs to be corrected is corrected through the consensus.py command to obtain the fasta file; finally, the fasta files after all blocks are assembled.

[0060] In the above embodiment, by transferring the database stored in the hard disk to the memory and establishing an exclusive lock, multiple parallel tasks and multiple computing nodes are prevented from being synchronized chaotically, which greatly reduces the time for assembling a large genome and improves the efficiency of genome assembly.

[0061] The genome assembly method provided by the embodiments of the present invention is described in detail below with reference to two specific embodiments.

[0062] For example, for the pb-assembly assembly test of about 4G tobacco, the modified scheme mainly includes the following steps: (1) Use the fasta2DB command to establish a Dazzler database for the three generations of pacbio data of tobacco. Generate five files: raw_reads.db, .raw_reads.idx, .raw_reads.bps, .raw_reads.dust.anno, and .raw_reads.dust.data, and calculate the length threshold as the seed sequence. (2) Use the DBsplit command to generate multiple blocks from the tobacco database file. (3) Use the HPC.TANmask and HPC.REPmask commands to calculate the tandem repeat sequences and general repeat sequences of each block, and mask the reads regions of these sequences. (4) The data in each block is aligned using the daligner program to obtain the aligned las file. (5) Use the flock command to establish an exclusive lock on the computing node. (6) The computing node determines whether the cp.done file is stored in the computer memory directory / dev / shm. If it exists, it outputs "db ok"; if it does not exist, execute steps 3-8. (7) Use the rsync command to synchronize the tobacco's .raw_reads.db file to the computer memory directory / dev / shm. (8) Use the rsync command to synchronize the tobacco's .raw_reads.idx file to the computer memory directory / dev / shm. (9) Use the rsync command to synchronize the tobacco's .raw_reads.bps file to the computer memory directory / dev / shm. (10) Use the rsync command to synchronize the tobacco's .raw_reads.dust.anno file to the computer memory directory / dev / shm. (11) Use the rsync command to synchronize the tobacco's .raw_reads.dust.data file to the computer memory directory / dev / shm. (12) Use the touch command to create the cp.done file under the computer memory directory / dev / shm. (13) For each block, based on the aligned las file, use consensus_task.py to extract the sequence that needs to be corrected from the database that has been synchronized to the memory and store it as a fasta file. (14) Use consensus.py to correct the fasta file in each block to obtain the corrected fasta file. (15) The corrected fasta file of each block is subsequently assembled to obtain the assembled tobacco genome file.

[0063] The test comparison results of tobacco are as follows:

[0064] Table 1

[0065] Default pb-assembly Modified pb-assembly time 28 days 5 days

[0066] Through the above comparison results, when conducting pb-assembly assembly test on tobacco of about 4G, the assembly efficiency was greatly improved by using the modified pb-assembly.

[0067] For another example, for the pb-assembly test of the 13G genome of the brittle star, the consensus steps are mainly modified as follows: (1) Use the fasta2DB command to establish a Dazzler database for the third-generation pacbio data of the brittle star. Generate five files: raw_reads.db, .raw_reads.idx, .raw_reads.bps, .raw_reads.dust.anno, and .raw_reads.dust.data, and calculate the length threshold as the seed sequence. (2) Use the DBsplit command to generate multiple blocks from the brittle star database file. (3) Use the HPC.TANmask and HPC.REPmask commands to calculate the tandem repeat sequences and general repeat sequences of each block, and mask the reads regions of these sequences. (4) Use the daligner program to align the data in each block to obtain the aligned las file. (5) Use the flock command to establish an exclusive lock on the computing node. (6) The computing node determines whether the cp.done file is stored in the computer memory directory / dev / shm. If it exists, it outputs "db ok"; if it does not exist, it executes steps 3-8. (7) Use the rsync command to synchronize the .raw_reads.db file of the brittle star to the computer memory directory / dev / shm. (8) Use the rsync command to synchronize the .raw_reads.idx file of the brittle star to the computer memory directory / dev / shm. (9) Use the rsync command to synchronize the .raw_reads.bps file of the brittle star to the computer memory directory / dev / shm. (10) Use the rsync command to synchronize the .raw_reads.dust.anno file of the brittle star to the computer memory directory / dev / shm. (11) Use the rsync command to synchronize the .raw_reads.dust.data file of the brittle star to the computer memory directory / dev / shm. (12) Use the touch command to create the cp.done file in the computer memory directory / dev / shm. (13) For each block, based on the aligned las file, use consensus_task.py to extract the sequence that needs to be corrected from the database that has been synchronized to the memory and store it as a fasta file. (14) Use consensus.py to correct the fasta file in each block to obtain the corrected fasta file. (15) The corrected fasta file of each block is subsequently assembled to obtain the assembled genome file of the brittle star.

[0068] The results of the comparison of brittle star tests are as follows:

[0069] Table 2

[0070] Default pb-assembly Modified pb-assembly time Not completed within 30 days 13 days

[0071] Based on the above comparison results, when the pb-assembly assembly test was performed on the 13G brittle star genome, the use of the modified pb-assembly also greatly improved the assembly efficiency.

[0072] As can be seen from the above, the genome assembly method provided by the embodiment of the present invention uses an exclusive lock to write the database into the cluster computer's own memory path, and can also use an exclusive lock to write the database into an SSD high-speed hard disk. The method of writing into the memory is much faster than writing into an SSD high-speed hard disk. In addition, by writing the database into the memory and then performing the consensus step, the pb-assembly assembly speed is significantly improved.

[0073] Example 2

[0074] According to another aspect of the embodiments of the present invention, a genome assembly device is also provided. Figure 3 Schematic diagram of a genome assembly device according to an embodiment of the present invention, Figure 3 As shown, the genome assembly device includes: a first generation module 31, a second generation module 32, a first acquisition module 33, a third generation module 34, a locking module 35 and a processing module 36. The genome assembly device is described in detail below.

[0075] The first generating module 31 is used to generate a predetermined database using the original sequencing data in response to a first command, wherein the predetermined database includes a plurality of files.

[0076] The second generating module 32 is used to generate a plurality of blocks based on a plurality of files in response to a second command.

[0077] The first acquisition module 33 is used to acquire target data in a plurality of blocks in response to the third command, so as to obtain a target file based on the target data.

[0078] The third generating module 34 is used to generate an exclusive lock in response to the fourth command, wherein the exclusive lock is used to lock other files except the currently running file.

[0079] The locking module 35 is used to lock other files when it is determined that there is a file currently being run, so as to process the file being run.

[0080] The processing module 36 is used to release the exclusive lock when it is determined that the processing of the running file is completed, so as to process the original sequencing data based on the target file to obtain the target genome.

[0081] It should be noted that the first generation module 31, the second generation module 32, the first acquisition module 33, the third generation module 34, the locking module 35 and the processing module 36 correspond to steps S101 to S106 in Embodiment 1, and the examples and application scenarios implemented by the above units and the corresponding steps are the same, but are not limited to the contents disclosed in the above Embodiment 1. It should be noted that the above units, as part of the device, can be executed in a computer system such as a set of computer executable instructions.

[0082] As can be seen from the above, in the above embodiments of the present application, the first generation module can be used to respond to the first command to generate a predetermined database using the original sequencing data, wherein the predetermined database includes multiple files; then the second generation module is used to respond to the second command to generate multiple blocks based on the multiple files; then the first acquisition module is used to respond to the third command to acquire the target data in the multiple blocks to obtain the target file based on the target data; the third generation module is then used to respond to the fourth command to generate an exclusive lock, wherein the exclusive lock is used to lock other files except the currently running file; and the locking module is used to lock other files when it is determined that there is a currently running file to process the running file; and the processing module is used to release the exclusive lock when it is determined that the processing of the running file is completed to process the original sequencing data based on the target file to obtain the target genome. The genome assembly device provided by the embodiment of the present invention realizes the synchronization operation only for the first task being synchronized in a computing node by establishing an exclusive lock. Other parallel tasks will wait due to the existence of the exclusive lock. After the first synchronization task is completed, other parallel tasks directly skip the synchronization process, thereby achieving the effect of improving the genome assembly speed and solving the technical problem in the related technology that when assembling genomes, the assembly efficiency is relatively low due to the limitation of computer cluster bandwidth and hard disk speed.

[0083] As an optional embodiment, the first acquisition module includes: a first acquisition unit, used to respond to a third command, to obtain a first repeating sequence and a second repeating sequence of each of a plurality of blocks; a masking unit, used to mask predetermined objects in the first repeating sequence and the second repeating sequence to obtain masked data in each of the plurality of blocks; and a comparison unit, used to compare the masked data to obtain a target file.

[0084] As an optional embodiment, the genome assembly device also includes: a judgment module, which is used to lock other files when it is determined that there is a running file currently, so as to determine whether there is a completed file under the predetermined path before processing the running file, and obtain a judgment result; an output module, which is used to output data processing completion when the judgment result indicates that there is a completed file under the predetermined path; and a second acquisition module continues to acquire the target file when the judgment result indicates that there is no completed file under the predetermined path.

[0085] As an optional embodiment, the exclusive lock includes an exclusive command, and a locking module, including: a reading unit, used to use the exclusive command to read multiple files to a predetermined path respectively; a detection unit, used to detect whether there are running files in the predetermined path and obtain a detection result; a locking unit, used to lock other files when the detection result indicates that there are currently running files in the predetermined path.

[0086] As an optional embodiment, the genome assembly device also includes: a creation unit, which is used to create a completed file when it is determined that multiple files have been read to a predetermined path.

[0087] As an optional embodiment, the processing module includes: an extraction unit, which is used to extract information from multiple files based on the target file to obtain a target sequence when it is determined that multiple files have been read to a predetermined path, wherein the target sequence contains an object that needs to be corrected; an error correction unit, which is used to perform error correction on the target sequence to obtain a target sequence after error correction; and an assembly unit, which is used to assemble the target sequence after error correction to obtain a target genome.

[0088] Example 3

[0089] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned genome assembly methods.

[0090] Example 4

[0091] According to another aspect of an embodiment of the present invention, a processor is provided, and the processor is used to run a computer program, wherein the computer program executes any of the above-mentioned genome assembly methods when running.

[0092] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0093] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0095] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0096] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0097] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0098] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for assembling a genome, characterized in that: include: In response to a first command, generating a predetermined database using the raw sequencing data, wherein the predetermined database includes a plurality of files; In response to a second command, generating a plurality of blocks based on the plurality of files; In response to a third command, acquiring target data in the plurality of blocks to obtain a target file based on the target data; In response to the fourth command, an exclusive lock is generated, wherein the exclusive lock is used to lock other files except the currently running file; When it is determined that the file being run currently exists, locking the other files to process the file being run; When it is determined that the processing of the running file is finished, releasing the exclusive lock to process the raw sequencing data based on the target file to obtain a target genome; In response to the third command, acquiring target data in the plurality of blocks to obtain a target file based on the target data includes: In response to the third command, obtaining a first repeating sequence and a second repeating sequence of each block in the plurality of blocks; Masking predetermined objects in the first repeating sequence and the second repeating sequence to obtain masked data in each block of the multiple blocks; Comparing the shielded data to obtain a target file; When it is determined that the running file currently exists, before locking the other files to process the running file, the method further includes: Determine whether there is a completed file under the predetermined path and obtain a determination result; When the judgment result indicates that the completed file exists in the predetermined path, the output data processing is completed; When the judgment result indicates that the completed file does not exist in the predetermined path, continue to obtain the target file.

2. The method according to claim 1, characterized in that: The exclusive lock includes an exclusive command, and when it is determined that the running file currently exists, the other files are locked, including: Using the exclusive command to read the multiple files into the predetermined path respectively; Detecting whether the running file exists in the predetermined path and obtaining a detection result; When the detection result indicates that the running file exists in the predetermined path, the other files are locked.

3. The method according to claim 2, characterized in that The method further comprises: When it is determined that the plurality of files have all been read to the predetermined path, the completed file is created.

4. The method according to claim 1, characterized in that When it is determined that the processing of the running file is finished, releasing the exclusive lock to process the raw sequencing data based on the target file to obtain a target genome, including: When it is determined that the multiple files have all been read to the predetermined path, extracting information from the multiple files based on the target file to obtain a target sequence, wherein the target sequence includes an object that needs to be corrected; performing error correction processing on the target sequence to obtain a target sequence after error correction; The error-corrected target sequence is assembled to obtain the target genome.

5. A genome assembly device, characterized in that: include: A first generating module, configured to generate a predetermined database using the original sequencing data in response to a first command, wherein the predetermined database includes a plurality of files; A second generating module, configured to generate a plurality of blocks based on the plurality of files in response to a second command; a first acquisition module, configured to acquire target data in the plurality of blocks in response to a third command, so as to obtain a target file based on the target data; A third generating module, configured to generate an exclusive lock in response to a fourth command, wherein the exclusive lock is used to lock other files except the currently running file; A locking module, used for locking the other files when it is determined that the file being run currently exists, so as to process the file being run; A processing module, configured to release the exclusive lock when determining that the processing of the running file is finished, so as to process the raw sequencing data based on the target file to obtain a target genome; The first acquisition module includes: A first acquisition unit, configured to acquire, in response to the third command, a first repeating sequence and a second repeating sequence of each of the plurality of blocks; a masking unit, configured to mask predetermined objects in the first repeating sequence and the second repeating sequence to obtain masked data in each of the plurality of blocks; A comparison unit, used for comparing the masked data to obtain a target file; The genome assembly device also includes: a judgment module, which is used to lock other files when it is determined that there is a running file currently, so as to judge whether there is a completed file under the predetermined path before processing the running file, and obtain a judgment result; an output module, which is used to output data processing completion when the judgment result indicates that there is a completed file under the predetermined path; and a second acquisition module continues to acquire the target file when the judgment result indicates that there is no completed file under the predetermined path.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute the genome assembly method described in any one of claims 1 to 4.

7. A processor, characterized in that: The processor is used to run a computer program, wherein the computer program, when running, executes the genome assembly method described in any one of claims 1 to 4.

Images