Computer program for index creation, computer program for mapping, computer device for index creation, computer device for mapping, and data structure of layered index

The hierarchical index creation method in genome analysis reduces memory access time and enhances processing speed by dividing the suffix tree into upper and lower trees, addressing the limitations of existing methods like BWA-MEM in genome analysis.

WO2024247786A1PCT designated stage expired Publication Date: 2024-12-05ADVANCED ACCELERATING SYST CO LTD +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/018489
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-26
Filing Date
2024-05-20
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Current genome analysis methods, such as BWA-MEM, require significant memory resources and suffer from limited processing speed due to frequent random access to main memory, leading to lengthy analysis times even with high-performance computers.

Method used

A computer program and device that create a hierarchical index by omitting lower layers in a suffix tree, dividing it into upper and lower trees, allowing for efficient mapping processing by storing lower tree data in cache memory, reducing memory access time.

Benefits of technology

This approach significantly reduces memory access time and enhances processing speed for genome analysis, enabling faster and more efficient identification of query string positions within the reference sequence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024018489_05122024_PF_FP_ABST
    Figure JP2024018489_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention is a computer program for index creation that realizes creation of an index to be used when identifying a partial character string in a reference character string on the basis of a query character string. The computer program for index creation includes a step for omitting layer data below a predetermined value by setting, to the predetermined value, a number of layers below the highest layer in a suffix tree obtained by layering a suffix array of the reference character string. Thus, given query data is analyzed quickly and / or efficiently using the reference data.
Need to check novelty before this filing date? Find Prior Art

Description

Computer program for creating indexes, computer program for mapping, computer device for creating indexes, computer device for mapping, and data structure of hierarchical indexes

[0001] The present invention relates to a computer program for creating an index suitable for genome analysis, a computer program for mapping, an apparatus for executing these programs, and a data structure of a hierarchical index used in these programs.

[0002] The human genome is a set of genetic information that a person possesses, and the material that carries this information is DNA (deoxyribonucleic acid), which is a string of approximately 3 billion pairs of bases. The bases are adenine (A), guanine (G), cytosine (C), and thymine (T). In other words, a person's genetic information is determined by the arrangement (sequence) of these bases.

[0003] A device called a sequencer is used to read the human genome. The sequencer reads the sample human genome, cuts it into a predetermined upper limit (about a few hundred base pairs), amplifies it, and outputs a huge data sequence consisting of hundreds of millions of pieces of genetic information. Current sequencers can read one human genome in about an hour.

[0004] The individual data fragments read by the sequencer are compared with a human genome reference sequence, which is defined as a standard human genome sequence, to reconstruct and analyze the original length of the human genome sequence. For example, the position of each data fragment is determined (mapped) in comparison with the human genome reference sequence. After mapping is complete, analysis is performed to determine what mutations are present. Since the data fragments read from the sequencer typically contain errors, statistical processing is performed using, for example, 30 times the amount of redundant data per sample (the equivalent of one person's human genome sequence) to reduce the errors. Therefore, analyzing the human genome sequence requires a huge amount of computation, and therefore, high-performance computers such as supercomputers, cluster computers, and FPGA (Field-Programmable Gate Array)-based computers are typically used.

[0005] For example, a programming tool such as BWA (Burrow-Wheeler Aligner) is used to map data fragments. BWA consists of three algorithms: BWA-backtrack, BWA-SW, and BWA-MEM. Of these, BWA-MEM is widely used as a high-speed algorithm that supports indels (insertion deletions). BWA-MEM is an algorithm that creates an index using a suffix sequence for the portion of a query string based on the read data fragment that appears repeatedly in the human genome reference sequence, and performs mapping (Heng Li, "Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM", May 26, 2013, arXiv:1303.3997 (q-bio.GN)).

[0006] The mapped data fragments are compared with the partial sequences at the same positions in the human genome reference sequence to analyze the presence of mutations. For example, an alignment algorithm is used to compare the partial sequences of the human genome reference sequence with the data fragments. The alignment algorithm uses dynamic programming to calculate the degree of variation (similarity) for each element of the sequence, known as an alignment table, and identifies similar character strings.

[0007] The conventional BWA-MEM described above is widely used as an algorithm that allows gaps and enables high-speed mapping, but has the problem of requiring a large amount of memory resources because the size of the index created from the reference sequence is very large. For example, in the case of analyzing the human genome, the amount of data for the index of the human genome reference sequence is enormous. This index must be stored in external memory (main memory) because it cannot be accommodated at once in a cache memory that can be accessed more quickly. As a result, external memory must be accessed a number of times corresponding to the number of characters (i.e., the number of bases) in the query string to be searched, which creates the problem that the processor's processing speed is limited by the time it takes to access the memory.

[0008] More specifically, in BWA-MEM, in order to identify the positions where the query strings read in fragmented form by the sequencer appear in the human genome reference sequence, random access to the main memory is performed a number of times equal to the total number of characters in the query strings. Here, the latency for one random access to the main memory (the time from when the access occurs until the data is actually obtained) is approximately 1 μs (0.000001 seconds) per bank on current computers. Therefore, in the case of analyzing a 3 billion base pair human genome, assuming redundancy of 30, the total access time T for the total number of characters is: T = 3,000,000,000 × 0.000001 × 30 = 90,000 (seconds). In other words, in analyzing a human genome, random access to the main memory alone would take approximately 90,000 seconds (approximately 25 hours). Therefore, even if a computer with a high-performance processor is used, the memory access time is a constraint, limiting the ability to reduce mapping time.

[0009] One object of the present invention is to provide a technique for creating a hierarchical index based on a reference string, which is suitable for a mapping process. Another object of the present invention is to provide a technique for performing a mapping process quickly and / or efficiently, for example, to identify the position in a reference sequence where a data fragment (query string) read by a sequencer or the like appears.

[0010] The present invention, which aims to solve the above problems, comprises the following invention-specifying matters and technical features.

[0011] In other words, the present invention, which relates to the above-mentioned problem, is a computer program for creating an index that enables a computing device to create an index to be used when identifying a substring in a reference string based on a query string, and is characterized by having a step of setting the number of levels from the top level to a predetermined value in a suffix tree in which the suffix array of the reference string is hierarchically organized, thereby omitting hierarchical data below the predetermined value.

[0012] In relation to the index creation computer program, the predetermined value for the number of hierarchical levels may be set to 1000 or less.

[0013] In relation to the index creation computer program, the predetermined value for the number of levels may be set to be equal to or greater than the number of characters in the query string.

[0014] The computer program for creating an index may further include a step of dividing the suffix tree at an intermediate level to generate a supertree that includes a level higher than the intermediate level, and a plurality of subtrees (hereinafter referred to as a group of subtrees) that include a level lower than the intermediate level and are connected to the lower side of the supertree.

[0015] In relation to the above-mentioned computer program for creating an index, the amount of data in the subtree may be set to a value that can be stored in a cache memory in the computing device.

[0016] In relation to the above-mentioned computer program for creating an index, the data volume of the subtree may be set to 10 MB or less.

[0017] The present invention, which relates to the above-mentioned problem, is a computer program for creating an index that enables a computing device to create an index to be used when identifying a substring in a reference string based on a query string, and is characterized by having a step of dividing a suffix tree, which is a hierarchical structure of a suffix array of the reference string, at an intermediate level to create a supertree that includes a level higher than the intermediate level, and a plurality of subtrees (hereinafter referred to as a group of subtrees) that include a level lower than the intermediate level and are connected to the lower side of the supertree.

[0018] The present invention, which relates to the above-mentioned problem, is a mapping computer program for causing a computing device to implement a method for identifying substrings in a reference string based on a query string, the method including, as part thereof, the above-mentioned index creation computer program, and further characterized in that the method includes mapping the query string to the reference string by referring to the hierarchical index in order to identify substrings in the reference string that match at least a portion of the query string.

[0019] In relation to the above-mentioned problem, the present invention is a mapping computer program for causing a computing device to implement a method for identifying substrings in a reference string based on a query string, wherein a hierarchical index generated based on the reference string is divided at an intermediate level to have a supertree that includes a level higher than the intermediate level, and a plurality of subtrees (hereinafter referred to as a group of subtrees) that include a level lower than the intermediate level and are connected to a lower side of the supertree, the method comprising: mapping the query string to the reference string by referring to the hierarchical index in order to identify a substring in the reference string that matches at least a part of the query string; the mapping of the query string comprises: a determining step of determining to which subtree in the group of subtrees each of the query strings belongs by performing a matching process on the plurality of query strings using the supertree; and an appearance position determining step of determining an appearance position of the query string relative to the reference string by performing a matching process on the plurality of query strings that belong to a specific subtree determined in the determining step, using the specific subtree.

[0020] In relation to the mapping computer program, the step of determining the occurrence positions in the subtree may be performed in parallel.

[0021] In relation to the mapping computer program, the determining step may be performed in parallel.

[0022] In relation to the above mapping computer program, in the step of determining the occurrence position, the subtree may be expanded in a cache memory in the computing device.

[0023] In relation to the mapping computer program, the computer program may be characterized in that it includes any one of the index creation computer programs described above as a part thereof.

[0024] The present invention, which relates to the above-mentioned problem, is an index-creating computer apparatus having a computing device including a storage device, which creates an index used to identify a substring in a reference string based on a query string, wherein the computing device has a partial suffix tree generation means which reads out the reference string stored in the storage device, sets the number of levels from the top level to a predetermined value in a suffix tree in which the suffix array of the reference string is hierarchically organized, and generates a partial suffix tree in which hierarchical data below the predetermined value is omitted.

[0025] The present invention, which relates to the above-mentioned problem, is a computer apparatus for creating an index, which has a computing device including a storage device and creates an index used to identify substrings in a reference string based on a query string, and is characterized in that the computing device has a suffix tree splitting means for splitting a suffix tree, which is a hierarchical structure of the suffix array of the reference string stored in the storage device, at an intermediate level to create a supertree including a higher level and a plurality of subtrees (hereinafter referred to as a group of subtrees) connected to the lower side of the supertree.

[0026] The present invention, which relates to the above-mentioned problem, is a mapping computer apparatus that has a computing device including a storage device and identifies substrings in a reference string based on a query string, wherein the storage device stores a supertree that includes a hierarchy higher than the intermediate hierarchy, and a plurality of subtrees (hereinafter referred to as a group of subtrees) that include a hierarchy lower than the intermediate hierarchy and are connected to lower sides of the supertree, the supertree being generated by dividing a hierarchical index generated based on the reference string at an intermediate hierarchy; the computing device has mapping means that maps the query string to the reference string by referring to the supertree and the subtrees in order to identify substrings in the reference string that match at least a part of the query string; the mapping means has: determination means that performs a matching process on the plurality of query strings by referring to the supertrees stored in the storage device to determine to which subtree in the group of subtrees each of the query strings belongs; and appearance position determination means that performs a matching process on the plurality of query strings that belong to a specific subtree determined by the determination means by referring to the specific subtree stored in the storage device to determine an appearance position of the query string relative to the reference string.

[0027] The present invention, which relates to the above-mentioned problem, is a data structure of a hierarchical index of a reference string, used in a mapping computer program for enabling a computing device to implement a method for identifying substrings in a reference string based on a query string, the hierarchical index data structure being a suffix tree in which the suffix array of the reference string is hierarchically arranged, and characterized in that the number of levels from the highest level down is set to a predetermined value, and hierarchical data below the predetermined value is omitted.

[0028] In relation to the data structure of the above-mentioned hierarchical index, the hierarchical index may be characterized in that the suffix tree is divided at an intermediate level, so that the hierarchical index has a data fragment of a supertree that includes a level higher than the intermediate level, and data fragments of multiple subtrees that include a level lower than the intermediate level and are connected to the lower side of the supertree.

[0029] The present invention, which relates to the above-mentioned problem, is a data structure of a hierarchical index of a reference string, used in a mapping computer program for enabling a computing device to implement a method for identifying substrings in a reference string based on a query string, wherein the hierarchical index is divided at an intermediate level, so as to have a data fragment of a supertree that includes a level higher than the intermediate level, and data fragments of multiple subtrees that include a level lower than the intermediate level and are connected to the lower side of the supertree.

[0030] The present invention also applies to a recording medium storing the computer program. The present invention also applies to an apparatus consisting of hardware and / or firmware configured to execute the method. Furthermore, when the index creation computer device, mapping computer device, etc. are realized by multiple hardware components (or functional modules that realize specific functions) logically cooperating with each other, these devices become index creation computer systems and mapping computer systems. Furthermore, when the index creation computer device, mapping computer device, etc. are realized by processor modules, these devices become index creation processor modules and mapping processor modules. In other words, it does not matter whether each device or functional module is contained within a single housing or whether it is realized by a single functional component.

[0031] In this specification, a means or unit does not simply mean a physical means, but also includes cases where the functions of the means or unit are realized by software. Also, the functions of one means or unit may be realized by two or more physical means, and the functions of two or more means or units may be realized by one physical means.

[0032] According to one aspect of the present invention, a hierarchical index based on reference strings can be created that is suitable for mapping operations, and this hierarchical index allows for fast and / or efficient mapping operations.

[0033] Furthermore, according to the present invention, analysis using a human genome reference sequence based on data fragments read by a sequencer can be performed quickly and / or efficiently.

[0034] Other technical features, objects, and operational effects or advantages of the present invention will become apparent from the following embodiments described with reference to the accompanying drawings. The effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be present.

[0035] FIG. 1 is a block diagram showing an example of a schematic configuration of a computer system according to an embodiment of the present invention. FIG. 2 is a flowchart outlining an example of approximate string matching processing performed by the computer system. FIG. 3 is a flowchart illustrating an example of index creation processing performed by the computer system. FIG. 4 is a diagram illustrating an example of a cyclic array generated from a reference string used in the computer system. FIG. 5 is a diagram illustrating an example of the cyclic array after sorting. FIG. 6 is a diagram illustrating an example of the structure of a suffix tree generated based on the sorted cyclic array. FIG. 7 is a diagram illustrating a process for generating a partial suffix tree using the suffix tree. FIG. 8 is a diagram illustrating an example of the structure of a partial suffix tree generated from the suffix tree. FIG. 9(A) is a diagram illustrating a process for splitting the partial suffix tree, and FIG. 9(B) is a diagram illustrating an example of a split supertree. FIG. 10 is a flowchart illustrating an example of mapping processing performed by the computer system. FIG. 11 is a diagram illustrating an example of a query string used in the computer system. FIGS. 12(A) to 12(D) are diagrams illustrating a mapping processing performed by the computer system using a subtree. Figures 13(A) to 13(D) are diagrams explaining the mapping process by the computer system using a subtree. Figures 14(A) to 14(D) are diagrams explaining the mapping process by the computer system using a subtree. Figure 15 is a diagram showing a modified example of the partial suffix tree. Figure 16 is a diagram explaining a modified example when dividing the partial suffix tree. Figure 17 is a diagram showing an example of the hardware configuration of the computing device. Figure 18 is a diagram showing an example of a processor module in the hardware configuration of the computing device. Figure 19 is a block diagram showing an example of the functional configuration of the computing device. Figure 20 is a flowchart explaining a modified example of the mapping process by the computer system.

[0036] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or applications of techniques not explicitly described below. The present invention can be implemented in various modifications (e.g., by combining the various embodiments) without departing from the spirit of the present invention. Furthermore, in the following description of the drawings, identical or similar parts are denoted by identical or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. Parts in the drawings may have different dimensional relationships or ratios.

[0037] FIG. 1 is a block diagram illustrating an example of a schematic configuration of a computer system according to an embodiment of the present invention. As shown in the figure, the computer system 1 includes, for example, a host computer 10, multiple subordinate computers 20(1) to 20(n), and a database 30. The host computer 10 and the subordinate computers 20(1) to 20(n) are communicatively connected via a predetermined interface. In this disclosure, the computer system 1 is configured as a centralized computer system in which the host computer 10 comprehensively controls the multiple subordinate computers 20(1) to 20(n). However, the present disclosure is not limited to this, and the computer system 1 may also be configured as, for example, a distributed computer system. In a distributed computer system, each computer can operate cooperatively through parallel distributed processing, but for a specific process, only one representative computer may execute that process.

[0038] The computing hardware common to the upper computer 10 and the lower computer 20 is a computing device 100, as exemplified in Fig. 17. The computing device 100 includes a memory module 102 serving as a main storage device such as a DRAM, a processor module 104 serving as a central computer, a chipset 106 serving as a bridge between the processor module 104 and external devices, an internal storage device 108 such as an HDD or SSD, an I / O controller 110, an output interface 112, an input / output interface 114, a communication interface 116, and an external storage device 118 such as an external HDD.

[0039] 18, the processor module 104 includes multiple cores A, B, C, and D that execute independent computational processes, first cache memories (L1 cache memories) L1a, L1b, L1c, and L1d and second cache memories (L2 cache memories) L2a, L2b, L2c, and L2d that function as internal storage functions, respectively associated with the cores A, B, C, and D, and a third cache memory (L3 cache memory) L3n that functions as an internal storage function shared by all the cores A, B, C, and D. The access latency of the cores A, B, C, and D is shortest for the L1 cache memories L1a, L1b, L1c, and L1d, followed by the L2 cache memories L2a, L2b, L2c, and L2d, and longest for the L3 cache memory L3n. Meanwhile, the access latency of the L1, L2, and L3 cache memories is significantly shorter than that of the memory module (main memory) 102. These structures allow the multiple cores A, B, C, and D in the processor module 104 to process tasks in parallel. For example, each of the multiple cores A, B, C, and D can perform analysis processing by comparing a given query string with a reference string. While the cache memory is divided into three levels here, the present invention is not limited to this configuration and may be divided into one, two, or four or more levels. Hereinafter, the first cache memory (L1 cache memory) L1a, L1b, L1c, and L1d, the second cache memory (L2 cache memory) L2a, L2b, L2c, and L2d, and the third cache memory (L3 cache memory) L3n may be collectively referred to as "cache memory." In the present invention, the number of cores in the processor module 104 is not particularly limited. The number of cores can be selected from a few to tens of thousands.

[0040] 17 and 18 illustrate examples of general-purpose hardware structures, but the present invention is not limited to these. Furthermore, all or part of these hardware structures may employ individually customized (programmed) hardware using an FPGA or the like. Furthermore, hereinafter, the memory module 102, the internal storage device 108, the external storage device 118, the first cache memories (L1 cache memories) L1a, L1b, L1c, and L1d, the second cache memories (L2 cache memories) L2a, L2b, L2c, and L2d, and the third cache memory (L3 cache memory) L3n, which are hardware capable of storing data, may be collectively referred to as a "storage device."

[0041] 1 , in the present disclosure, the lower-level computers 20 process tasks in parallel under the control of the upper-level computer 10. For example, each of the multiple lower-level computers 20 performs analysis processing by comparing a given query string with a reference string. Hereinafter, the lower-level computers 20(1) to 20(n) may be simply referred to as "lower-level computers 20" unless there is a need to particularly distinguish between them.

[0042] The database 30 stores various data, such as reference strings, under the control of the host computer 10. As an example, the database 30 stores a human genome reference sequence. The human genome reference sequence is data indicating a base pair sequence defined as a standard human genome sequence. The database 30 also stores an index created based on the reference string. The index is a type of data array structure used to search for the reference string. Note that in the figure, the database 30 is configured to be connected so as to be accessible only to the host computer 10, but this is not limited thereto; the database 30 may also be connected so as to be accessible to the lower-level computer 20. The lower-level computer 20 may also have a similar database. Here, an example is shown in which a human genome reference sequence is used, but a metagenomic reference sequence that compiles multiple organisms (animals), plants, etc., may also be used.

[0043] 2 is a flowchart outlining an example of approximate string matching processing by a computer system 1 according to an embodiment of the present invention. This processing is realized, for example, by the host computer 10 and the multiple lower-level computers 20 executing an approximate string matching program in the processor module 102 in cooperation with other hardware components. As a result, the computer system 1, the host computer 10, or the multiple lower-level computers 20 function as an approximate string matching computer system 200 shown in FIG. 19. The approximate string matching program has multiple processing blocks (subroutines). As a result, the approximate string matching computer system 200, more specifically, includes an index creation computer system 201, a mapping computer system 202, a string pair creation computer system 203, and an approximate string derivation computer system 204.

[0044] As shown in FIG. 2 , first, the host computer 10 reads the reference string stored in the database 30 and creates an index based on the read reference string (index creation process: S201). Specifically, the host computer 10 creates an index having a predetermined data array structure by expanding and / or sorting substrings of the reference string according to predetermined conditions. The data array structure is a hierarchical tree structure. In this disclosure, such an index is referred to as a hierarchical index. The host computer 10 stores the created index based on the reference string in the database 30. Details of the process for creating a hierarchical index based on the reference string will be described later. Note that if a hierarchical index based on the reference string has already been created and stored in the database 30, the index creation process S201 can be omitted. This allows the computer system 1 to function as an index creation computer system 201.

[0045] Next, the host computer 10 receives a query string from an external device (not shown), references the hierarchical index based on the received query string, and performs a mapping process of the query string to the reference string (mapping process: S202). In one example, the external device is a sequencer that reads the human genome, and the query string is a data fragment of, for example, approximately 150 to 200 base pairs output from the sequencer. The query string may be temporarily stored in the database 30. The number of characters in the query string is preferably 50 or more, more preferably 100 or more. On the other hand, the number of characters in the query string is preferably 1,000 or less, desirably 300 or less, and more preferably 200 or less. In the present disclosure, the lower-level computer 20 also performs mapping by reference to the hierarchical index based on the assigned query string under the control of the higher-level computer 10. This allows the computer system 1 to function as a mapping computer system 202.

[0046] Mapping is a process for identifying substrings (matching strings) in the reference string that match at least a portion of the query string. That is, mapping identifies the starting position and length (number of characters) of substrings in the reference string that match at least a portion of the query string. In the mapping process according to the present disclosure, the starting position of the query string in the reference string is identified by searching or traversing a hierarchical index for the query string. To speed up processing, the created hierarchical index is preferably loaded into a high-speed memory such as a cache memory in memory module 102 or processor module 104.

[0047] In the above example, the upper computer 10 and the lower computer 20 each perform mapping according to the assigned query string, but this is not limited to this. For example, only the upper computer 10 may perform a search by referring to a hierarchical index based on the assigned query string.

[0048] Next, the host computer 10 and the lower computer 20 each create a string pair for approximate string matching (described later) (string pair creation process: S203). A string pair consists of a matched string in the reference string and a match string in the query string, including a matching string identified by mapping. This allows the computer system 1 to function as a string pair creation computer system 203.

[0049] Specifically, the host computer 10 and the lower computer 20 create a matched string by adding a corresponding character string of a predetermined length in the reference string to the beginning and end of the matching string identified by mapping. In other words, the matched string is a character string in the vicinity of the matching string identified in the reference string, including the matching string. For example, if the reference string is "CCGATCTGTATACCCTACGA" and the matching string is "TACC," the string to be matched is "TATACCCT," which is obtained by adding two characters to the beginning and two characters to the end. In the case of human genome analysis, the length of bases added to the beginning and end of the matching string for the reference string may be, for example, about 50 bases each.

[0050] Furthermore, the host computer 10 creates a matching string by adding a corresponding character string of a predetermined length in the query string to the end of the matching string. In the case of analyzing the human genome, the length of the character string added to the end of the matching string for the query string is, for example, about 50 bases.

[0051] In the present disclosure, for a reference string, a character string of a predetermined length is added to the beginning and end of the matching string, but this is not limited to this, and for example, a character string of a predetermined length may be added to only one of the beginning or end. Furthermore, for a query string, a character string of a predetermined length may be added to the beginning and end of the matching string, or the matching string itself may be treated as a matching string without adding any character string.

[0052] Next, the host computer 10 and the lower computer 20 derive at least one approximate string based on a string pair consisting of a matched string based on the reference string and a match string based on the query string (approximate string derivation process: S204). A predetermined alignment using dynamic programming is applied to derive the approximate string. Alignment is a technique for comparing two sequences (strings) while allowing substitution, insertion, and deletion of elements, and calculating the degree of variation (similarity) according to a defined score / penalty. Known algorithms for achieving alignment include the Smith-Waterman algorithm and the Needleman-Wunsch algorithm. Dynamic programming is a technique for calculating the optimal solution for the next stage based on the optimal solution obtained at a given stage. In other words, in calculating the degree of variation, a score is calculated for each element in an array-like alignment table according to dynamic programming, and the element with the highest score is determined, thereby deriving at least one approximate string. This allows the computer system 1 to function as a computer system 204 for approximate string derivation.

[0053] As described above, in the approximate string matching process of this embodiment, a hierarchical index based on a reference string is created, and then a matching string (and its length) is identified by searching the hierarchical index according to a given query string. An approximate string is derived by performing approximate string matching on a string pair consisting of a matched string and a match string based on the identified matching string. In other words, the approximate string matching program of this embodiment includes an index creation program based on the reference string, a mapping program that searches the hierarchical index according to the query string, a program that creates string pairs consisting of a matched string and a match string, and an alignment program that performs approximate string matching using the string pairs to derive an approximate string. Since the program that creates string pairs and the alignment program can be well-known programs, detailed description thereof will be omitted.

[0054] <Index Creation Program> Fig. 3 is a flowchart illustrating an example of index creation processing by a computer system according to this embodiment. Specifically, Fig. 3 shows details of the hierarchical index creation processing (S201) shown in Fig. 2. Fig. 19 also shows the functional block configuration of the index creation computer system 201, which includes a reference string receiving means 302, a suffix tree generating means 304, a partial suffix tree generating means 306, and a partial suffix tree dividing means 308. Each functional block will be described below with reference to the flowchart and the like.

[0055] (Reference string receiving means 302) First, as shown in FIG. 3, the host computer 10 receives a reference string for creating a hierarchical index (S302). For example, in the case of a human genome reference sequence, the host computer 10 accesses the database 30 and reads out the stored human genome reference sequence. For ease of understanding, the following description will be given assuming that the human genome reference sequence is a 20-character reference string "CCGATCTGTATACCCTACGA" consisting of four characters: "A," "C," "G," and "T." Note that the actual human genome reference sequence is 3 billion characters long.

[0056] (Suffix Tree Generation Means 304) Next, the host computer 10 generates a suffix tree for the received reference string (S304). Specifically, first, for example, "$" is added as a terminal character to the reference string. This terminal character is the smallest character not included in the reference string. The reference string, which now contains 21 characters including this terminal character, is cycled character by character to generate 21 cyclic strings (see FIG. 4). Note that in FIG. 4, the sequence number of the cyclic string is attached to the beginning (left end) of each cyclic string. Each sequence number indicates the start position of each cyclic string relative to the reference cyclic string number 0 (reference string). Note that if the actual human genome reference sequence contains 3 billion characters, the number of cyclic strings (number of sequences) will be 3 billion.

[0057] Furthermore, the host computer 10 cuts off the strings following the terminal character "$" in the cyclic string (see the shaded area in Figure 4), then sorts these cyclic strings (see Figure 5) to generate a suffix tree represented by a hierarchical tree structure. This suffix tree is shown in Figure 6. The number of levels in the suffix tree is shown at the top of Figure 6. In this embodiment, when the number of nodes in the lower levels of the tree structure reaches one, the remaining strings are grouped together at their terminal nodes (leaves). As a result, including the terminal nodes (leaves), the number of levels in the suffix tree in this embodiment is three. In the case of an actual human genome reference sequence, the number of levels is much larger (e.g., approximately 20 levels). Note that the above procedure for generating a suffix tree is a reference example, and the present invention is not limited thereto. Various known methods can be used for the program for generating a suffix tree.

[0058] (Partial Suffix Tree Generation Means 306) Next, the host computer 10 generates a partial suffix tree from the generated suffix tree (S306). Specifically, as shown in the shaded area in FIG. 7, hierarchical data below a predetermined level is forcibly cut, as shown in FIG. 7. A suffix tree whose lower levels have been cut is referred to as a partial suffix tree, and the predetermined number of levels is referred to as the cut level. In this embodiment, for convenience of explanation, the cut level is set to two, and hierarchical data below this level is cut. The partial suffix tree is shown in FIG. 8. Note that in the partial suffix tree shown in FIG. 8, if multiple characters are stored at a terminal node (leaf), the tree structure is simplified by retaining the first character (or a predetermined number of characters from the beginning) and cutting the rest of the string. A suffix tree requires information on the position and length of the matching cyclic string for each node in the tree. However, by cutting levels, this information can be replaced with the characters themselves without significantly increasing the amount of data, thereby speeding up searches.

[0059] In the partial suffix tree of Figure 8, the array element numbers of each cyclic string are attached to the right end of the terminal nodes (leaves) of the partial suffix tree. For example, in the case of the root(#)-C-G-A system, in the original suffix tree (see Figure 7), there are two nodes (branches) lower down than this, "root(#)-C-G-A-$" and "root(#)-C-G-A-T...", so to the right of the leaf of root(#)-C-G-A in the partial suffix tree, the array element number 17 of the upper (older brother) node in sorting order and the array element number 1 of the lower (younger brother) node are attached in that order. Also, for example, in the case of the root(#)-T-A-C system, since the original suffix tree (see FIG. 7) has two nodes (branches) lower down than this, "root(#)-T-A-C-C..." and "root(#)-T-A-C-G...", the upper (older brother) node, sequence number 10, and the lower (younger brother) node, sequence number 15, are assigned to the right of the leaf of root(#)-T-A-C in the partial suffix tree in that order. In these cases, this means that the occurrence positions of the identified query string are identified in two places.

[0060] In the case of an actual human genome reference sequence, the number of levels in the (entire) suffix tree is enormous. In this case, the value of the number of cut levels set to create a partial suffix tree is preferably 1,000 or less, desirably 500 or less, more preferably 300 or less, and even more preferably 200 or less. On the other hand, the value of the number of cut levels is preferably 30 or more, desirably 50 or more, and even more preferably 80 or more. Furthermore, the value of the set number of cut levels is preferably set to be equal to or greater than the number of characters in the query string. For example, if the query string is 150 base pairs, the number of cut levels is set to be equal to or greater than 150. Specifically, if the number of cut levels for the human genome reference sequence is set to 150, the (entire) suffix tree (data volume of approximately 250 GB) is shortened to a 150-level partial suffix tree (data volume of approximately 100 GB), thereby significantly reducing the amount of data.

[0061] In the simplified example of Figure 8, of the 19 terminal nodes (leaves) in the partial suffix tree, two, the "root(#)-C-G-A" system and the "root(#)-T-A-C" system, are assigned multiple sequence numbers. However, in the case of a partial suffix tree created based on the human genome reference sequence, the enormous number of levels is reduced to 150 levels, and as a result, each terminal node (leaf) is assigned multiple sequence numbers, although this number rarely exceeds three. In genome searches, there is a high probability of mutations, so narrowing down the sequence number to one often results in matches to the wrong location. For this reason, in order to improve mapping accuracy, it is desirable to cut the tree at the stage where multiple sequence numbers remain.

[0062] Furthermore, in the above embodiment, the partial suffix tree is generated through two means, the suffix tree generation means 304 and the partial suffix tree generation means 306, but the procedure for generating the partial suffix tree is not limited to this. Any process for generating a partial suffix tree using the received reference string can be employed.

[0063] (Partial Suffix Tree Splitting Means 308) Next, the host computer 10 splits the partial suffix tree at intermediate levels (intermediate nodes) to create a supertree containing the upper level and multiple subtrees (hereinafter referred to as subtrees) connected to the lower levels of the supertree (S308). The splitting condition for the intermediate nodes that allows splitting is, for example, "when tracing the intermediate nodes from the terminal node back to the upper level, the number of terminal nodes (leaves) belonging to the lower level than the intermediate node is equal to or less than a predetermined value." For example, FIG. 9A shows a case where splitting occurs when "when tracing the intermediate nodes from the terminal node back to the upper level, the number of terminal nodes (leaves) belonging to the lower level than the intermediate node is equal to or less than two." In this example, the partial suffix tree is split into one supertree containing the root (#) and 14 subtrees connected to the lower level of this supertree. Each subtree is assigned a management number. The rules for assigning this management number are arbitrary, but for example, the oldest number in the array element number belonging to the subtree (the highest number in the sort order) can be used.Specifically, the subtrees are explained in sort order as follows: the first subtree of control number 20 split at the junction of "root(#)-$", the second subtree of control number 19 split at the junction of "root(#)-A-$", the third subtree of control number 11 split at the junction of "root(#)-A-C", the fourth subtree of control number 9 split at the junction of "root(#)-A-T", the fifth subtree of control number 12 split at the junction of "root(#)-C-C-C", the sixth subtree of control number 0 split at the junction of "root(#)-C-C-G", the seventh subtree of control number 13 split at the junction of "root(#)-C-C-T", the eighth subtree of control number 17 split at the junction of "root(#)-C-G". The following subtrees are generated: the 9th subtree of control number 14 divided at the junction of "root(#)-C-T", the 10th subtree of control number 18 divided at the junction of "root(#)-G-A-$", the 11th subtree of control number 2 divided at the junction of "root(#)-G-A-T", the 10th subtree of control number 18 divided at the junction of "root(#)-G-A-$", the 11th subtree of control number 7 divided at the junction of "root(#)-G-T", the 12th subtree of control number 10 divided at the junction of "root(#)-T-A", the 13th subtree of control number 4 divided at the junction of "root(#)-T-C", and the 14th subtree of control number 6 divided at the junction of "root(#)-T-G". The amount of data pieces in each subtree of the group of subtrees is preferably set to a size that can fit in the cache memory (particularly the L1 cache memory) of the processor module 104. In other words, it is preferable to set the "number of terminal nodes (leaves) belonging to a lower layer (predetermined value)" in the splitting condition of an intermediate node for which splitting is established, taking into consideration the capacity of the cache memory.

[0064] Figure 9(B) shows the partial suffix tree with the supertree, split node (terminal node in the supertree), and subtree management number remaining. In this way, the data volume of the supertree is significantly reduced by separating the data fragments from the subtree. The supertree should preferably be small enough to fit in a memory module (main memory), and even more preferably, small enough to fit in cache memory. As a result, hierarchical index data is completed, in which the partial suffix tree is divided into supertree data fragments and subtree data fragments.

[0065] (Data Structure of Hierarchical Index Data) As shown in FIG. 9A , the data structure of the hierarchical index data generated by this embodiment is a suffix tree structure in which the suffix array of the reference string is layered. By setting a predetermined value for the number of layers from the top to the bottom, data at layers lower than the predetermined value is omitted. As a result, it is possible to reduce the amount of data in the hierarchical index data. In particular, in this embodiment, while a data structure with the number of layers proportional to the length of the reference string would be generated in the worst case, by setting the predetermined value to 1,000 or less, a partial suffix tree with 1,000 or fewer layers is used, thereby dramatically reducing the amount of data. The currently popular shotgun sequencing method uses query strings with a length of approximately 150 to 200 characters output from a sequencer, so even a partial suffix tree with 1,000 or fewer layers can be adequately mapped. This allows for a reasonable balance between reducing data volume and maintaining mapping quality.

[0066] Furthermore, the data structure of the hierarchical index data generated by this embodiment is such that the partial suffix tree is split at an intermediate level, with a data fragment of a supertree that includes a level higher than the intermediate level, and data fragments of multiple subtrees that include levels lower than the intermediate level and are connected to the lower side of the supertree. The phrase "split into data fragments" refers to a state in which the computer can store the data fragments as separate data in the memory module 102 (storage device), or the processor module 104 can process the data fragments as separate data. This reduces the data volume of the supertree data fragment and each subtree data fragment, allowing them to be stored in the memory module 102 and even to expand each data fragment in its entirety to cache memory. As a result, when using a so-called general-purpose computer, when the processor module 104 performs the mapping process (described later) in parallel, the probability of a search hit in the cache memory is extremely high, enabling high-speed mapping.

[0067] Although the present embodiment illustrates a case where the number of division stages in the hierarchy from a terminal node (leaf) in a partial suffix tree to the root (root (#)) is one, the present invention is not limited to this. For example, as shown in FIG. 16 , multiple division stages may be achieved by providing multiple division stages (gray nodes) in the hierarchy from a terminal node (leaf) to the root (root (#)). In FIG. 16 , division is performed when "the number of terminal nodes (leaves) belonging to a lower hierarchy than the terminal node (leaf) at the intermediate node passed through when tracing back from the terminal node (leaf) to the root (root (#)) is three or less." Furthermore, for the subtree divided at this division node, the total number of nodes is virtually converted to one, and then further division is performed by tracing back from the division node to the upper hierarchy, similarly, when the number of terminal nodes (leaves) belonging to a lower hierarchy than the terminal node passed through is three or less.

[0068] For a partial suffix tree reduced to 150 levels based on the human genome reference sequence, the partitioning condition, "when tracing back intermediate nodes from a terminal node toward a higher level, the number of terminal nodes (leaves) belonging to a lower level than the intermediate node is equal to or less than a predetermined value," is preferably set within the range of 30,000 to 300,000. Desirably, the predetermined value is set within the range of 50,000 to 200,000. For example, setting the predetermined value to 30,000 allows partitioning into one top-level tree including the root (root(#)) and approximately 1 million subtrees belonging to this top-level tree. For example, setting the predetermined value to 100,000 allows partitioning into one top-level tree including the root (root(#)) and approximately 300,000 subtrees belonging to this top-level tree. For example, setting the predetermined value to 30,000 allows partitioning into one top-level tree including the root (root(#)) and approximately 100,000 subtrees belonging to this top-level tree. Under these conditions, the data size of the top-level tree will be 10 MB or less (for example, about 1 MB), a size that can be fully expanded in the memory module 102 (main memory). Furthermore, under these conditions, the data amount of each subtree will be 10 MB or less, preferably 1 MB or less, a size that can be fully expanded in the memory module 102 (main memory), and will also be a size that can fit in the cache memory.

[0069] <Mapping Program> FIG. 10 is a flowchart illustrating an example of mapping processing by a computer system according to this embodiment. That is, FIG. 10 shows details of the mapping processing (S202) of a query string to a reference string shown in FIG. 2. This mapping processing maps the query string to the reference string and identifies the start position of its appearance in the reference string. FIG. 19 also shows a functional block configuration of the mapping computer system 202, including a super-tree expanding means 402, a query string group receiving means 404, a query string group expanding means 405, a query string group determining means 406, a first iteration means 407, a sub-tree and query string group set expanding means 408, a remaining memory determining means 409, a query string appearance position determining means 410, and a second iteration means 412. Each functional block will be described below with reference to a flowchart, etc. Note that while the mapping processing by the host computer 10 will be described below, the mapping processing by the subordinate computer 20, which operates in parallel, is similar.

[0070] 10, the host computer 10 reads the super tree in the hierarchical index from the database 30, and expands and stores it on the memory module 102 (S402). From the viewpoint of speed, the host computer 10 continuously expands and stores the data constituting the super tree on the memory module 102. Here, continuously expanding includes arranging the data at consecutive memory addresses in the same bank.

[0071] (Query string group receiving means 404) Next, the host computer 10 receives multiple query strings (query string group) for mapping to the reference string (S404). The received query strings are temporarily stored in a storage means such as the database 30. For example, if the reference string is a human genome reference sequence, the query strings are base pair data fragments read from a sequencer. For ease of understanding, the following description will be given assuming that the query strings are three characters long and there are 40 of them, as shown in FIG. 11. However, it should be noted that the query strings of base pair data fragments actually read from a sequencer are approximately 150 characters long, totaling approximately 700 million. Note that while the example here illustrates a case where the query string group is temporarily stored in the database 30, if the external sequencer itself has a database function, the database on the sequencer itself may be used as is.

[0072] (Query string group expansion means 405) Next, the host computer 10 reads the query string group stored in the database 30, expands it on the memory module 102, and stores it (S405). From the perspective of increasing speed, the host computer 10 continuously expands and stores the data constituting the query string group on the memory module 102. However, because the total data volume of the query string group is large and expansion of the supertree to the memory module 102 by the supertree expansion means 402 must be prioritized, only a portion of the query string group is expanded to the memory module 102 of the host computer 10.

[0073] (Query string group determination means 406) Next, the host computer 10 extracts each query string from the query string group expanded in the memory module 102 and determines to which subtree (subtree management number) the query string belongs by referring to the supertree ( FIG. 9B ) expanded in the same memory module 102 (S406). This determines the subtree management number associated with each query string, and the data is saved in the memory module 102. As a result, as shown in FIG. 11, each of the 40 query strings is stored in the memory module 102 with the subtree management number to which it belongs. The subtree management number data associated with each query string is finally saved in the database 30. Note that, as shown in FIG. 18, when the processor module has four cores A, B, C, and D, it is preferable to speed up the determination process by processing it in parallel.

[0074] (First repetition means 407) As described in the query string group expansion means 405, only a portion of the query string group can be expanded in the memory module 102 of the host computer 10 due to its upper capacity limit. Therefore, the host computer 10 refers to the database 30 and determines whether or not any undetermined query string groups from step S406 remain (S407). If any undetermined query string groups exist in the database 30, the process returns to step S405, where the remaining query string groups are expanded and stored in the memory module 102, and the subtree management number associated with each query string is determined (S406).

[0075] On the other hand, if there are no undetermined query strings in step S407, all query strings have been grouped by subtree, and the process proceeds to step S408. Note that in the query strings shown in Fig. 11, shaded strings indicate strings that were not used when determining by referring to the supertree.

[0076] (Subtree and query string group set expansion means 408) Next, the host computer 10 reads a specific subtree in the hierarchical index from the database 30 and expands it on the main memory module 102. The host computer 10 also expands and stores, on the same memory module 102, the query string group that belongs to the same subtree management number as the expanded specific subtree. As a result, the specific subtree and the query string group belonging to it are expanded as a set on the memory module 102 (S408). There are various methods for expanding the query string group on the memory module 102. For example, the query string group may be arranged in an array by connecting the previous query string and the subsequent query string with a pointer so that the query strings in the query string group are successively read by the processor module 104.

[0077] (Memory remaining capacity determination means 409) Furthermore, the host computer 10 references the remaining capacity of the memory module 102, and if there is still free space available to expand another specific subtree and query string groups that belong to the same subtree as the subtree, the process returns to step S408 and expands the other specific subtree and query string group as a set. As a result, multiple sets of subtrees and query string groups are simultaneously expanded in the memory module 102. In this embodiment, four sets of subtrees and query string groups with subtree management numbers 11, 9, 12, and 0 are expanded in the memory module 102.

[0078] (Query string appearance position determining means 410) Next, the host computer 10 sequentially extracts each query string from the set-expanded query string group, searches the subtree to which it belongs, and identifies to which array element (appearance position) each query string ultimately belongs (S410).

[0079] More specifically, for example, if a processor module has four cores A, B, C, and D, the identification process for four sets of subtrees and query strings with subtree management numbers 11, 9, 12, and 0 is assigned to each of cores A, B, C, and D, performing parallel processing. Because the data volume of each subtree is preset to be equal to or less than the capacity of the cache memory, as shown in FIGS. 12A to 12D, during the identification process, each subtree data is expanded into four L1 cache memories L1a, L1b, L1c, and L1d. As a result, the search process for each subtree in each of cores A, B, C, and D is completed within the L1 cache memories L1a, L1b, L1c, and L1d, achieving high-speed processing. Note that for ease of explanation, in FIGS. 12A to 12D, the number of levels is indicated above the top of each subtree, and the management number of each subtree is assigned to the left of each subtree. Additionally, the array number (occurrence position) of each terminal node (leaf) of each subtree is shown to the left of each subtree. The final array number (occurrence position) is shown to the right of each query string identified by cores A, B, C, and D. Note that shaded characters in the query string indicate characters that were not used when determining the occurrence position by referencing the subtree (characters used in determining the supertree). In step S410, the occurrence positions are identified for all subtrees and query strings set and expanded in memory module 102. Data related to the identification results is stored in database 30.

[0080] (Second repeating means 412) Next, the host computer 10 determines whether any unprocessed sets of subtrees and query strings remain in the database 30 (S412). In FIG. 12, four sets of subtrees and query strings have been processed out of the sixteen sets. Therefore, the process returns to step S408, and the four sets of subtrees and query strings with subtree management numbers 13, 17, 14, and 18 are expanded and stored in the memory module 102 as sets, as shown in FIGS. 13(A) to 13(D). Then, the four cores A, B, C, and D refer to the subtrees to which each query string belongs, set by set, to determine to which array element number (appearance position) each query string ultimately belongs (S410). It is further determined whether any unprocessed sets of subtrees and query strings remain (S412). Since there are still four sets of unprocessed subtrees and query strings, the process returns to step S408, and as shown in Figures 14(A) to 14(D), the four sets of subtrees and query strings with subtree management numbers 7, 10, 4, and 6 are expanded and stored in memory module 102 as sets. The four cores A, B, C, and D then refer to the subtree to which each query string belongs, set by set, to identify to which array element number (appearance position) each query string ultimately belongs (S410). Next, it is determined whether any unprocessed sets of subtrees and query strings remain (S412). Since there are no more unprocessed sets, the mapping process ends.

[0081] The processing of S408 to S412 is performed in parallel by the four cores A, B, C, and D in the processor module 104 of the host computer 10, thereby significantly reducing the processing time. Although not shown, the mapping process can also be performed simultaneously (distributed processing) by multiple lower-level computers 20(1) to 20(n) to achieve further time reduction.

[0082] 10 , if unprocessed query strings remain in step S407, information about the processed query strings is written to the database 30 and the memory module 102 is released. The unprocessed query strings are then read from the database 30 into the memory module 102, and the subtree management numbers associated with each query string are determined (S406). While the example illustrates a case in which step S407 completes the determination process for all query strings in the database 30, and the process proceeds to steps S408 to S412 to perform the identification process, the present invention is not limited to this. For example, as shown in FIG. 20 , once the determination process is completed in step S406 for the query strings expanded in the memory module 102 in step S405, the process proceeds directly to step S408, where all subtrees corresponding to the subtree management numbers determined in the determination process are imported and placed in a set expansion state, and the identification process can be performed in steps S410 and S412. Thereafter, in step S407, it may be determined whether or not there is an unprocessed query string, and if there is an unprocessed query string, the process may return to step S405.

[0083] As described above, the mapping computer program of this embodiment enables the host computer 10 to determine in advance which subtrees multiple query strings belong to by using data fragments of the upper tree with a reduced data volume. Furthermore, after the determination is complete, the subtrees and the query strings belonging to them are expanded as a set, and cores A, B, C, and D of the processor module 104 continuously search for multiple query strings belonging to the specific subtree, thereby continuously identifying the positions where query strings appear. During the identification process, each subtree is stored in cache memory, significantly reducing the number of random accesses to the memory module 102.

[0084] Although the above mapping process has been described as an example of identifying a reference string that matches the entire query string, the present invention is not limited to this. For example, a partial string may be extracted from the query string, and the mapping process may be performed on this partial string.

[0085] <Modification of Hierarchical Index Data Structure> In the above embodiment, the suffix tree used in the hierarchical index is generated by hierarchically organizing (forming a tree) multiple cyclic strings (see FIG. 5) obtained by cycling through a reference string having an ACGT character by character, as shown in FIG. 6. However, the present invention is not limited to this. A hierarchical suffix tree may be generated for a cyclic string (see FIG. 5) by grouping multiple characters into a set. FIG. 15 illustrates a case in which string pairs are formed for a cyclic string (see FIG. 5) in units of two characters, and a suffix tree is generated using these string pairs. This reduces the number of levels, thereby further reducing the amount of data for partial suffixes, supertrees, subtrees, etc. On the other hand, hierarchizing sets of, for example, five or more characters increases the number of nodes, potentially increasing the amount of data. Therefore, when generating a suffix tree by grouping multiple strings into sets, it is preferable that the number of characters in the set be between two and four.

[0086] The above-described embodiments are merely examples for explaining the present invention, and are not intended to limit the present invention to these embodiments. The present invention can be embodied in various forms without departing from the spirit of the present invention. For example, the present embodiments illustrate the creation of a hierarchical index and mapping for a reference character string that is genetic information, but the present invention is not limited to this and can be applied to the creation of a hierarchical index and mapping for a reference character string other than genetic information.

[0087] Also, for example, in the methods disclosed herein, steps, operations, or functions may be performed in parallel or in a different order, unless the results are inconsistent. The steps, operations, and functions described are provided merely as examples, and some of the steps, operations, and functions may be omitted or combined into one, or other steps, operations, or functions may be added, without departing from the spirit of the invention.

[0088] Furthermore, in the above embodiment, the hardware configuration of the computing device is a general-purpose structure, but the present invention is not limited to this. For example, individually customized hardware such as an FPGA (Field Programmable Gate Array) may be employed. Specifically, all or part of the functional blocks shown in FIG. 19 may be realized by an FPGA.

[0089] Furthermore, the detailed representation method (notation method) for implementing the hierarchical index data structure shown in the above embodiment is not particularly limited. For example, if the tree structure is represented by the LOUDS (Level-Ordered Unary Degree Sequence) method, it is possible to further reduce the memory size.

[0090] Furthermore, although various embodiments are disclosed in this specification, specific features (technical matters) in one embodiment can be added to or substituted for specific features in other embodiments, with appropriate modifications, and such forms are also included in the spirit of the present invention.

Claims

1. A computer program for creating an index that enables a computing device to create an index to be used when identifying a substring in a reference string based on a query string, the computer program comprising a step of setting a predetermined value as the number of levels from the top level to lower levels in a suffix tree in which the suffix array of the reference string is hierarchically organized, thereby omitting hierarchical data below the predetermined value.

2. The index creating computer program according to claim 1, wherein the predetermined value for the number of hierarchical levels is set to 1,000 or less.

3. The index creating computer program according to claim 1, wherein the predetermined value for the number of levels is set to be equal to or greater than the number of characters in the query string.

4. The computer program for creating an index according to claim 1, further comprising a step of dividing the suffix tree at an intermediate level to generate a supertree including a level higher than the intermediate level, and a plurality of subtrees (hereinafter referred to as a group of subtrees) including a level lower than the intermediate level and connected to the lower side of the supertree.

5. The index creating computer program according to claim 4, wherein the amount of data in the subtree is set to a value that can be stored in a cache memory in the computing device.

6. The index creating computer program according to claim 4, wherein the data amount of the subtree is set to 10 MB or less.

7. A computer program for creating an index that enables a computing device to create an index to be used when identifying a substring in a reference string based on a query string, the computer program comprising a step of dividing a suffix tree, which is a hierarchical arrangement of a suffix array of the reference string, at an intermediate level to create a supertree that includes a level higher than the intermediate level, and a plurality of subtrees (hereinafter referred to as a group of subtrees) that include a level lower than the intermediate level and are connected to the lower side of the supertree.

8. A computer program for mapping, causing a computing device to implement a method for identifying substrings in a reference string based on a query string, the computer program for indexing comprising as part thereof any one of claims 1 to 7, the method further comprising: mapping the query string to the reference string by referencing the hierarchical index to identify substrings in the reference string that match at least a portion of the query string.

9. A mapping computer program for causing a computing device to realize a method for identifying a substring in a reference string based on a query string, wherein a hierarchical index generated based on the reference string is divided at an intermediate level to have a supertree including a level higher than the intermediate level and a plurality of subtrees including a level lower than the intermediate level and connected to a lower side of the supertree (hereinafter referred to as a group of subtrees), the method comprising: mapping the query string to the reference string by referring to the hierarchical index in order to identify a substring in the reference string that matches at least a part of the query string, the mapping of the query string comprising: a determination step of determining to which of the subtrees in the group of subtrees each of the query strings belongs by performing a matching process on the plurality of query strings using the supertree; and an appearance position determination step of determining an appearance position of the query string relative to the reference string by performing a matching process on the plurality of query strings that belong to a specific subtree determined by the determination step, using the specific subtree.

10. The mapping computer program according to claim 9, wherein the step of determining the occurrence positions in the subtree is processed in parallel.

11. The mapping computer program according to claim 9, wherein the discrimination step is processed in parallel.

12. The computer program for mapping according to claim 9, wherein in said occurrence position determining step, said subtree is expanded in a cache memory in said computing device.

13. A computer program for mapping according to claim 9, comprising as a part thereof a computer program for creating an index according to any one of claims 1 to 7.

14. A computer apparatus for creating an index, the computer apparatus having a computing device including a storage device, for creating an index used to identify a substring in a reference string based on a query string, the computing device having partial suffix tree generation means for reading out the reference string stored in the storage device, setting to a predetermined value the number of levels from the top level to lower levels in a suffix tree in which the suffix array of the reference string is hierarchically organized, and generating a partial suffix tree in which hierarchical data lower than the predetermined value is omitted.

15. A computer apparatus for creating an index, the computer apparatus having a computing device including a storage device, for creating an index used to identify a substring in a reference string based on a query string, the computing device having a suffix tree splitting means for splitting a suffix tree, which is a hierarchical arrangement of a suffix array of the reference string stored in the storage device, at an intermediate level to create a supertree including a higher level and a plurality of subtrees (hereinafter referred to as a group of subtrees) connected to the lower side of the supertree.

16. A mapping computer apparatus having a computing device including a storage device and for identifying a substring in a reference string based on a query string, wherein the storage device stores a supertree including a hierarchy higher than an intermediate hierarchy, the supertree being generated by dividing a hierarchical index generated based on the reference string at an intermediate hierarchy, and a plurality of subtrees including a hierarchy lower than the intermediate hierarchy and connected to a lower side of the supertree (hereinafter referred to as a group of subtrees), the computing device having a mapping means for mapping the query string to the reference string by referring to the supertree and the subtree in order to identify a substring in the reference string that matches at least a part of the query string, the mapping means having: a determining means for determining which of the subtrees in the group of subtrees each of the query strings belongs to by performing a matching process on the plurality of query strings with reference to the supertree stored in the storage device; and an appearance position determining means for determining an appearance position of the query string relative to the reference string by performing a matching process on the plurality of query strings that belong to a specific subtree determined by the determining means with reference to the specific subtree stored in the storage device.

17. A hierarchical index data structure for a reference string, used in a mapping computer program for enabling a computing device to implement a method for identifying a substring in a reference string based on a query string, the hierarchical index data structure being a suffix tree in which a suffix array of the reference string is hierarchically arranged, the number of levels from the top level to the lower levels being set to a predetermined value, and hierarchical data below the predetermined value being omitted.

18. The data structure of a hierarchical index according to claim 16, characterized in that the hierarchical index has a data piece of a supertree that includes a hierarchy higher than the intermediate hierarchy by dividing the suffix tree at an intermediate hierarchy, and data pieces of a plurality of subtrees that include a hierarchy lower than the intermediate hierarchy and are connected to the lower side of the supertree.

19. A data structure of a hierarchical index of a reference string, used in a mapping computer program for implementing on a computing device a method for identifying a substring in a reference string based on a query string, characterized in that the hierarchical index is split at an intermediate level to have a data piece of a supertree that includes a level higher than the intermediate level, and data pieces of multiple subtrees that include a level lower than the intermediate level and are connected to the lower side of the supertree.

Citation Information

Patent Citations

  • A method and a system for encoding and decoding of suffix tree and searching within encoded suffix tree

    EP3136607A1

  • Method for pattern-search, apparatus thereof, computer program and record medium

    JP2002229987A

  • Efficient Seeding For Read Alignment

    US20200265923A1