A method and system for ultra-parallel alignment

Abstract

The method of single cycle super parallel comparison is designed by using FPGA, programmable logic or TCAM chip, which can complete the bitwise comparison between the key item and multiple table rows in a single logic cycle, output the address of the matched table row, and output the statistics data and position information of the same and different points. The algorithm supports table reconfiguration, same and different point processing, filter filtering, table mapping, one-dimensional array, two-dimensional data and multi-dimensional data comparison; the system includes a comparator array, reconfigurable logic, same and different point processor, mapping memory, filter, communication interface. It can form an independent comparison server and PCIE acceleration card. The method can speed up more than 10 9 orders of magnitude compared with the fastest CPU von Neumann computer comparison algorithm when comparing 10M table rows.

Description

[0001] This application is a divisional application of the invention application No. 201910775554.1 filed on August 21, 2019. TECHNICAL FIELD

[0002] The present application relates to the field of information technology and bioengineering, in particular to the field of biological sequence alignment in genetic engineering and protein engineering, and the field of natural language and image recognition. BACKGROUND

[0003] Biological sequence alignment, natural language processing and image alignment are emerging disciplines. After searching and researching, the inventor found that almost all existing alignment technologies are Feng's computer alignment algorithm. Some try to use GPU parallel algorithm, but its limitations are too great and cannot truly realize high-speed parallel alignment of specific data.

[0004] TCAM (Ternary Content Addressable Memory) is a typical parallel alignment operation. It is a non-Feng computer alignment operation. Through bit comparison technology, a large number of memories can be arranged and compared with a specific memory in a single cycle, and the first matching memory address is output. The speed difference is exponential and multiplicative.

[0005] In the design of routers, TCAM parallel search technology has been used to locate the next hop path of the router in a single cycle in a massive routing table. For example, the input end of the router needs to match an output end in a million-level output end, and TCAM can achieve single-cycle alignment matching. Although there is a large proportion of similarity in biological sequence alignment, the difference is that the router routing table has a very short width. Taking IPV6 as an example, it does not contain check bits, and the width is 128 bits (bit), 16 bytes (byte). Adding check bits, it is 9 x 16 = 144 bits. Moreover, the routing table does not support data insertion, deletion and masking at any bit position, and it does not support different bit position number statistics during alignment.

[0006] However, most specific data, such as biological sequence data, is recorded in bytes in character format. For example, a DNA (Deoxyribo Nucleic Acid) fragment in the SAM (The Sequence Alignment / Map format) format may be 150 bytes, and with a check bit, it is 1350 bits. Additionally, the length of the routing table is fixed, while the length of biological sequences varies. And during alignment, differences need to be counted, and insertions, deletions, and masking of alignment data are often required, and the positions of these operations are also not fixed. For example, in natural language processing, in Japanese, "私はご飯を食べる,我吃饭", "ご飯を食べる私,吃饭的我", "ご飯は私に食べられた,饭被我吃了" all contain the keywords "我", "饭", "吃". As long as these keywords are retrieved in a sentence, the general meaning of the sentence can be roughly understood. Another example is in image processing. A human face appears in different pictures, often blurred, but there are still some key pixel sets. Retrieving them can improve the recognition rate of the human face. These are not very suitable for TCAM alignment. Perhaps, this is the reason why TCAM has not appeared in biological sequence alignment, natural language alignment, image alignment, etc.

[0007] The deficiencies of the prior art are as follows:

[0008] 1. CPU algorithms

[0009] CPU (Central Processing Unit) algorithms are all based on the computing methods of von Neumann computers, characterized by sequential order, sequential alignment, and search. Whether it is linear search, binary tree search, HASH search, BWT, BWA-MEM, etc., they are all computing methods based on serial order.

[0010] 2. GPU algorithms

[0011] GPU(Graphics Processing Unit, graphics processing unit) algorithm, for example, NVIDIA(Compute Unified Device Architecture, general-purpose parallel computing architecture) uses multiple CUDA cores similar to CPU, and a thread is driven for each CUDA to complete the so-called multi-thread comparison operation. This is actually a serial operation of von Neumann computer, but it is synchronized by up to thousands of CUDA to complete serial comparison. Compared with CPU, the speed is at most thousands of times faster. However, for comparison operations, regardless of which algorithm is used, there is a logical sequence of before and after, so the actual acceleration of GPU is far from the number of CUDA compared with CPU, that is, far from thousands of times.

[0012] 3. TCAM algorithm

[0013] Although TCAM can realize single-cycle comparison of millions of data, its original design is for the routing table of the router, which is seriously different from the comparison requirements of biological data, natural language data and image data, and does not meet the requirements of biological comparison.

[0014] The intention of the present application is to provide an implementation step and equipment system of a single-cycle super-parallel comparison technology based on a hardware circuit, which fundamentally solves the speed problem of data comparison and is suitable for biological sequence data, natural language data comparison processing, image data comparison processing and other massive data comparison work. SUMMARY

[0015] The purpose of the present application is to provide a super-parallel comparison method and system that can complete data comparison in a single logical cycle and is reconfigurable, which can be used for high-speed DNA comparison, RNA(Ribo Nucleic Acid, ribonucleic acid) comparison, protein comparison, natural language comparison and image comparison. It should be noted that the step numbers described below are only for recording convenience and do not mean the order of numbering. The relationship between steps is arbitrary unless it is specifically stated that certain steps have a sequence, otherwise there is no sequence.

[0016] The purpose of the present application is achieved by using the following technical solutions:

[0017] A super-parallel comparison method, including but not limited to:

[0018] S100 step: including but not limited to comparing units composed of table entries and key entries, completing super-parallel comparison of key entry data and table entry row data in the table entries.

[0019] S200 steps include, but are not limited to, inputting the parent string into the table entry, inputting the substring into the key entry, looping the comparison, and outputting the result.

[0020] S300 steps include, but are not limited to, generating substring selections based on the comparison algorithm, inputting them into table entries, cyclically shifting and extracting the parent string into the keyword entries, comparing and outputting the results.

[0021] The alignment algorithms include biological sequence alignment algorithms (such as BWA (Burrows-Wheeler alignmenttool) algorithm and GateKeeper algorithm), natural language alignment algorithms, and image alignment algorithms.

[0022] The parent string and child string include steps for changing and compressing data formats. Data formats include, but are not limited to, SAM (sequence alignment map format) data of biological sequences, natural language data, and image format data, and their data forms include, but are not limited to, database formats. All operations on the parent string and child string include, but are not limited to, compressing data from Chinese character encoding, hexadecimal, decimal, octal, and ASCII (American Standard Code for Information Interchange) into binary encoding. This includes recording the operation type and the results of the operation steps. It includes, but is not limited to, steps for communicating with external network devices, gene sequencing equipment, voice equipment, and image equipment via Ethernet ports. The alignment service is completed through the processor system and working steps.

[0023] Based on the foregoing technical solutions, other solutions of the present invention may employ one or more of the following combinations of partial improvements:

[0024] The S100 step specifically includes, but is not limited to:

[0025] In step S101, the table entry includes, but is not limited to, one or more table entry rows. Each table entry row stores table entry row data, and each keyword entry stores keyword data. The keyword entry data is simultaneously compared with the data from one or more table entry rows, position by position, within the same logical cycle. If a table entry row matches the keyword entry data, the address of the matching table entry row is output. If multiple table entry rows match the keyword entry data, the address of the first or last matching table entry row in the address order is output. If no table entry row matches the keyword entry data, no table entry row address is output.

[0026] like Figure 1As shown, 1101 stores the keyword item data, 1102 is the keyword item address, 1200 is the table item, which includes n table item rows: 1211 is the first table item row, which stores the first table item row data, 1212 is the address of the first table item row; 1221 is the second table item row, which stores the second table item row data, 1222 is the address of the second table item row; 12n1 is the nth table item row, which stores the nth table item row data, 12n2 is the address of the nth table item row.

[0027] The same logic cycle includes the time length of one keyword item and more than one table item row completing one comparison, which can be a combination of several time periods in work. If the comparison unit is composed of a TCAM chip, the single cycle is the comparison cycle of the TCAM. For example, a device with a main frequency of 250MHz, the comparison cycle is 4nS. If the comparison unit is composed of a FPGA (Field-Programmable Gate Array) chip with a main frequency of 500MHz, the comparison cycle is 2nS. The more than one table item row, for example, the designed table item depth is 10 million rows (10M), then the method can complete the comparison of the 10 million rows of data with the keyword row data at the same time within 4nS or 2nS. If the 800010th row in the table item row is consistent with the keyword item data, the address 800010 of the table item row is output.

[0028] As the string comparison algorithm of Von Neumann computer, it is usually executed by the CPU with a loop comparison program. According to the estimation of the fastest CPU with a main frequency of 4GHz, the CPU clock cycle is 0.25nS. Since the string comparison algorithm cannot be logically resolved into multi-thread execution, it can only be executed by a single thread. It is estimated that the string comparison algorithm with a length of 150 bytes is programmed by using the least time-consuming assembly language. The time cost of one loop is about 40 clock cycles, i.e. 40*0.25nS=10nS. 150 loops take a total of 1.5μS. It takes at least 10000000*1.5μS=15S to complete the comparison of 10 million rows! Compared with 4nS, it is 3.75x10 9 times, compared with 2nS, it is 7.5x10 9 times larger if programmed in a high-level language. It can be seen that the comparison method of the present application is amazing in performance improvement, which is fundamentally different from the loop comparison of Von Neumann computer.

[0029] Compared with GPU (Graphic Processing Unit) accelerated algorithm, CUDA (Compute Unified Device Architecture) contained in GPU can be regarded as a simple CPU, but it is still a serial loop comparison of von Neumann computing. Although CUDA can contain thousands in GPU, on the one hand, we cannot design programs for each CUDA to solve the comparison problem in the face of complex comparison data; on the other hand, even if we do so, it is still a serial loop comparison process of von Neumann computer. In fact, the design of GPU is driven by the unified processing of image data. For example, operations such as improving image color and gray. The characteristics of these operations are to do the same operation on all data in the image area, rather than different operations for each pixel. Therefore, the acceleration performance effect of using GPU for data comparison operation is not obvious. There are reports that in the extreme ideal case, the speedup is 10 3 orders of magnitude compared with CPU method. By analogy, the speedup of the present application compared with GPU method is more than 10 6 orders of magnitude.

[0030] The matching refers to that the data of a table item row and the data of the key item are the same at each bit point in the order from left to right or from right to left. For example, the data of the key item is "ATCGGTCCAATG", and the data of a table item row is also "ATCGGTCCAATG", so the table item row matches the key.

[0031] In step S102, the bit point includes but is not limited to a single binary bit, a character composed of more than one binary bit, and an information unit composed of more than one binary bit. The comparison includes but is not limited to the operation algorithm of and, or, not, exclusive or, equal, opposite, not equal or their combinations of the data of the key item and the table item row at the same arrangement position. The combination includes the algorithm provided by some formulas including these operations.

[0032] In Figure 1 , the data in the key item and the table item shows binary number, and only 0 and 1. In Figure 2 , the data in the key item and the table item is single-byte ASCII code representing DNA sequence, and can also be multi-byte other encoding, such as Chinese character encoding, image encoding, video encoding. The comparison includes binary bit comparison, single-byte ASCII code comparison, and other encoding comparison.

[0033] For biological data, since the data types are limited to only adenine (A), guanine (G), cytosine (C), thymine (T), and uracil (U), either 1-byte ASCII encoding or 3-bit bit-compression encoding can be used. As for the alignment operation, it can be either byte-based or bit-based.

[0034] In step S103, a logic definition is used to connect one or more FPGA chips or programmable logic chips to form the table entry and keyword entry. The table entry and keyword entry are expanded, and the connection method includes, but is not limited to, connecting in the width direction or the depth direction of the table entry.

[0035] In step S104, the logical definition also includes, but is not limited to, defining a mask for a specified site in a table entry row or keyword field. The mask includes, but is not limited to, a mask bit and a specified bit. The mask bit prevents the site from participating in the comparison, and the specified bit replaces the site data with specified data.

[0036] like Figure 2 As shown, the mask is marked with # and *, where # is the specified bit and * is the mask bit. Specifically, for example, in 2101, the specified bit A is used to replace the original content, and mask bits are set for 2102 and 2103. It should be noted that the mask bit setting shown does not directly replace the data of 2102 and 2103 with *, but rather performs a logical operation on the signal units at the positions corresponding to 2102 and 2103, and the result forms the mask. For example, a logical "AND" relationship is used to create a high-impedance state for the corresponding model line, thus achieving the masking operation. For TCAM chips, each binary bit in the table row has a corresponding tri-state mask unit, so only the mask bit needs to be set. For FPGAs or other programmable logic chips, the mask is set through logical definition.

[0037] In step S105, the logic definition includes, but is not limited to, defining the NP and CPU in the FPGA chip and programmable logic chip to perform the specified calculation functions.

[0038] Step S106, using logical definitions, also includes, but is not limited to, connecting one or more TCAM chips or a combination of TCAM chips and FPGA chips to form the table entries and keyword entries, expanding table entries and keyword entries, and connection methods including, but not limited to, connecting in the width direction and connecting in the depth direction of the table entries.

[0039] In step S107, the logical definition also includes, but is not limited to, defining the tri-state mask bit of a specified site in the table row or keyword field of the TCAM chip, so that the site does not participate in the comparison.

[0040] In step S108, the logical definition also includes, but is not limited to, defining the NP (Networking Processor) and CPU in the TCAM chip to perform the specified calculations.

[0041] In step S109, the logical definition also includes, but is not limited to, the definition based on the different positions, including but not limited to bit comparison, single-byte comparison, and double-byte comparison. The bit comparison is performed bit by bit, the single-byte comparison is performed byte by byte, and the double-byte comparison is performed two-byte by two-byte.

[0042] It's important to note that for biological sequence alignment, since biological sequence files mostly use byte-based encoding, byte-based alignment should be used. However, for natural language alignment, such as for Chinese, since double-byte or multi-byte encoding is used, multi-byte alignment should be used. For image alignment, the image encoding should be considered; for example, if a pixel uses 24 bits, 3-byte alignment can be used.

[0043] Based on the foregoing technical solutions, one or more of the following partial improvements can be adopted in other solutions of the present invention:

[0044] The S200 step specifically includes, but is not limited to:

[0045] In step S201, calculate the volume MV of the parent string and the volume TV of the table entry. If MV ≤ TV, the parent string is not decomposed. If MV > TV, the parent string is decomposed or the parent string is decomposed cyclically until the decomposition ends.

[0046] The volume, or total length, is the product of the number of points in the row and the number of columns.

[0047] In step S202, according to the width TW of the table entries, the parent string or the decomposed parent string is cut and extracted sequentially from the beginning, and then input into the table entries in the order of the table entry depth TD.

[0048] For example Figure 2 The table contains rows 1, 2, and so on up to row n.

[0049] Here, the parent string is usually the original target data, such as the DNA sequence of a gene, a piece of natural language text, an image, or a video. The substring is a DNA fragment, a phrase, a small part of an image, or a small part of a video that needs to be searched and compared. The comparison work is to search the parent string to see if there is a substring and where the substring is located.

[0050] S203, calculate the width of the substring CW and the width of the keyword item TW, if CW≤TW, use mask to compensate the width of the substring CW to TW, if CW> TW, decompose the substring or adjust TW to TW> CW; the mask includes but is not limited to masking bit and designated bit, the masking bit makes the bit not participate in the comparison, and the designated bit is replaced by the designated data.

[0051] For example Figure 2 The keyword item in the table item.

[0052] Generally, for the convenience of comparison, the number of width bits of the table item and the number of width bits of the keyword item are equal. If the width of the keyword is less than the width of the table item, the keyword item can be extended by using a mask to extend the keyword item to the same width as the table item row, so that the comparison is facilitated.

[0053] S204, according to the comparison algorithm, insert, delete more than one designated bit in the table item row, and record the operation and repair other table items in the parent string.

[0054] For biological sequence comparison, there are operations such as insertion, deletion, and replacement of specific bits in the parent string. For natural language processing and image video, there are also similar repair operations.

[0055] Actually, there are similar operations for the keyword item, which can be completed when the keyword item is arranged or in S300.

[0056] S205, input the substring to the keyword item, compare the keyword item with the data of all table item rows in the table item, and output the address of the matched table item row.

[0057] When the keyword item is compared with a large number of table item rows, multiple matched table item rows often appear. According to the set strategy, the address of the first matched table item row can be output, or the address of the last matched table item row can be output.

[0058] S206, if there is a matched table item row address output, record the table item row address, clear the table item row data, and loop comparison until the matching is unsuccessful.

[0059] The clearing of the matched table item row is to clear the data of the table item row after recording and outputting the address of the matched table item row. The purpose of this operation is to find out whether there is a next table item row matching the keyword item. Through the loop, all table item rows matching the keyword item are found.

[0060] S207, the table entry data is taken as a one-dimensional array, a loop body is executed, the loop body includes but is not limited to left shifting or right shifting one bit point, S205 and S206 are executed, the loop body is executed TW-1 times, a comparison result of the loop body is output, comparison of one table entry for one key item is ended, and a result is calculated and output.

[0061] Since the width of the table entry row is TW bit points, when all the mother strings are connected into a one-dimensional array, only TW-1 times of loop are needed to complete all the comparisons of the mother strings in the table entry and the sub-strings in the key item.

[0062] On the basis of the foregoing technical solution, one prominent innovation of the present application is to compare the content of the sub-string and the content of the mother string in reverse, to pre-combine the possibility of the sub-string (for example, including but not limited to the following steps: sub-string E1, sub-string E2, sub-string E3, sub-string E4), to input the data of the combinations into the table entry, to let the mother string flow over the key item in the form of data stream one bit point by one bit point, to compare one table entry one bit point, and to output the table entry row address that is possibly matched, which has more suitable and more efficient effects for biological sequence comparison and natural language processing. The following one or more local improvement measures can be used:

[0063] The S300 step specifically includes but is not limited to:

[0064] S301, the width CW of the sub-string and the width TW of the table entry are calculated, if CW≤TW, the width CW of the sub-string is extended to TW by using a mask, if CW> TW, the sub-string is decomposed or TW is adjusted to TW≥CW; the mask includes but is not limited to a shield bit and a specified bit, the shield bit makes the bit point not participate in comparison, and the sub-string after the execution of the step is sub-string E1; the specified bit is replaced by a specified data at the bit point.

[0065] The mask is realized by logic setting in an FPGA chip, a programmable logic chip, and a TCAM chip. The shield bit shields the data of the corresponding bit point in the table entry row, so that the data of the bit point does not affect the comparison result; the specified bit is replaced by a specified data at the corresponding bit point during comparison. The shield bit and the specified bit are both data with specified position information, and attention must be paid during setting.

[0066] S302, according to the comparison algorithm, one or more than one sub-string E2 is generated by using one or more than one shield bit to shield one or more than one specified bit point for the sub-string E1, and each sub-string E2 and the shield bit point are recorded in a sub-string mapping table.

[0067] S303, according to the alignment algorithm, for the substring E1, using more than one specified bit to replace more than one specified site to generate more than one substring E3, record each substring E3 and the replacement site to the substring mapping table.

[0068] S304, according to the alignment algorithm, for the substring E1, using more than one specified site to generate more than one substring E4, record each substring E4 and the insertion, deletion site to the substring mapping table.

[0069] S305, input the substring E2, substring E3, substring E4 to the table entry, record the address of the substring E2, substring E3, substring E4 to the substring mapping table.

[0070] In fact, for the operation of the substring E1, substring E2, substring E3, substring E4 of the substring, according to the attribute of the alignment, there are other operation modes, which are included but not limited to the relationship.

[0071] S306, for the mother string, establish a pointer SP, according to the width of the substring CW, start from SP=1 to cut the mother string, input to the key item, compare the key item data with the data of all table entry rows in the table entry, and output the comparison result.

[0072] S307, if there is a matching table entry row address output, record the table entry row address and the pointer SP, and modify the substring mapping table.

[0073] S308, if there is no address output, modify the pointer, assign SP+1 to SP, and execute the steps including but not limited to S305 and S306 circularly until the end of the mother string, and calculate the output result.

[0074] The above operation includes but is not limited to loop and recursion, and also includes another loop and recursion operation for a specific interval to realize multiple scanning alignment and step-by-step optimization effect.

[0075] On the basis of the foregoing technical solutions, in order to better support the alignment operation, the reconfigurable method of the table entry structure is designed, mainly to modify the width and depth of the table entry. Specifically, one or more of the following local improvement measures are adopted:

[0076] Including but not limited to S400 table entry reconfigurable step, specifically including but not limited to:

[0077] S401, calculate the width of the longest substring EM according to the following formula: TW0=1+MOD(CW-MAX / 8), wherein: MOD is the integer function, TW0 is the reconfigurable table entry width, and CW-MAX is the width of the longest substring EM.

[0078] S402, according to TW0, under the premise of constant table volume TV, calculate TD0=TV / TW0, and take TD0 as the reconstructed table depth.

[0079] In fact, here the decimal in TD0 is processed by truncation, that is, if it is not an integer, the row is removed.

[0080] S403, according to TW0 and TD0, modify the comparison parameters and perform the comparison operation.

[0081] The comparison parameters include the width TW0 and the depth TD0 of the sub-string in the upper software.

[0082] It should be noted here that when the depth of the sub-string is greater than 1, that is, when the sub-string is truncated, the sub-string in this stage needs to be connected to calculate in the upper comparison software. However, in general, the sub-string is not truncated unless the sub-string is too long.

[0083] S404, using the NP, CPU system in the FPGA chip, programmable logic chip and / or TCAM chip to execute and manage the steps including but not limited to S401, S402 and S403.

[0084] Here includes the method designed to perfect the truncation of the sub-string.

[0085] On the basis of the foregoing technical solutions, the present application designs a method for counting the same and different sites during comparison. This can calculate the relevant probability for the comparison and retrieval of biological sequences and natural language processing. Specifically, one or more of the following local improvements can be used:

[0086] Including but not limited to S500, the same and different site processing step, specifically including but not limited to:

[0087] S501, the logic definition further includes but is not limited to defining the logic unit of the FPGA chip, programmable logic chip and TCAM chip, completing the calculation of the same number TW-YES and different number TW-NO of the data in each site of the key item and each table row in the table during comparison, including calculating the probability and the position of the site.

[0088] S502, according to the TW-YES and TW-NO of each table row in the table, selecting the maximum or minimum value, and outputting the maximum or minimum value and the row number of the table row where it is located.

[0089] S503, for S502, the maximum or minimum value and the data of the table entry row are cleared, and S501 and S502 are re-executed until the values of TW-YES and TW-NO reach the specified value.

[0090] This is a calculation method designed to find the maximum or minimum probability of coincidence or non-coincidence in the table entry.

[0091] S504, the logic definition further includes but is not limited to defining hardware logic to complete the steps of S501, S502, and S503.

[0092] The defined hardware logic includes logic units and unit groups for FPGA chips, programmable gate array chips, and TCAM chips.

[0093] S505, using NP, CPU system in the FPGA chip, programmable logic chip, and TCAM chip, using software to execute the steps including but not limited to S501, S502, and S503.

[0094] S506, using NP, CPU system in the FPGA chip, programmable logic chip, and TCAM chip, sorting and outputting TW-YES and TW-NO for each table entry row in the table entry.

[0095] As shown in Figure 3 This is a principle verification example designed for binary bits. Figure 3 In the above technical solution, the key item and one of the many table entry rows are operated by 3301 to 330n operators, for example, using bitwise operations including but not limited to "XOR", then the key item is 1001…0, one table entry row is 1000…0, the result of the operator operation is 0001…0, assuming that the omitted numbers "…" are not calculated, the result is stored in the position register and sent to the counter, including but not limited to calculating the number of "1"s. Thus, the data of the position register is 0001…0, the counter counts 1, and the two data are output, which is the result of the same and different bit processing.

[0096] On the basis of the foregoing technical solutions, in some other schemes of the present application, one or more of the following partial improvement measures can be used, which is an effective method for repairing and filtering related noise when including but not limited to biological sequence alignment and natural language alignment operations:

[0097] Including but not limited to the step of filtering by S600 filter, specifically including but not limited to:

[0098] S601, during the comparison of the keyword item and the table item row, covering, inserting and deleting different sites, and recording the length of the covering, inserting and deleting different sites.

[0099] S602, during the comparison of the keyword item and the table item row, covering, inserting, replacing and deleting continuous different sites of a specified length and continuous same sites of a specified length, and recording the length of the covering, inserting, replacing and deleting different sites.

[0100] S603, between the multiple comparisons, according to the comparison algorithm, performing the mask processing of the mask bit and the specified bit for the data of the table item and the keyword item.

[0101] S604, between the multiple comparisons, according to the comparison algorithm, performing the covering, inserting, replacing, deleting and filtering processing for the data of the table item and the keyword item.

[0102] On the basis of the foregoing technical solutions, in other schemes of the present application, one or more of the following partial improvement measures can be adopted:

[0103] The table item mapping step S700 includes but is not limited to the following steps:

[0104] S701, in addition to the FPGA chip, programmable logic chip and TCAM chip, a storage area is established by using a memory, corresponding to the table item, and the data and intermediate data and result data of the table item and the keyword item are copied to the storage area to become a mapping of the table item.

[0105] S702, synchronizing the operation and modification record of the table item and the keyword item to the mapping.

[0106] S703, storing the mapping to a database, and processing by the NP, CPU system in the FPGA chip, programmable logic chip and TCAM chip or the NP, CPU system outside the FPGA chip, programmable logic chip and TCAM chip.

[0107] The purpose of this method is to establish a backup to play a role in subsequent backtracking of intermediate data and processes. In fact, in some cases, this mapping can be stored in a database including electronic hard disks and mechanical hard disks for subsequent analysis.

[0108] On the basis of the foregoing technical solutions, in other schemes of the present application, one or more of the following partial improvement measures can be adopted:

[0109] including but not limited to when the substring is a substring KU, including multiple substrings KU1 to multiple substrings KUn , need to find and compare KU1 to KU n S800 multi-substring unordered comparison step, including but not limited to:

[0110] S801 step, determine the substring KU including multi-substring KU1 to multi-substring KU n , determine KU1 to KU n No front and back order.

[0111] S802 step, determine the search range in the mother string, cut the range of the mother string content input to a designated comparison unit entry, to KU1 to KU n for the loop range, in each loop, in turn, input KU1 to KU n to the key item of the designated comparison unit, execute including but not limited to S300 step, record output.

[0112] S803 step, if all KU1 to KU n are matched successfully, the loop ends, output KU matching successful data, if KU1 to KU n There is no match, output KU matching failure, exit the loop.

[0113] The multi-substring refers to a substring including several pieces of data that need to be compared, including but not limited to some specific small fragments in a piece of DNA. These fragments may not need to have a strict front and back order, in which case the S800 step is suitable for this comparison operation.

[0114] On the basis of the foregoing technical solutions, for some data with two-dimensional characteristics, such as but not limited to DNA double helix structure data, image, video data, in some other schemes of the application, one or more of the following local improvement measures can be used:

[0115] Including but not limited to S900 two-dimensional comparison step, including but not limited to:

[0116] S901 step, according to the two-dimensional arrangement characteristics of the comparison, establish two-dimensional mother string S (xs,ys) and two-dimensional substring K (xk,yk) .

[0117] Wherein, XS, YS is the two-dimensional row and column coordinates of the mother string, XK, YK is the two-dimensional row and column coordinates of the substring. It should be noted that as two-dimensional data, if you need to shift, you must shift the entire two-dimensional arrangement as a whole, such as left shift, right shift, up shift, down shift, and cannot move only one row or one column.

[0118] It is particularly emphasized here that as the comparison unit, the entries therein can only be one-dimensional truncation, and the key items therein are also one-dimensional. The two-dimensional comparison to multi-dimensional comparison of the present application is actually for the parent string and the substring. All dimensions higher than the one-dimensional dimension of the comparison unit are completed by the preparation step outside the input and output comparison unit. For example, the data comparison of the double helix structure and folding structure of DNA, and the context association of three-dimensional image data and natural language, before entering the comparison unit and before, the data is decomposed into one-dimensional entry and one-dimensional key item structure suitable for the comparison unit by the software and steps of the present application, and then compared in the comparison unit. That is, before entering the comparison unit, the data needs to be reduced to one dimension, and then compared in the comparison unit. The results of the comparison are increased in dimension to restore the original dimension. Once recursion or loop, the two-dimensional to multi-dimensional comparison operation is completed.

[0119] S902 step, input two-dimensional parent string to the entry.

[0120] S903 step, copy a row of data in the two-dimensional substring K (xk,yk) to the key item in turn and compare, execute S500 difference site processing step, output the comparison results of each row in the two-dimensional comparison.

[0121] S904 step, loop copy the next row of data in the two-dimensional substring K (xk,yk) to the key item and compare until the comparison of all rows in the two-dimensional substring K (xk,yk) is completed.

[0122] Limited to the logical structure of the circuit, for the current one-dimensional circuit design, the comparison of a two-dimensional substring and a two-dimensional parent string can be to disassemble the substring into each behavior unit, to use each row of the substring as the key item in the entry, and to loop and compare with several rows of entries. As the future, two-dimensional, three-dimensional and high-dimensional logical circuits appear, according to this method, without disassembling the two-dimensional substring and looping, the entire two-dimensional substring is compared once.

[0123] S905 step, according to the comparison results of each row, calculate and output the comparison matching rate of the two-dimensional substring in the two-dimensional parent string.

[0124] S906 step, for the entire two-dimensional substring, shift left or right as a whole, re-execute the steps including but not limited to S903, S904, S905, output the comparison results until the two-dimensional substring shift is completed.

[0125] On the basis of the foregoing technical solutions, for some data with three-dimensional characteristics, such as but not limited to the double helix three-dimensional structure of DNA, high-dimensional structure, in some other schemes of the present application, one or more of the following local improvement measures can be adopted:

[0126] Including but not limited to SA00 multi-dimensional comparison step, specifically including but not limited to:

[0127] SA01 step, according to the three-dimensional arrangement characteristics of the comparison, establishing a three-dimensional mother string S (xs,ys,zs) And a three-dimensional substring K (xk,yk,zk) .

[0128] Wherein XS, YS, ZS are the X, Y, Z axis coordinates of the mother string, XK, YK, ZK are the X, Y, Z axis coordinates of the substring.

[0129] SA02 step, for the three-dimensional mother string and the three-dimensional substring, a dimension reduction step for the third dimension Z is established, and the third dimension Z is used as the recursion or cycle times, and the S900 step is recursively or cyclically performed. When the recursive or cyclic SA00 is successful, the third dimension Z is recursively or cyclically performed, until the recursion or cycle ends, and the comparison result is output.

[0130] SA03 step, for the mother string and the substring of four or more dimensions, recursively or cyclically including but not limited to SA01, SA02, SA03 steps.

[0131] On the basis of the foregoing technical solutions, in some other schemes of the present application, one or more of the following local improvement measures can be adopted:

[0132] Including but not limited to SB00 multi-comparison step, specifically including but not limited to:

[0133] SB01 step, two or more comparison units work cooperatively, wherein the table entry of each comparison unit includes the width and depth of each comparison unit, and the key item of each comparison unit includes the width and depth of each comparison unit.

[0134] SB02 step, each comparison unit includes but is not limited to the respective management, comparison algorithm, mother string and substring, structure, calculation algorithm and output result.

[0135] SB03 step, the intermediate process and output of one comparison unit are used as the input of another comparison unit.

[0136] The multiple alignment is applied in the same system by multiple alignment units, for example, including but not limited to a large table item and a small table item, wherein the large table item unit is used for preliminary selection of some alignment, and the small table item unit is used for accurate alignment or further alignment, which is more suitable for biological sequence, natural language, graph and adaptive alignment.

[0137] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0138] The SC00 management step includes but is not limited to the following steps:

[0139] The SC01 step uses the NP and CPU in the alignment unit to complete modification and perform working steps.

[0140] The SC02 step uses the NP and CPU outside the alignment unit to complete modification and perform working steps.

[0141] The SC03 step provides task decomposition and synthesis, manages and coordinates the access and disconnection between the alignment units, and realizes the step of supporting the connection expansion between independent alignment units.

[0142] The SC04 step modifies and records the logic units and logic definitions of the FPGA chip, programmable logic chip and TCAM chip in more than one alignment unit when initializing the alignment unit.

[0143] The SC05 step modifies and records the logic units and logic definitions of the FPGA chip, programmable logic chip and TCAM chip in more than one alignment unit according to the alignment algorithm when the alignment unit is working.

[0144] The SC06 step executes the steps of SC01 to SC04 according to the properties of alignment.

[0145] The SC07 step provides the rules and protocols for communication and negotiation between the present method and external devices.

[0146] In the present application, the NP and CPU are contained in the unit, which can be independently programmed and managed, and the CPU outside the unit is also included for managing and coordinating the work of each unit, so the present application is a multi-CPU and multi-management system, and the programming, management system and operating system of each CPU can adopt different methods and different systems.

[0147] On the basis of the foregoing technical solutions, in the system structure of the specific product of the present application, one or more of the following partial improvement measures can be adopted:

[0148] As Figure 4 And Figure 5 shown.

[0149] A system of super-parallel comparison, comprising but not limited to one or more comparison units composed of a comparator array and a management module; wherein:

[0150] The comparison unit comprises but is not limited to one or more FPGA chips or one or more programmable logic chips or one or more TCAM chips and their any combination, which are composed of but not limited to table entries and key entries, and the any combination includes but is not limited to the connection of the table entries in the width direction, the depth direction of the table entries by circuit, logical definition, to form a comparator array.

[0151] The logical connection includes but is not limited to the connection defined by NP, CPU, and management work steps.

[0152] The comparator array specifically comprises but is not limited to one or more table entry rows with equal width to form table entries and one or more key entries, which, under the support of the management module, complete the bit-by-bit comparison of the data of one key entry and the data of one or more table entry rows in the row direction at the same time and output the comparison result.

[0153] The management module is composed of the NP, CPU, and management work steps included in the comparison unit, and can also be composed of the CPU and management work steps outside the comparison unit.

[0154] The connection between the one or more comparison units includes but is not limited to the connection of the output of one comparison unit and the input of another comparison unit, and the connection mode includes circuit connection or the logical connection.

[0155] On the basis of the foregoing technical solutions, one or more of the following partial improvement measures can be adopted in other schemes of the present application:

[0156] The comparison module and the management module support comparison data in formats including but not limited to character format and binary format, and perform simultaneous and parallel comparison of each row of the table entries and the key entry in the comparison module and output the address of the table entry row matching the key entry.

[0157] It should be noted that, as a hardware circuit, the comparison is completed in the comparator array, which includes but is not limited to a character comparator array and a binary comparator array, the character comparator array compares characters including but not limited to 8-bit ASCII characters, 16-bit, 32-bit Chinese characters, and color coding characters, and the binary comparator array compares binary bits.

[0158] The comparison module includes, but is not limited to, circuitry and logic to enable simultaneous and parallel comparison of one or more data rows in the key item and table item, and to record and output the address of the table item row that matches the key item.

[0159] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0160] The shift controller includes, but is not limited to, a shift controller for left or right shifting of a specified length for character format-based data, and a shift controller for left or right shifting of a specified length of binary bits for binary format data; the table item and the key item each include, but are not limited to, the shift controller.

[0161] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0162] The reconfigurable logic includes, but is not limited to, table item and key item width reconfigurable logic, table item depth reconfigurable logic, table item row mask bit reconfigurable logic, and key item mask bit reconfigurable logic; the width and the mask bit include, but are not limited to, a number of characters and a number of binary bits.

[0163] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0164] The same or different bit position processor includes, but is not limited to, a processor for calculating the number of same or different bit positions between each table item row in the table item and the corresponding position in the key item, recording the position of the same or different bit positions, and outputting the same or different bit positions during comparison.

[0165] In the specific design, as shown in Figure 3 The register can be used to store the intermediate results for management and use by the management module.

[0166] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0167] The comparison module includes, but is not limited to, filter circuitry or logic to provide hardware-based data operations including, but not limited to, covering, inserting, and deleting different bit positions in the key item and the table item.

[0168] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0169] The comparison module includes but is not limited to a memory, and a circuit or logic of a table entry mapping, and realizes a function of copying storage in an FPGA chip, a programmable logic chip, and a TCAM chip to the storage, and the storage includes but is not limited to a random memory, an electronic hard disk, and a mechanical hard disk.

[0170] On the basis of the foregoing technical solutions, in some other schemes of the application, one or more of the following partial improvement measures can be adopted:

[0171] The system includes but is not limited to a host structure and a daughter card structure, the host structure includes but is not limited to an expansion card slot and one or more comparison modules and one or more management modules, the daughter card structure includes but is not limited to an expansion card and one or more comparison modules and one or more management modules, the expansion card slot and the expansion card can be connected, so that the host can use the expansion card as an extended comparison system, and the expansion card slot and the expansion card include but are not limited to a PCIE interface.

[0172] On the basis of the foregoing technical solutions, in some other schemes of the application, one or more of the following partial improvement measures can be adopted:

[0173] The system includes but is not limited to an external communication module, specifically including but not limited to an Ethernet interface module, a USB interface module, and a special interface module, and the Ethernet interface module, the USB interface module, and the special interface module are connected with the management module. The system also includes but is not limited to a separate device with a shell, a hard disk, and a display, which realizes communication between the system and the outside world. The system also includes but is not limited to a computer built-in expansion card type device with a PCIE interface, which is connected with a PCIE interface inside a computer and connected with an expansion card slot of the host structure.

[0174] On the basis of the foregoing technical solutions, in some other schemes of the application, one or more of the following partial improvement measures can be adopted:

[0175] The system includes but is not limited to a storage module, an auxiliary module, a power module, a connector, and a driver software.

[0176] Advantages

[0177] Compared with the prior art, the application realizes the intended purpose and has the following advantages:

[0178] 1. The comparison speed is greatly improved, which is 10 9 orders of magnitude faster than the CPU comparison of Feng's calculation, and 10 6 orders of magnitude faster than the GPU calculation.

[0179] 2. The comparison time is independent of the length of the comparison data and the complexity of the key.

[0180] 3. Can be used for biological sequence alignment, image recognition, natural language processing. BRIEF DESCRIPTION OF DRAWINGS

[0181] Figure 1 : Alignment unit core structure diagram

[0182] Figure 2 : Biological sequence alignment core structure diagram

[0183] Figure 3 : Similarity and difference site processor structure diagram

[0184] Figure 4 : Alignment unit structure diagram

[0185] Figure 5 : Alignment server structure diagram

[0186] Figure 6 : PCIE acceleration card structure diagram DETAILED DESCRIPTION

[0187] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described below in combination with specific embodiments of the present application and corresponding drawings.

[0188] The specific embodiments of the present application are as follows:

[0189] Embodiment 1: Biological sequence super-parallel alignment server

[0190] I. Module structure diagram explanation

[0191] As Figure 5 is an embodiment of the biological sequence super-parallel alignment server of the present application. In the figure, two alignment units are adopted, wherein the alignment unit 1 is larger, adopts a table item width of 256 bytes, a depth of 1M table item rows, and a single cycle of 1 million table item rows alignment; the alignment unit 2 is smaller, the table item width is 64 bytes, the depth is 16, and the single cycle of 16 table item rows alignment. If it is a SAM type file for biological sequences, the alignment can be performed in byte Byte bit units, at this time, the logical definition of the byte-based alignment unit as shown in Figure 2 is adopted, and Figure 4 the alignment array and Figure 5 also adopt the logical definition of the byte-based alignment array; otherwise, the logical definition in binary bit units is adopted. In addition, the embodiment also includes a management module, a hard disk, a display, an Ethernet interface module, a USB interface module and a special interface module, and also includes a power supply and auxiliary system.

[0192] Figure 4Is the structure diagram of the comparison unit 1 and the comparison unit 2. Including the comparator array, the shift controller, the reconfigurable logic, the management module, the same and different site processor, the filter and the output of the comparator array, also including the memory, the table item mapping, the PCIE and other interfaces.

[0193] The implementation of the server includes the use of FPGA chips and TCAM chips. Among them, when using FPGA chips, the logic definition function can be realized flexibly, for example, the statistics and output of the same and different sites can be realized, and various required functions can be designed, which are mastered by intermediate engineering and technical personnel in the industry and can be completed without innovative work. And using TCAM chips, the comparison array can be efficiently and lowly realized, but as the statistics and output of the same and different sites, due to the structure of the chip itself, it cannot be realized.

[0194] The embodiment is designed as a complete and independent server, including a shell, which can be used as a matching equipment based on a sequencer.

[0195] II. Working step explanation

[0196] It needs to be stated here that: the step numbers described below are only for recording convenience, and do not mean that there is a sequential order of numbering. Unless it is specifically stated that there is a sequential order between certain steps, otherwise it is an arbitrary combination without sequential order.

[0197] A super parallel comparison method, including but not limited to:

[0198] S100 step: including but not limited to the comparison unit composed of table items and key items, completing the super parallel comparison of key item data and table item row data in the table item.

[0199] S200 step: including but not limited to inputting the mother string to the table item, inputting the substring to the key item, cyclic comparison and outputting the result.

[0200] S300 step: including but not limited to generating the selection item of the substring according to the comparison algorithm and inputting it to the table item, cyclically shifting and intercepting the mother string input to the key item, comparing and outputting the result.

[0201] The comparison algorithm includes biological sequence comparison algorithm, natural language comparison algorithm, image comparison algorithm.

[0202] The parent string and the sub-string include a step of changing and compressing data format, the data format includes but is not limited to the SAM format data of biological sequence, the data of natural language and the data of image format, and the data form includes but is not limited to the database format. All operations on the parent string and the sub-string, including but not limited to, the binary code compressed by the Chinese character code, hexadecimal, decimal, octal, ASCII (American Standard Code for Information Interchange: American Standard Code for Information Interchange) are included. The results of the operation type and the operation step are included. The steps of communicating with external network devices, gene sequencing devices, voice devices and image devices through an Ethernet port are included. The comparison service is completed by the processor system and the working step.

[0203] On the basis of the foregoing technical solutions, in other schemes of the application, one or more combinations of the following partial improvement measures can be adopted:

[0204] The S100 step specifically includes but is not limited to:

[0205] The S101 step, the table item includes but is not limited to one or more table item rows, the table item row stores table item row data, the key item stores key data, and the key item data is compared with the data of one or more table item rows, bit by bit, in the same arrangement position in the same logical period. If there is a table item row matching the key item data, output the address of the matched table item row, if there are multiple table item rows matching the key item data, output the address of the first or last matched table item row in the address arrangement order, if there is no table item row matching the key item data, do not output the table item row address.

[0206] As shown in Figure 1 1101 stores the key item data, 1102 is the key item address, 1200 is the table item, which includes n table item rows: 1211 is the first table item row, which stores the first table item row data, 1212 is the address of the first table item row; 1221 is the second table item row, which stores the second table item row data, 1222 is the address of the second table item row; 12n1 is the nth table item row, which stores the nth table item row data, 12n2 is the address of the nth table item row.

[0207] The same logic cycle includes the time length of one key item and more than one table item line to complete a comparison, which can be a combination of several time periods in operation. If the comparison unit is composed of TCAM chips, the single cycle is the comparison cycle of TCAM. For example, the device with a main frequency of 250MHz, the comparison cycle is 4nS. If the comparison unit is composed of FPGA (Field-Programmable Gate Array) chips with a frequency of 500MHz, the comparison cycle is 2nS. The more than one table item line, for example, the designed table item depth is 10 million lines (10M), then the method can complete the comparison of 10 million lines of data with the data of the key line at the same time within 4nS or 2nS. If the 800010th line in the table item line is consistent with the key item data, the address 800010 of the table item line is output.

[0208] As the string comparison algorithm of Von Neumann computer, the CPU usually executes a loop comparison program. According to the estimation of the fastest CPU with a main frequency of 4GHz, the CPU clock cycle is 0.25nS. Since the string comparison algorithm cannot be logically resolved into multi-thread execution, it can only be executed by single thread. It is estimated that the string comparison algorithm with a length of 150 bytes is programmed by using the least time-consuming assembly language. The time cost of one loop is about 40 clock cycles, i.e. 40*0.25nS=10nS. 150 loops take a total of 1.5μS. It takes at least 10000000*1.5μS=15S to complete the comparison of 10 million lines. Compared with 4nS, it is 3.75x10 9 times, and compared with 2nS, it is 7.5x10 9 times. If a high-level language is used for programming, the difference is greater. It can be seen that the comparison method of the present application is amazing in performance improvement, which is fundamentally different from the loop comparison of Von Neumann computer.

[0209] Compared with GPU(Graphic Processing Unit, image processor) acceleration algorithm, CUDA(Compute Unified Device Architecture, unified computing device architecture) contained in GPU can be regarded as a simple small CPU, but this is still the serial loop comparison mode of von Neumann computing. Although CUDA can contain thousands in GPU, on the one hand, in the face of complex comparison data, we cannot design programs for each CUDA to solve the comparison problem; on the other hand, even if we do so, it is still the serial loop comparison process of von Neumann computer. In fact, the design of GPU is driven by the unified processing of image data. For example, operations such as improving image color and gray level. The characteristics of these operations are to do the same operation on all data in the image area, rather than different operations for each pixel. Therefore, the acceleration performance effect of using GPU for data comparison operation is not obvious. There are reports that in the extreme ideal case, the speedup is 10 3 orders of magnitude compared with the CPU method. By analogy, the acceleration of the present application is more than 10 6 orders of magnitude compared with the GPU method.

[0210] The matching refers to the data of a certain table item row and the data of the key item, which are the same in each bit point in the order from left to right or from right to left. For example, the data of the key item is "ATCGGTCCAATG", and the data in a table item row is also "ATCGGTCCAATG". Therefore, the table item row matches the key.

[0211] In step S102, the bit point includes but is not limited to a single binary bit, a character composed of more than one binary bit, and an information unit composed of more than one binary bit. The comparison includes but is not limited to the operation algorithm of the data of the key item and the table item row in the same arrangement position, including but not limited to and, or, not, exclusive or, equal, opposite, not equal, or these combinations. The combination includes the algorithm provided by some formulas including these operations.

[0212] In Figure 1 , the data in the key item and the table item shows binary number, and only 0 and 1. In Figure 2 , the data in the key item and the table item is single-byte ASCII code representing DNA sequence, and can also be multi-byte other encoding, such as Chinese character encoding, image encoding, video encoding. The comparison includes binary bit comparison, single-byte ASCII code comparison, and other encoding comparison.

[0213] For biological data, since the data types are very few, only adenine A, guanine G, cytosine C, thymine T and uracil U, both 1 byte ASCII code and 3 bit bit compression encoding can be used. As for the comparison operation, both byte comparison and bit comparison can be used.

[0214] In step S103, one or more FPGA chips or programmable logic chips are connected to form the table item and the key item according to the logical definition, and the table item and the key item are expanded. The connection modes include but are not limited to the connection in the width direction of the table item or the connection in the depth direction.

[0215] In step S104, the logical definition further includes but is not limited to the definition of the mask of the specified bit position in the table item row and the key item. The mask includes but is not limited to the mask bit and the specified bit. The mask bit makes the bit position not participate in the comparison, and the specified bit replaces the data in the bit position with the specified data.

[0216] As shown in FIG. 21, Figure 2 the mask is marked with # and *, where # is the specified bit and * is the mask bit. Specifically, as shown in 2101, the specified bit A is used to replace the original content, and 2102 and 2103 are set as the mask bits. It should be noted that the setting of the mask bits is not directly replacing the data in 2102 and 2103 with *, but logically operating the signal units in the positions corresponding to 2102 and 2103, and the result forms the mask, for example, using the logical AND relationship to form the corresponding type line to generate a high impedance state, thereby realizing the mask operation. For the TCAM chip, each binary bit in the table item row has a corresponding tri-state mask unit, so it only needs to be set as the mask bit. For the FPGA or other programmable logic chip, the setting is realized by the logical definition.

[0217] In step S105, the logical definition further includes but is not limited to the definition of the NP and CPU in the FPGA chip or the programmable logic chip to complete the specified calculation.

[0218] In step S106, the logical definition further includes but is not limited to the connection of one or more TCAM chips or the combination of the TCAM chip and the FPGA chip to form the table item and the key item, and the table item and the key item are expanded. The connection modes include but are not limited to the connection in the width direction of the table item or the connection in the depth direction.

[0219] In step S107, the logical definition further includes but is not limited to the definition of the tri-state mask bit of the specified bit position in the table item row and the key item in the TCAM chip, so that the bit position does not participate in the comparison.

[0220] In step S108, the logical definition also includes, but is not limited to, defining the NP (Networking Processor) and CPU in the TCAM chip to perform the specified calculations.

[0221] Step S109, the logical definition, also includes single-byte comparison, which is a comparison of each byte individually.

[0222] Based on the foregoing technical solutions, one or more of the following partial improvements can be adopted in other solutions of the present invention:

[0223] The S200 step specifically includes, but is not limited to:

[0224] In step S201, calculate the volume MV of the parent string and the volume TV of the table entry. If MV ≤ TV, the parent string is not decomposed. If MV > TV, the parent string is decomposed or the parent string is decomposed cyclically until the decomposition ends.

[0225] The volume, or total length, is the product of the number of points in the row and the number of columns.

[0226] In step S202, according to the width TW of the table entries, the parent string or the decomposed parent string is cut and extracted sequentially from the beginning, and then input into the table entries in the order of the table entry depth TD.

[0227] For example Figure 2 The table contains rows 1, 2, and so on up to row n.

[0228] Here, the parent string is usually the original target data, such as the DNA sequence of a gene, a piece of natural language text, an image, or a video. The substring is a DNA fragment, a phrase, a small part of an image, or a small part of a video that needs to be searched and compared. The comparison work is to search the parent string to see if there is a substring and where the substring is located.

[0229] In step S203, the width CW of the substring and the width TW of the keyword item are calculated. If CW ≤ TW, a mask is used to compensate the width CW of the substring to TW. If CW > TW, the substring is decomposed or TW is adjusted to TW > CW. The mask includes, but is not limited to, a mask bit and a specified bit. The mask bit prevents the site from participating in the comparison, and the specified bit replaces the site with specified data.

[0230] For example Figure 2 Keyword items in.

[0231] Generally, for the convenience of comparison, the number of bit points of the width of the table entry and the number of bit points of the width of the keyword entry are equal. If the width of the keyword is smaller than the width of the table entry, the keyword entry can be extended by using a mask to extend the keyword entry to be consistent with the width of the table entry row, so that the comparison can be facilitated.

[0232] In step S204, according to the comparison algorithm, more than one specified bit point is inserted or deleted for the table entry row, and the operation and the repair of other table entries in the parent string are recorded.

[0233] For biological sequence comparison, there are operations such as insertion, deletion, and replacement for specific bit points of the parent string. For natural language processing and image video, there are also similar repair operations.

[0234] In fact, there are also similar operations for the keyword entry. In this patent application, this step of operation can be completed when the keyword entry is arranged or in step S300.

[0235] In step S205, the sub-string is input to the keyword entry, the keyword entry is compared with the data of all table entry rows in the table entry, and the address of the matched table entry row is output.

[0236] When the keyword entry is compared with a large number of table entry rows, multiple matched table entry rows often appear. According to the set strategy, the address of the first matched table entry row can be output, or the address of the last matched table entry row can be output.

[0237] In step S206, if there is a matched table entry row address output, the table entry row address is recorded, the data of the table entry row is cleared, and the comparison is looped until the matching is unsuccessful.

[0238] The clearing of the matched table entry row is to clear the data of the table entry row after recording and outputting the address of the matched table entry row. The purpose of this operation is to find out whether there is a next table entry row that matches the keyword entry. Through the loop, all table entry rows that match the keyword entry are found out.

[0239] In step S207, the table entry data is taken as a one-dimensional array, a loop body is executed, the loop body includes but is not limited to left shifting or right shifting one bit point, and the execution includes but is not limited to step S205 and step S206. The comparison result of the loop body is output, and the comparison of one table entry for one keyword entry is ended. The calculation and output result are output.

[0240] Since the width of the table entry row is TW bit points, when all parent strings are connected into a one-dimensional array, only TW-1 times of loop are needed to complete the comparison of all parent strings in the table entry with the sub-string in the keyword entry.

[0241] On the basis of the foregoing technical solutions, one prominent innovation of the present application is to compare the content of a substring with the class content of a mother string in reverse, to pre-combine the possibility of comparison for the substring (for example, including but not limited to the following steps: substring E1, substring E2, substring E3, substring E4), to input the data of these combinations into a table item, to let the mother string flow over a key item in the form of a data stream, one bit point at a time, to compare one table item at a time, and to output the table item row address of possible matching, which has more suitable and more efficient effects, especially for biological sequence comparison and natural language processing. The following one or more local improvement measures can be used:

[0242] The S300 step specifically includes but is not limited to:

[0243] The S301 step is to calculate the width CW of the substring and the table item width TW, if CW≤TW, to extend the width CW of the substring to TW using a mask, and if CW> TW, to decompose the substring or adjust TW to TW≥CW; the mask includes but is not limited to a shield bit and a specified bit, the shield bit makes the bit point not participate in comparison, and the substring after the execution of the present step is substring E1; the specified bit is to replace the data of the bit point with specified data.

[0244] The mask is realized by logic setting in an FPGA chip, a programmable logic chip, and a TCAM chip. The shield bit is to shield the data of the corresponding bit point in the table item row, so that it does not participate in comparison, that is, the data of these bit points does not affect the comparison result; the specified bit is to replace the data of the corresponding bit point during comparison with specified data. The shield bit and the specified bit are both data with specified position information, and attention must be paid during setting.

[0245] The S302 step is to generate one or more substrings E2 from the substring E1 using one or more shield bits to shield one or more specified bit points according to the comparison algorithm, and to record each substring E2 and the shield bit point to a substring mapping table.

[0246] The S303 step is to generate one or more substrings E3 from the substring E1 using one or more specified bits to replace one or more specified bit points according to the comparison algorithm, and to record each substring E3 and the replacement bit point to a substring mapping table.

[0247] The S304 step is to generate one or more substrings E4 from the substring E1 using one or more specified bit points to insert or delete according to the comparison algorithm, and to record each substring E4 and the insertion or deletion bit point to a substring mapping table.

[0248] The S305 step is to input the substrings E2, E3, and E4 to the table item, and to record the addresses of the substrings E2, E3, and E4 to the substring mapping table.

[0249] In fact, for the substring E1, substring E2, substring E3, substring E4 operation, according to the attribute of the comparison, there are other operation modes, here is including but not limited to the relationship.

[0250] S306 step, for the parent string, the pointer SP is established, the parent string is cut from SP=1 according to the substring width CW, and the key item is input, the key item data is compared with the data of all table item rows in the table item, and the comparison result is output.

[0251] S307 step, if there is a matching table item row address output, the table item row address and the pointer SP are recorded, and the substring mapping table is modified.

[0252] S308 step, if there is no address output, modify the pointer, assign SP+1 to SP, and execute the steps including but not limited to S305 and S306, until the parent string ends, and the output result is calculated.

[0253] The above operation includes but is not limited to loop and recursion, and also includes another loop and recursion operation for a specific interval to realize multiple scanning comparison and step-by-step optimization effect.

[0254] On the basis of the foregoing technical solutions, in order to better support the comparison operation, a reconfigurable method of changing the table item structure is designed, mainly modifying the width and depth of the table item. Specifically, one or more of the following local improvement measures are adopted:

[0255] Including but not limited to S400 table item reconfigurable step, specifically including but not limited to:

[0256] S401 step, the width of the longest substring EM is calculated according to the following formula: TW0=1+MOD(CW-MAX / 8), wherein: MOD is the integer function, TW0 is the reconfigurable table item width, and CW-MAX is the width of the longest substring EM.

[0257] S402 step, according to TW0, the TD0=TV / TW0 is calculated under the premise that the table item volume TV is unchanged, and TD0 is taken as the table item depth after reconstruction.

[0258] In fact, here the decimal in TD0 is processed by cutting, that is, if it is not an integral row, the row is removed.

[0259] S403 step, according to TW0 and TD0, the comparison parameters are modified, and the comparison operation is executed.

[0260] The comparison parameters include modifying the width TW0 and the depth TD0 of the substring in the upper layer software.

[0261] It is noted here that when the substring appears with a depth greater than 1, i.e. the substring is truncated, the substring in this stage needs to be connected in the upper comparison software for calculation. However, in general, the substring is not truncated, unless the substring is too long.

[0262] In the S404 step, the NP, CPU system in the FPGA chip, programmable logic chip and / or TCAM chip is used to execute and manage the steps including but not limited to S401, S402 and S403.

[0263] Here is a method designed to improve the truncation of the substring.

[0264] On the basis of the foregoing technical solutions, the present application designs a method for counting the same and different sites during comparison. This can calculate the relevant probability for the comparison and retrieval of biological sequences and natural language processing. Specifically, one or more of the following local improvements can be used:

[0265] The S500 same and different site processing step includes but is not limited to the following steps:

[0266] In the S501 step, the logic definition further includes but is not limited to defining the logic unit of the FPGA chip, programmable logic chip and TCAM chip to complete the calculation of the same number TW-YES and different number TW-NO of the data in each site of the key item and each table row in the table item during comparison, including the calculation of probability and the position of the site.

[0267] In the S502 step, the TW-YES and TW-NO of each table row in the table item are selected to output the maximum or minimum value and the row number of the table row where the maximum or minimum value is located.

[0268] In the S503 step, the maximum or minimum value and the data of the table row where the maximum or minimum value is located are cleared in a loop based on the S502 step, and the S501 step and the S502 step are re-executed until the TW-YES and TW-NO values reach the specified value.

[0269] This is a calculation method designed to find the maximum or minimum probability of matching or not matching in the table item.

[0270] In the S504 step, the logic definition further includes but is not limited to defining the hardware logic to execute the steps of S501, S502 and S503.

[0271] The definition of hardware logic includes the logic unit and unit group of the FPGA chip, programmable gate array chip and TCAM chip.

[0272] S505, using the NP in the FPGA chip, programmable logic chip, TCAM chip, CPU system, using software, performing the steps including but not limited to S501, S502, S503.

[0273] S506, using the NP in the FPGA chip, programmable logic chip, TCAM chip, CPU system, sorting and outputting the TW-YES and TW-NO of each table row in the table item.

[0274] As shown in Figure 3 , this is a principle verification example designed for binary bits. Figure 3 In the above technical solutions, on the basis of the above technical solutions, in some other schemes of the present application, one or more of the following partial improvement measures can be used, which is an effective method for repairing and filtering related noise when including but not limited to biological sequence alignment and natural language alignment operations:

[0275] including but not limited to the step of filtering S600, specifically including but not limited to:

[0276] S601, when performing alignment of the key item and the table row, covering, inserting and deleting different bits, same bits, and recording the length of the covered, inserted and deleted different bits.

[0277] S602, when performing alignment of the key item and the table row, covering, inserting, replacing and deleting continuous different bits of a specified length, continuous same bits of a specified length, and recording the length of the covered, inserted, replaced and deleted different bits.

[0278] S603, between performing multiple alignments, according to the alignment algorithm, performing the mask processing of the mask bits and the specified bits for the data of the table item and the key item.

[0279] S603, between performing multiple alignments, according to the alignment algorithm, performing the mask processing of the mask bits and the specified bits for the data of the table item and the key item.

[0280] S604, between the multiple comparisons, according to the comparison algorithm, the data of the table item and the key item are processed by covering, inserting, replacing, deleting and filtering.

[0281] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0282] The S700 table item mapping step includes but is not limited to the following steps:

[0283] S701, in addition to the FPGA chip, programmable logic chip and TCAM chip, a storage area is established by using a memory, corresponding to the table item, and the data and intermediate data, result data of the table item and the key item are copied to the storage area to become the mapping of the table item.

[0284] S702, the operation and modification record of the table item and the key item are synchronized to the mapping.

[0285] S703, the mapping is stored in a database, and is processed by the NP, CPU system in the FPGA chip, programmable logic chip and TCAM chip or the NP, CPU system outside the FPGA chip, programmable logic chip and TCAM chip.

[0286] The purpose of this method is to establish a backup to play a role in subsequent backtracking of intermediate data and processes. In fact, in some cases, this mapping can be stored in a database including electronic hard disks and mechanical hard disks for subsequent analysis.

[0287] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0288] The S800 multiple substring unordered comparison step includes but is not limited to the following steps: n , and it is not necessary to check and compare KU1 to KU n in the parent string in order.

[0289] S801, it is determined that the substring KU includes multiple substrings KU1 to KU n , and it is determined that KU1 to KU n have no order.

[0290] S802, the search range in the parent string is determined, the content of the range of the parent string is input to a designated comparison unit table item, and KU1 to KU nFor the cycle range, KU1 to KU are input in turn at each cycle n The keyword item to the designated comparison unit is executed, including but not limited to S300 steps, and the output is recorded.

[0291] S803 steps, if all KU1 to KU n are matched successfully, the cycle ends, and the KU matching successful data is output, if KU1 to KU n There are matching failures, output KU matching failure, and exit the cycle.

[0292] The multiple substrings refer to a substring including several pieces of data that need to be compared, including but not limited to some specific small fragments in a piece of DNA. These fragments may not need to have a strict order, in which case the S800 step is suitable for this comparison operation.

[0293] On the basis of the foregoing technical solutions, for some data with two-dimensional characteristics, such as but not limited to DNA double helix structure data, images, and video data, in some other schemes of the application, one or more of the following local improvement measures can be used:

[0294] Including but not limited to S900 two-dimensional comparison steps, specifically including but not limited to:

[0295] S901 step, according to the two-dimensional arrangement characteristics of the comparison, establish two-dimensional mother string S (xs,ys) And two-dimensional substring K (xk,yk) .

[0296] Wherein, XS, YS are the two-dimensional row and column coordinates of the mother string, XK, YK are the two-dimensional row and column coordinates of the substring. It should be noted that as two-dimensional data, if shifting is required, the entire two-dimensional arrangement must be shifted as a whole, such as left shift, right shift, up shift, and down shift, and cannot move only one row or one column.

[0297] It is particularly emphasized here that as the comparison unit, the entries therein can only be one-dimensional truncation, and the key items therein are also one-dimensional. The two-dimensional comparison to multi-dimensional comparison of the present application is actually for the parent string and the substring. All dimensions higher than the one-dimensional dimension of the comparison unit are completed by the preparation step outside the input and output comparison unit. For example, the data comparison of the double helix structure and folding structure of DNA, and the context association of three-dimensional image data and natural language, before entering the comparison unit and before, the data is decomposed into one-dimensional entry and one-dimensional key item structure suitable for the comparison unit by the software and steps of the present application, and then compared in the comparison unit. That is, before entering the comparison unit, the data needs to be reduced to one dimension, and then compared in the comparison unit. The result of the comparison is increased in dimension to restore the original dimension. Once recursion or loop, the two-dimensional to multi-dimensional comparison operation is completed.

[0298] S902, input the two-dimensional parent string to the entry.

[0299] S903, copy a row of data in the two-dimensional substring K (xk,yk) to the key item and perform comparison in sequence, execute S500 difference and same site processing step, and output the comparison result of each row in the two-dimensional comparison.

[0300] S904, copy the next row of data in the two-dimensional substring K (xk,yk) to the key item and perform comparison in a loop until the comparison of all rows in the two-dimensional substring K (xk,yk) is completed.

[0301] Limited to the logical structure of the circuit, for the current one-dimensional circuit design, the comparison of a two-dimensional substring and a two-dimensional parent string can be to disassemble the substring into each behavior unit, take each row of the substring as the key item in the entry, and compare with several rows of entries in a loop. As the future, two-dimensional, three-dimensional and high-dimensional logical circuits appear, according to this method, without disassembling the two-dimensional substring and looping, the entire two-dimensional substring is compared once.

[0302] S905, according to the comparison result of each row, calculate and output the comparison matching rate of the two-dimensional substring in the two-dimensional parent string.

[0303] S906, for the entire two-dimensional substring, re-execute the steps including but not limited to S903, S904, S905 by moving left or right as a whole, output the comparison result until the two-dimensional substring shifting is completed.

[0304] On the basis of the foregoing technical solutions, for some data with three-dimensional characteristics, such as but not limited to the double helix three-dimensional structure of DNA and high-dimensional structure, in some other schemes of the application, one or more of the following local improvement measures can be adopted:

[0305] The SA00 multi-dimensional comparison step includes but is not limited to the following steps:

[0306] The SA01 step is to establish a three-dimensional mother string S (xs,ys,zs) and a three-dimensional substring K (xk,yk,zk) according to the three-dimensional arrangement characteristics of the comparison.

[0307] Wherein XS, YS and ZS are the X, Y and Z axis coordinates of the mother string, and XK, YK and ZK are the X, Y and Z axis coordinates of the substring.

[0308] The SA02 step is to establish a dimension reduction step for the third dimension Z for the three-dimensional mother string and the three-dimensional substring, and to recursively or cyclically perform the S900 step with the third dimension Z as the recursion or cycle number. When the recursive or cyclic SA00 is successful, the third dimension Z is recursively or cyclically performed until the recursion or cycle ends, and the comparison result is output.

[0309] The SA03 step is to recursively or cyclically perform the steps including but not limited to SA01, SA02 and SA03 for the mother string and the substring of four or more dimensions.

[0310] On the basis of the foregoing technical solutions, in some other schemes of the application, one or more of the following local improvement measures can be adopted:

[0311] The SB00 multi-comparison step includes but is not limited to the following steps:

[0312] The SB01 step is to cooperatively work between two or more comparison units, wherein the table entry of each comparison unit includes the width and depth thereof, and the key entry of each comparison unit includes the width and depth thereof.

[0313] The SB02 step includes but is not limited to the respective management, comparison algorithm, mother string and substring, structure, calculation algorithm and output result of each comparison unit.

[0314] The SB03 step is to use the intermediate process and output of one comparison unit as the input of another comparison unit.

[0315] The multiple alignment is applied in the same system by multiple alignment units, for example, including but not limited to a large table item and a small table item, wherein the large table item unit is used for preliminary selection of some alignment, and the small table item unit is used for accurate alignment or further alignment, which is more suitable for biological sequence, natural language, graph and adaptive alignment.

[0316] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0317] The SC00 management step includes but is not limited to the following steps:

[0318] The SC01 step uses the NP and CPU in the alignment unit to complete modification and perform working steps.

[0319] The SC02 step uses the NP and CPU outside the alignment unit to complete modification and perform working steps.

[0320] The SC03 step provides task decomposition and synthesis, manages and coordinates access and disconnection between the alignment units, and realizes the step of supporting connection expansion between independent alignment units.

[0321] The SC04 step modifies and records the logic unit and logic definition of the FPGA chip, programmable logic chip and TCAM chip in more than one alignment unit when the alignment unit is initialized.

[0322] The SC05 step modifies and records the logic unit and logic definition of the FPGA chip, programmable logic chip and TCAM chip in more than one alignment unit according to the alignment algorithm when the alignment unit is working.

[0323] The SC06 step executes the steps of SC01 to SC04 according to the properties of alignment.

[0324] The SC07 step provides the rules and protocols for communication and negotiation between the present method and external devices.

[0325] In the present application, since the NP and CPU are contained in the unit, they can be independently programmed and managed, and the CPU outside the unit is also included for managing and coordinating the work of each unit, so the present application is a multi-CPU and multi-management system, and the programming, management system and operating system of each CPU can adopt different methods and different systems.

[0326] On the basis of the foregoing technical solutions, in the system structure of the specific product of the present application, one or more of the following partial improvement measures can be adopted:

[0327] As shown in Figure 4 And Figure 5 .

[0328] A system of super-parallel comparison, comprising but not limited to one or more comparison units composed of a comparator array and a management module; wherein:

[0329] The comparison unit comprises but is not limited to one or more FPGA chips or one or more programmable logic chips or one or more TCAM chips and their any combination, which are composed of but not limited to table entries and key entries, and the any combination includes but is not limited to the connection of the table entries in the width direction, the depth direction of the table entries by circuit, logical definition, to form a comparator array.

[0330] The logical connection includes but is not limited to the connection defined by NP, CPU, and management work steps.

[0331] The comparator array specifically comprises but is not limited to one or more table entry rows with equal width to form table entries and one or more key entries, which, under the support of the management module, complete the bit-by-bit comparison of the data of one key entry and the data of one or more table entry rows in the row direction at the same time and output the comparison result.

[0332] The management module is composed of the NP, CPU, and management work steps included in the comparison unit, and can also be composed of the CPU and management work steps outside the comparison unit.

[0333] The connection between the one or more comparison units includes but is not limited to the connection of the output of one comparison unit and the input of another comparison unit, and the connection mode includes circuit connection or the logical connection.

[0334] On the basis of the foregoing technical solutions, one or more of the following partial improvement measures can be adopted in other schemes of the present application:

[0335] The comparison module and the management module support comparison data in formats including but not limited to character format and binary format, and perform simultaneous and parallel comparison of each row of the table entries and the key entry in the comparison module and output the address of the table entry row matching the key entry.

[0336] It should be noted that, as a hardware circuit, the comparison is completed in the comparator array, which includes but is not limited to a character comparator array and a binary comparator array, the character comparator array compares characters including but not limited to 8-bit ASCII characters, 16-bit, 32-bit Chinese characters, and color coding characters, and the binary comparator array compares binary bits.

[0337] The comparison module includes, but is not limited to, circuitry and logic to enable simultaneous and parallel comparison of one or more data rows in the key item and table item, and to record and output the address of the table item row that matches the key item.

[0338] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0339] The shift controller includes, but is not limited to, a shift controller for left or right shifting a specified length of character format-based data, and a shift controller for left or right shifting a specified length of binary bits of binary format-based data; the table item and the key item each include, but are not limited to, the shift controller.

[0340] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0341] The reconfigurable logic includes, but is not limited to, table item and key item width reconfigurable logic, table item depth reconfigurable logic, table item row mask bit reconfigurable logic, and key item mask bit reconfigurable logic; the width and the mask bit include, but are not limited to, the number of characters and the number of binary bits.

[0342] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0343] The same / different bit position processor includes, but is not limited to, a processor for calculating the number of same / different bit positions between each table item row in the table item and the corresponding position in the key item, recording the positions of the same / different bit positions, and outputting the same / different bit positions.

[0344] In the specific design, as shown in Figure 3 The register is used to store the intermediate results for management and use by the management module.

[0345] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0346] The comparison module includes, but is not limited to, filter circuitry or logic to provide hardware-based data operations including, but not limited to, covering, inserting, and deleting different bit positions of the key item and the table item.

[0347] On the basis of the foregoing technical solutions, in some other solutions of the present application, one or more of the following partial improvement measures can be adopted:

[0348] The comparison module includes but is not limited to a memory, and a circuit or logic of table entry mapping, and realizes the function of copying the storage in the FPGA chip, programmable logic chip, TCAM chip to the storage.

[0349] On the basis of the foregoing technical solutions, in some other schemes of the application, one or more of the following partial improvement measures can be adopted:

[0350] The system includes but is not limited to a host structure and a daughter card structure, the host structure includes but is not limited to an expansion slot and one or more comparison modules and one or more management modules, the daughter card structure includes but is not limited to an expansion card and one or more comparison modules and one or more management modules, the expansion slot and the expansion card can be connected, so that the host can use the expansion card as an extended comparison system, and the expansion slot and the expansion card include but are not limited to a PCIE interface.

[0351] On the basis of the foregoing technical solutions, in some other schemes of the application, one or more of the following partial improvement measures can be adopted:

[0352] It includes but is not limited to an external communication module, specifically including but not limited to an Ethernet interface module, a USB interface module, and a special interface module, the Ethernet interface module, the USB interface module, and the special interface module are connected with the management module. It also includes but is not limited to a separate device with a shell, a hard disk, and a display, which realizes the communication between the system and the outside world. It also includes but is not limited to a computer built-in expansion card type device with a PCIE interface, which is connected with the internal PCIE interface of the computer and the expansion slot of the host structure.

[0353] On the basis of the foregoing technical solutions, in some other schemes of the application, one or more of the following partial improvement measures can be adopted:

[0354] It includes but is not limited to a storage module, an auxiliary module, a power module, a connector, and a driver software.

[0355] Specific embodiment two: PCIE card type super-speed comparison system

[0356] As shown in the figure, this embodiment is another application of the application, a PCIE card type super-speed comparison system. The same as the previous embodiment will not be repeated here, and only the different parts will be emphasized here. Figure 6

[0357] 1. A TCAM chip or FPGA is used to design a PCIE card type super-speed comparison system. The system includes a comparator array, a management module, a shift controller, reconfigurable logic, a memory and table entry mapping, and a PCIE interface. ​

[0358] 2. This system needs to be inserted into a PCIe slot to run and does not have the ability to run independently.

[0359] 3. This system includes a comparison unit, which uses the CPU and program design built into the TCAM chip as the management module of this system.

[0360] 4. The management module of this system has a management interface designed for the host PC.

[0361] 5. The table entries in this system are 128 bytes wide and 1M rows deep.

[0362] Specific Implementation Example 3: General Data High-Speed ​​Comparison System

[0363] like Figure 6 As shown, this embodiment is another application of the present invention—a general-purpose data high-speed comparison system. The same features as the previous embodiment will not be repeated here; only the differences will be emphasized.

[0364] 1. Figure 4 The alignment array uses a bit-alignment-based logical definition, such as... Figure 1 As shown.

[0365] Therefore, for step S109, the logical definition states that the site is a bit, and the comparison is performed bit by bit.

[0366] 2. For image data comparison, the width of the table entries should be appropriate for the image size. For example, for a 4K RGB three-primary-color video image format, when each frame has 3840*2160 pixels, the table entry width is determined to be 3840*3*24bits=270Kbits, and the table entry depth is 2160*3*24bits=151875bits=152Kbits. The table entry depth can also be increased, and each table entry can store multiple images. For example, for the NTSC video standard with 30 progressive scan frames per second, storing 1 second of the frame would require 30*152Kbits=4560 bits.

[0367] 3. For comparison of natural language data, the width of the table entries should be suitable for the size of the natural language. For example, GBK encoded Chinese characters use double-byte encoding, with each position using 16 bits of double bytes. Assuming each table entry row contains 100 Chinese characters, the table entry width is 100 * 16 bits = 1.6K bits. As for the depth of the table entries, it can be designed as needed.

[0368] 4. Based on the actual content of the comparison and the width and depth of the entries, modify S900 to SC00 so that the logical definition supports two-dimensional, three-dimensional and multi-dimensional comparison.

[0369] 5. Based on all the comparison steps, design comparison functions to adapt to the specific comparison steps, so that the upper-level language can compile and call these functions to complete the comparison business.

[0370] 6. Employ cloud computing, big data, and artificial intelligence algorithms to further filter data, use deep learning, and reason and judge the comparison results.

Claims

1. A method of super-parallel matching, comprising: S100: forming a matching unit by a table item and a key item, and performing super-parallel matching of key item data with table item row data in the table item; S200: inputting a mother string to the table item, inputting a substring to the key item, synchronously matching the key item with all table item rows in the table item and outputting a matching address, clearing matched table item row data, and performing a shift cycle matching of table item data in a one-dimensional array until full-bitpoint matching is completed and a result is output.

2. The method of claim 1, wherein, The S100 step specifically comprises: S101: the table item comprises more than one table item row, the table item row stores table item row data, the key item stores key data, the key item data is matched with data of more than one table item row, data of the same bitpoint in the same logical period and the same arrangement position are simultaneously matched, if there is a table item row matched with the key item data, an address of the matched table item row is output, if there are multiple table item rows matched with the key item data, an address of the first or last matched table item row in the address arrangement order is output, and if there is no table item row matched with the key item data, no table item row address is output; S102: the bitpoint comprises a single binary bit, a character comprising more than one binary bit, and an information unit comprising more than one binary bit, and the matching comprises operations of and, or, not, exclusive or, equal, opposite, not equal, or a combination thereof on data of the same bitpoint in the same arrangement position of the key item and the table item row; S103: one or more FPGA chips and / or programmable logic chips are connected to form the table item and the key item according to a logical definition, the table item and the key item are expanded, and the connection mode comprises connection in a width direction and / or a depth direction of the table item; or S104: the logical definition further comprises definition of a mask of a specified bitpoint in the table item row and / or the key item, the mask comprises a shielding bit and a specified bit, the shielding bit makes the bitpoint not participate in the matching, and the specified bit is replaced by specified data; or S105: the logical definition further comprises definition of an NP and / or a CPU in the FPGA chip and / or the programmable logic chip to complete a specified calculation function; or S106: one or more TCAM chips or a combination of a TCAM chip and an FPGA chip are connected to form the table item and the key item according to a logical definition, the table item and the key item are expanded, and the connection mode comprises connection in a width direction and / or a depth direction of the table item; or S107: the logical definition further comprises definition of a tri-state shielding bit of a specified bitpoint in the table item row in the TCAM chip and / or the key item, which makes the bitpoint not participate in the matching; or S108: the logical definition further comprises definition of an NP and / or a CPU in the TCAM chip to complete a specified calculation function. The step S109, the logic definition further comprises defining bit comparison, single-byte comparison and double-byte comparison based on the different bit positions, the bit comparison is bit-by-bit comparison in bit unit, the single-byte comparison is byte-by-byte comparison in single-byte unit, and the double-byte comparison is double-byte-by-double-byte comparison in two-byte unit.

3. The method of claim 1, wherein, The step S200 specifically comprises: The step S201, calculating the volume MV of the mother string and the volume TV of the table entry, if MV≤TV, the mother string is not decomposed, if MV>TV, the mother string is decomposed or cyclically decomposed until the decomposition is completed; The step S202, cutting the mother string or the decomposed mother string from the beginning according to the width TW of the table entry, and inputting to the table entry in sequence according to the depth TD of the table entry; The step S203, calculating the width CW of the substring and the width TW of the key entry, if CW≤TW, using a mask to compensate the width CW of the substring to TW, if CW>TW, decomposing the substring or adjusting TW to TW>CW; the mask comprises a shielding bit and a specified bit, the shielding bit makes the bit position not participate in comparison, and the specified bit is replaced by a specified data at the bit position; The step S204, according to the comparison algorithm, inserting and / or deleting more than one specified bit position in the table entry row, and recording the insertion and / or deletion operation and repairing other table entries in the mother string; The step S205, inputting the substring to the key entry, comparing the key entry with the data of all table entry rows in the table entry at the same time, and outputting the address of the matched table entry row; The step S206, if there is a matched table entry row address output, recording the table entry row address, clearing the table entry row data, and cyclically comparing until the matching is unsuccessful; The step S207, regarding the table entry data as a one-dimensional array, executing a loop body, the loop body comprises left shifting or right shifting one bit position, executing the step S205 and the step S206, until the loop body is executed TW-1 times, outputting the comparison result of the loop body, and ending the comparison of one table entry to one key entry.

4. The method of claim 2, wherein, The step S400 of reconstructing the table entry specifically comprises: The step S401, calculating the width of the longest substring EM according to the following formula: TW0=1+MOD(CW-MAX / 8), wherein MOD is an integral function, TW0 is the width of the reconstructable table entry, and CW-MAX is the width of the longest substring EM; The step S402, calculating TD0=TV / TW0 as the depth of the reconstructed table entry on the premise that the volume TV of the table entry is unchanged according to TW0; The step S403, modifying the parameters of the comparison according to TW0 and TD0, and executing the comparison operation; The step S404, using the NP, CPU system in the FPGA chip, programmable logic chip and / or TCAM chip to execute and manage the steps S401, S402 and S403.

5. The method of claim 2, wherein, The step S500 of processing the same and different bit positions specifically comprises: S501, the logic definition further comprises defining the logic unit of the FPGA chip, programmable logic chip and / or TCAM chip, to complete the calculation of the same number TW-YES and different number TW-NO of data at each bit of the key item and each bit of the table item in the comparison, including calculating the probability, the position of the bit; S502, for each table item in the table item, select the maximum or minimum value of TW-YES and TW-NO, and output the maximum or minimum value and the row number of the table item; S503, for S502, clear the maximum or minimum value and the data of the table item, and re-execute S501 and S502 until the value of TW-YES and TW-NO reaches a specified value; S504, the logic definition further comprises defining the hardware logic to complete the steps of S501, S502 and S503; and / or, S505, using the NP, CPU system in the FPGA chip, programmable logic chip and / or TCAM chip, using software to execute the steps of S501, S502 and S503; and / or, S506, using the NP, CPU system in the FPGA chip, programmable logic chip and / or TCAM chip, sorting and outputting TW-YES and TW-NO of each table item in the table item.

6. The method of claim 2, wherein, S600, the filter filtering step, specifically including: S601, during the comparison of the key item and the table item, covering, inserting and deleting different bits and / or same bits, and recording the length of the covered, inserted and deleted different bits; or, S602, during the comparison of the key item and the table item, covering, inserting, replacing and deleting continuous different bits and / or continuous same bits of a specified length, and recording the length of the covered, inserted, replaced and deleted different bits; or, S603, between multiple comparisons, according to the comparison algorithm, performing mask processing on the shield bits and specified bits of the table item and the key item; or, S604, between multiple comparisons, according to the comparison algorithm, performing covering, inserting, replacing, deleting and filtering processing on the data of the table item and the key item.

7. The method of claim 2, wherein, S700, the table item mapping step, specifically including: S701, outside the FPGA chip, programmable logic chip and / or TCAM chip, using a memory to establish a storage area corresponding to the table item, copying the data and intermediate data, result data of the table item and the key item to the storage area, becoming the mapping of the table item; and / or, S702, synchronizing the operation and modification record of the table item and the key item to the mapping; and / or, S703, storing the mapping to a database, processed by the NP, CPU system in the FPGA chip, programmable logic chip and / or TCAM chip, or the NP, CPU system outside the FPGA chip, programmable logic chip and / or TCAM chip.

8. The method of claim 7, wherein, S900, two-dimensional comparison step, specifically comprising: S901 step, according to the two-dimensional arrangement of the features of the comparison, the establishment of two-dimensional mother string S （xs,ys） And two-dimensional substring K （xk,yk） ; S902, inputting the two-dimensional parent string into the table entry; S903 step, in the two-dimensional substring K （xk,yk） In the two-dimensional comparison, a line of data is copied to the keyword item one by one and compared, the S500 difference and same site processing step is executed, and the comparison results of each line in the two-dimensional comparison are output. S904, copying and comparing the next row of data in the two-dimensional substring K （xk,yk） to the keyword item until all rows in the two-dimensional substring K （xk,yk） are compared. S905, calculating and outputting the matching rate of the two-dimensional child string in the two-dimensional parent string according to the comparison results of each row; S906, shifting the two-dimensional child string left or right as a whole, re-executing S903, S904 and S905, and outputting the comparison result until the shifting of the two-dimensional child string is completed.

9. The method of claim 8, wherein, SA00, multi-dimensional comparison step, specifically comprising: SA01, establishing a three-dimensional parent string S(xs, ys, zs) and a three-dimensional child string K(xk, yk, zk) according to the three-dimensional arrangement characteristics of the comparison; SA02, for the three-dimensional parent string and the three-dimensional child string, establishing a dimension reduction step for the third dimension Z, recursively or circularly executing S900 for the third dimension Z as the number of recursions or circulations, and recursively or circularly executing SA00 when the recursion or circulation is successful, and recursively or circularly executing SA00 until the recursion or circulation is completed, and outputting the comparison result; SA03, for parent strings and child strings with more than four dimensions, recursively or circularly executing the steps of SA01, SA02 and SA03.

10. The method of claim 8, wherein, SB00, multi-comparison step, specifically comprising: SB01, two or more comparison units work cooperatively, wherein the table entry of each comparison unit includes its own width and depth, and the key entry of each comparison unit includes its own width and depth; or SB02, each comparison unit includes its own management, comparison algorithm, parent string and child string, structure, calculation algorithm and output result; or SB03, the intermediate process and / or output of one comparison unit is used as the input of another comparison unit.

11. The method of claim 9, wherein, SC00, management step, specifically comprising: SC01, using the NP, CPU in the comparison unit to modify and execute its working steps; SC02, using the NP, CPU outside the comparison unit to modify and execute its working steps; SC03, providing task decomposition and synthesis, managing and coordinating the access and disconnection between the comparison units, and realizing the connection expansion support between independent comparison units; SC04, modifying and recording the logic unit and logic definition of the FPGA chip, programmable logic chip and / or TCAM chip in one or more comparison units when initializing the comparison unit; SC05, modifying and recording the logic unit and logic definition of the FPGA chip, programmable logic chip and / or TCAM chip in one or more comparison units according to the comparison algorithm when the comparison unit is working; SC06, executing the steps of SC01 to SC04 according to the properties of the comparison; SC07, a step for providing rules and protocols for the communication and negotiation between the method and external devices.

12. A system for super-parallel comparison, comprising one or more comparison units each comprising a comparator array and a management module; wherein: the comparison unit comprises one or more FPGA chips or one or more programmable logic chips or one or more TCAM chips and any combination thereof, which are connected by circuits and / or defined by logic to form a comparator array comprising table entries and key entries, and the any combination comprises a connection and / or a logical definition in the width direction of the table entries and / or in the depth direction of the table entries; the logical definition comprises a connection defined by the NP, CPU and / or management work steps; the comparator array specifically comprises one or more rows of table entries with equal width and one or more key entries, and under the support of the management module, the data of one key entry and the data of one or more rows of table entries are compared bit by bit in the row direction at the same time, and the comparison result is outputted; the management module is formed by the NP, CPU and / or management work steps included in the comparison unit, and can also be formed by the CPU and management work steps outside the comparison unit; the connection between the one or more comparison units comprises the connection between the output of one comparison unit and the input of another comparison unit, and the connection mode comprises a circuit connection or the logical definition; the work of the management module at least includes: inputting a mother string into the table entries, inputting a substring into the key entries, synchronously comparing the key entries with all rows of table entries in the table entries and outputting the matching addresses, clearing the data of the matched table entry rows, and performing a shift cycle comparison on the table entry data in the form of a one-dimensional array until the full-bit comparison is completed and the result is outputted.

13. The system according to claim 12, wherein: the comparison unit and the management module support comparison data in character format and binary format, and simultaneously compare each row of the table entries in the comparison unit and the key entries at the same time and output the addresses of the table entry rows matching the key entries; the comparison unit comprises circuits and logic for simultaneously comparing one or more rows of data in the key entries and the table entries at the same time, recording and outputting the addresses of the table entry rows matching the key entries.

14. The system of claim 12, wherein shift controllers, specifically shift controllers for left or right shifting a specified length of data based on character format, and shift controllers for left or right shifting a specified number of binary bits of data based on binary format; the table entries and the key entries each comprise the shift controllers.

15. The system of claim 12, wherein reconfigurable logic, at least including: table entry and key entry width reconfigurable logic, table entry depth reconfigurable logic, table entry row mask bit reconfigurable logic, and key entry mask bit reconfigurable logic; the width and the mask bit include a number based on characters and a number based on binary bits.

16. The system of claim 12, wherein same / different bit position processors, specifically processors for calculating the number of same / different bit positions between each table entry row in the table entries and the corresponding positions in the key entries, recording the positions of the same / different bit positions and outputting them.

17. The system of claim 12, wherein The comparison unit includes filter circuit or logic, providing hardware-based data operation including covering, inserting and deleting different bit points for keyword items and table items.

18. The system according to claim 12, characterized in that... The comparison unit includes memory and table item mapping circuit or logic, realizing the function of copying the storage in FPGA chip, programmable logic chip and / or TCAM chip to the storage, and the memory includes random memory, electronic hard disk and mechanical hard disk.

19. The system of claim 12, wherein The system includes host structure and sub-card structure, the host structure includes expansion card slot and more than one comparison unit and more than one management module, the sub-card structure includes expansion card and more than one comparison unit and more than one management module, the expansion card slot and the expansion card can be connected, so that the host can use the expansion card as an extended comparison system, and the expansion card slot and the expansion card include PCIE interface.

20. The system of claim 12, wherein It includes external communication module, specifically at least including Ethernet interface module and / or USB interface module and / or special interface module, and the Ethernet interface module and / or USB interface module and / or special interface module are connected with the management module; it also includes independent device with shell, hard disk and display, realizing the communication between the system and the outside world; or, it also includes PCIE interface computer built-in expansion card type device, connected with the internal PCIE interface of the computer or connected with the expansion card slot of the host structure.

Images