High-performance hash table implementation method suitable for FPGA (Field Programmable Gate Array)

Abstract

The invention discloses a high-performance hash table implementation method suitable for an FPGA (Field Programmable Gate Array), and a hash algorithm comprises the following steps: step 1, a hash table establishment stage: storing a hash table in an off-chip DDR (Double Data Rate) memory of the FPGA, and storing actual data of the hash table in each row in the table; step 2, a hash table establishment stage: storing an index table in an RAM (Random Access Memory) in an FPGA chip, storing one bit of information in each address in the table, and marking whether data is written into a corresponding address in the hash table by the information, 0 is not written into the corresponding address, and 1 is written into the corresponding address in the hash table; 3, in the query stage of the hash table, firstly querying the index table, and if the corresponding address mark in the index table is queried to be 0, indicating that no data is written into the hash table, the DDR memory is not accessed and the hash table is directly returned; and if the corresponding identifier in the index table is queried to be 1, searching in the DDR memory. According to the method, the frequency of accessing the off-chip DDR memory can be effectively reduced, so that the query efficiency of the FPGA large-scale hash table is improved.

Description

Technical Field

[0001] The invention belongs to the field of computer system structure, and in particular relates to a high-performance hash table implementation method suitable for FPGA. Background Art

[0002] Hash tables are widely used in search engines, virus protection, cloud computing and other fields, and the use of FPGAs for computational offloading and acceleration in these fields is the direction of computer system technology development. The characteristic of FPGA is that there are only a limited number of RAM storage resources on the chip, and large-capacity storage can be expanded through DDR off-chip. FPGA can access the on-chip RAM at high speed, but the efficiency of accessing DDR memory is low, especially when reading or writing small amounts of data. When building a large-scale hash table based on FPGA, the hash table is generally stored in an off-chip DDR memory, and access to the DDR memory will reduce the access efficiency of the hash table. Summary of the invention

[0003] This patent proposes a high-performance hash table implementation method suitable for FPGA. The hash table is saved in the FPGA external DDR, and an index table is set in the FPGA internal RAM. The index table is used to reduce the number of accesses to the DDR memory and improve the efficiency of hash queries. The principle is: a large-scale hash table is established on the DDR, and the hash_ddr (key) hash function is used. The hash result generated by it can be used as an address to access the entire hash table stored in the DDR. At the same time, a small-scale index table is constructed in the FPGA internal RAM, which is used to identify whether a certain address in the hash table has written data. In the case of the existence of an index table, the query for the hash table can first quickly predict whether a certain address in the hash table in the DDR has been written in the on-chip RAM. If it is determined through the index table that a certain address in the hash table has not been written, there is no need to access the DDR memory and it is directly returned. This can reduce the number of accesses to the DDR memory, thereby improving the query efficiency of the hash table.

[0004] In order to achieve the above object, the present invention has the following technical solutions:

[0005] A high-performance hash table implementation method suitable for FPGA includes the following steps:

[0006] Step 1: Construct a large-scale hash table on DDR and use the hash_ddr(key) hash function. The hash result generated by the function can be used as the address to access the entire hash table stored in DDR.

[0007] Step 2: Build a small-scale index table in the FPGA internal RAM. This table is used to identify whether a certain address in the hash table has been written to data. Unlike the hash table, which needs to save all the table entry information, the index table only records whether the address has been written to data, which only requires 1 bit of information. To reduce the size of the index table, the index table uses the hash_ram(key) hash function, which generates a smaller hash result width than the hash_ddr function, so only a small amount of RAM space is needed to save the index bit.

[0008] Step 3: During the insertion phase of the hash table, after the address is calculated by the hash_ddr function, data is written to the corresponding address of the hash table in DDR; at the same time, in RAM, the RAM address is calculated by the hash_ram function, and 1 is written to the corresponding address in RAM.

[0009] Step 4: During the query process of the hash table, if the data stored in the address generated by the hash_ram function is 0 in the RAM, it means that there is no data written in the main hash table, and there is no need to access the DDR memory, and it returns directly.

[0010] Step 5: Since the index table is smaller than the hash table, there is still a situation where the data in the hash table does not exist when the index table is marked as 1, resulting in a false hit. This patent reduces the false hit situation by using a dual index table, that is, creating two index tables, using two different hash functions, hash_ram1 function and hash_ram2 function, to generate addresses. When both index tables are marked as 1, the hash table in DDR is searched.

[0011] Compared with the prior art, the present invention has the following beneficial effects: since the access efficiency of the RAM on the FPGA chip is much higher than the access efficiency of the memory such as the DDR on the chip, the method of the present invention marks whether the hash table has been written with data by establishing an index table in the RAM on the FPGA chip, and does not need to access the DDR memory if the data has not been written. The reduction in the number of DDR memory accesses can effectively improve the query efficiency of the hash table. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Schematic diagram of the hash table after adding the index table;

[0013] Figure 2 Test results of reducing DDR access times by accessing the hash table after adding the index table. DETAILED DESCRIPTION

[0014] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0015] The present invention proposes a high-performance hash table implementation method suitable for FPGA, which can improve the efficiency of large-scale hash table query in FPGA, save hash table information in a large-capacity memory such as DDR outside the FPGA, set an index table in the RAM inside the FPGA, and improve the efficiency of hash query by means of the built-in RAM in the FPGA. The principle is: construct a large-scale hash table on the DDR, the hash table adopts the hash_ddr (key) hash function, and the hash result generated by the hash table can be used as an address to access the entire hash table stored in the DDR memory. At the same time, a small-scale index table is constructed in the RAM inside the FPGA, and the table is used to identify whether the corresponding address of the hash table is written. In the case of the existence of the index table, the query for the hash table can first quickly pre-judge whether the queried data exists in the hash table in the on-chip RAM, and then access the DDR when it is determined that the hash table exists with a great possibility, so that the number of accesses to the DDR memory can be effectively reduced, thereby improving the query efficiency of the hash table.

[0016] A high-performance hash table implementation method suitable for FPGA includes the following steps:

[0017] Step 1: Construct a large-scale hash table on DDR, using the hash_ddr(key) hash function, and use the hash result as the address to access the hash table stored in the DDR memory. Assuming that the external DDR memory is 4GByte, the output width of the hash_ddr function is set to 32 bits, so that the address generated by the hash_ddr function can access the hash table in the entire DDR memory.

[0018] Step 2: Build a small-scale index table in the RAM inside the FPGA, which is used to identify whether the corresponding address in the hash table has been written with data. Unlike the need to save all table item information in the hash table, each address in the index table only needs to save 1 bit of data. In order to reduce the scale of the index table, the index table uses the hash_ram(key) hash function, which generates a hash result with a width smaller than the hash_ddr function, which can reduce the required RAM space. The function of the index table is to predict whether the corresponding position in the hash table has been written with data during the hash search process. Assuming that the storage capacity of the RAM inside the FPGA is 256Mbit, the output width of the hash_ram function is set to 28 bits, so that the output address of the hash_ram function can access all units of the index table in the RAM.

[0019] Step 3: During the insertion phase of the hash table, after the address is calculated by the hash_ddr function, the table entry data is saved at the corresponding address in the DDR; at the same time, in the RAM, the RAM address is obtained by the hash_ram function, and the state 1 is saved at the corresponding address in the RAM.

[0020] Step 4: During the hash table query process, if the data in the address generated by the hash_ram function is 0 in the RAM, it means that the data does not exist in the hash table. In this case, there is no need to access the hash table in the DDR memory and it is returned directly, which can effectively reduce the number of DDR accesses.

[0021] Step 5: Since the index table is smaller than the hash table, when the index table is marked as 1, there is still a situation where the corresponding position in the hash table has no data written, resulting in a false hit. The present invention reduces the false hit situation by means of a dual index table, that is, two index tables are created, and two different hash functions, hash_ram1 function and hash_ram2 function, are used for marking. When both index hash tables are marked as 1, the hash table in DDR is searched again, which can further reduce the false hit.

[0022] Compared with the prior art, the present invention has the following beneficial effects: Since the access efficiency of FPGA on-chip RAM is much higher than that of off-chip DDR and other memories, the method of the present invention establishes an index table in the FPGA on-chip RAM to identify whether the corresponding unit of the hash table has been written with data. If no data has been written, it is not necessary to access the DDR memory and directly return. The reduction in the number of DDR memory accesses can effectively improve the efficiency of Hash queries.

[0023] See also Figure 1 and Figure 2 By adding a RAM index table, the present invention can reduce the number of times that FPGA accesses the DDR external memory, and effectively improve the query efficiency of the hash table. Figure 2 The test results are on the existing FPGA development platform: the size of each hash table entry is set to 1Kbyte, random data is added, and 16M hash table queries are performed. Tests are performed under different DDR capacities and different numbers of entries. The test results show that by adding a RAM index table, the number of DDR memory accesses can be reduced by up to 94%.

[0024] The above description is only a preferred embodiment of the present invention and is not intended to impose any limitation on the technical solution of the present invention. Those skilled in the art should understand that, without departing from the spirit and principles of the present invention, the technical solution can also be subjected to several simple modifications and substitutions, and these modifications and substitutions are also within the scope of protection covered by the claims.

Claims

1. A high-performance hash table implementation method suitable for FPGA, characterized in that The hash table is stored in the DDR memory outside the FPGA chip, and the index table of the hash table is stored in the RAM inside the FPGA chip. During the query process of the hash table, the index table in the RAM inside the chip is queried first. If the identification field in the index table indicates that the queried hash table item record has been written with data, the hash table in the DDR memory is queried again. If the identification field in the index table indicates that the queried hash table entry record has no data written, it is directly returned without querying the DDR memory, thereby improving the hash table query efficiency by reducing the number of accesses to the off-chip DDR; the following steps are included: Step 1: Hash table establishment phase: save the hash table in the DDR memory outside the FPGA chip, and the hash table stores the actual data; Step 2: During the hash table establishment phase, the index table of the off-chip hash table is saved in the RAM memory inside the FPGA chip. Each row in the table saves 1 bit of information to mark whether the address has data saved in the hash table. If not, it is 0, and if it is saved, it is 1. Step 3: In the hash query stage, first query the index table. If the corresponding mark in the index table is 0, it means that the information does not exist in the hash table, and the DDR memory is not accessed, and it is returned directly; if the corresponding mark in the index table is 1, query is performed in the DDR memory.

2. The high-performance hash table implementation method suitable for FPGA according to claim 1, characterized in that: In the process of establishing the hash table, in addition to establishing the hash table in an external large-capacity memory such as DDR, an index table of the hash table is also established in the on-chip RAM. The index table is used to identify whether the table items in the hash table have been written.

3. The high-performance hash table implementation method suitable for FPGA according to claim 2, characterized in that: In the hash table query phase, if the corresponding entry flag bit in the index table is 0, it indicates that the queried content in the hash table has not been written and is returned directly without accessing the DDR memory.

Images