A high-speed accurate matching search method
By combining the HASH algorithm with RAM/collision pool storage, the problems of slow table lookup speed and high cost under high speed are solved, realizing high-speed and accurate matching lookup, reducing hardware costs and improving query speed.
Patent Information
- Application Number
- CN202210297992.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-03-24
AI Technical Summary
In the existing technology, address-based table entry lookup methods cannot meet performance requirements under high-speed conditions, and TCAM devices are expensive, resulting in slow lookup speeds and high costs.
The HASH algorithm is used to input data in parallel into N modules, generating N mapping addresses. An M-level lookup method combining RAM and a collision pool is used for priority storage and retrieval. Valid data is stored in RAM, and data that does not meet the conditions is stored in the collision pool. Storage space is optimized through an aging update mechanism.
It achieves high-speed and accurate matching and searching, reduces hardware costs, improves query speed, and supports configurable aging time for table entries. It also has the advantages of strong scalability and predictable resources.
Smart Images

Figure CN114615196B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of communication technology, and in particular relates to a high-speed accurate matching search method. Background Art
[0002] In network communication systems, table search processing is often involved. Traditional table search usually uses an address-based search method, that is, the table entries are searched one by one in the order of the address. For network communication, the table search is mostly quintuples or multituples. Under high-speed requirements, general linear search methods cannot meet performance requirements. In recent years, content addressable memory (Content Addressable MeMory) has gradually emerged to achieve fast table search. Its storage and query methods are diverse, and TCAM devices are currently used to achieve this. Although TCAM devices have a fast search speed, they are expensive and will greatly increase the hardware cost of the project. Summary of the invention
[0003] 1. Technical issues to be resolved
[0004] The technical problem to be solved by the present invention is how to provide a high-speed accurate matching search method to solve the problem of slow query speed and high cost.
[0005] (II) Technical solution
[0006] In order to solve the above technical problems, the present invention provides a high-speed accurate matching search method, which includes a table building process: the data to be stored lab_data is input into N HASH algorithm modules in parallel, the N HASH values output are used as N mapping addresses of lab_data, the N mapping addresses are used as the query addresses of RAM, and the RAM is searched at M levels. The corresponding data of the table items obtained by each mapping address are marked as data0, data1, ..., dataN-1 respectively; the table item storage format includes vld, vld represents the valid bit of the table item,
[0007] If the vld bit in data0 is invalid, or vld is valid and the lab_data in data0 is equal to the lab_data to be stored, the table entry storage condition is met, and the current table entry is stored in RAM, and the write address is addr0;
[0008] If the vld bit in data1, data2, ..., dataN-1 is invalid, or vld is valid and the lab_data in data0 is equal to the lab_data to be stored, the table entry storage condition is met, and the current table entry is stored in RAM, and the write address is addr1, addr2, ..., addrN-1.
[0009] Furthermore, when the entry is stored in the RAM, the current storage time, the current entry state and the validity flag are recorded.
[0010] Furthermore, the address range of the table entry is 0-65535.
[0011] Furthermore, the data lab_data to be stored is a quintuple or a decrypted triplet.
[0012] Furthermore, the HASH value is 16-bit data.
[0013] Furthermore, the method also includes: if there are two or more addrs that meet the storage conditions at the same time, the priority is addr0>addr1>…>addrN-1, and each lab_data only needs to be stored once.
[0014] Furthermore, the method also includes: if N lab_data values do not meet the table entry storage conditions, query the conflict pool, repeat the above comparison query, meet the table entry storage conditions, store the current table entry in the conflict pool, and do not meet the table entry storage conditions and enter aging update.
[0015] Furthermore, if at least one of the N HASH values of lab_data matches the RAM / conflict pool address, and the current RAM / conflict pool address is empty and unoccupied, the address query priority decreases from hash0 to hashN-1;
[0016] Reserve storage space as a conflict pool. After the M-level table lookup, the lab_data that does not meet the conditions for storage in RAM is stored in the conflict pool if the conflict pool is not full. The conflict pool address is addressed according to the needs.
[0017] When the conflict pool is full, read the timestamps of all entries, compare them with the current time, delete the original entry at the address with the longest time, and update it to the entry currently to be written.
[0018] Furthermore, when N=4, M=3 and the table entry is an spd table entry, performing an M-level table lookup on the RAM specifically includes:
[0019] First level table lookup:
[0020] Perform four hash algorithms on a certain lab_data and obtain four hash values as the RAM first-level table lookup addresses addr0, addr1, addr2, and addr3;
[0021] addr0 is mapped to one of the four hash values a0, a1, a2, and a3 of spd10, and the RAM at the current addr0 address is checked to see if it is not empty;
[0022] addr1 is mapped to one of the four hash values b0, b1, b2, and b3 of spd11, and the RAM address of addr1 is checked to see if it is not empty.
[0023] addr2 is mapped to one of the four hash values c0, c1, c2, c3 of spd12, and the current addr2 address RAM is checked to see if it is not empty;
[0024] addr3 is mapped to one of the four hash values d0, d1, d2, and d3 of spd13, and the RAM at the current addr3 address is checked to see if it is not empty;
[0025] Secondary table lookup:
[0026] The RAM corresponding to the address of the first-level table lookup is not empty and the lab_data in data is not equal to the lab_data to be stored, and the second-level table lookup is entered;
[0027] The secondary table lookup is based on the priority. The hash values a0, a1, a2, and a3 corresponding to spd10 are first extracted as the secondary table lookup addresses, and the primary table lookup process is repeated.
[0028] All non-empty addresses corresponding to the hash values of spd10 are occupied, and the hash values b0, b1, b2, and b3 corresponding to spd11 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated;
[0029] All non-empty address RAMs corresponding to the hash values of spd11 are occupied, and the hash values c0, c1, c2, and c3 corresponding to spd12 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated;
[0030] All non-empty addresses corresponding to the hash values of spd12 are occupied, and the hash values d0, d1, d2, and d3 corresponding to spd13 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated;
[0031] Three-level lookup table
[0032] The corresponding address RAM of the second-level table lookup is not empty and the lab_data in data is not equal to the lab_data to be stored, and the third-level table lookup is entered;
[0033] The third-level table lookup is based on the priority. The hash values e0, e1, e2, and e3 corresponding to spd20 are first extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated.
[0034] All non-empty addresses corresponding to the hash values of spd20 are occupied, and the hash values f0, f1, f2, and f3 corresponding to spd21 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated;
[0035] All non-empty address RAMs corresponding to the hash values of spd21 are occupied, and the hash values g0, g1, g2, and g3 corresponding to spd22 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated;
[0036] All non-empty address RAMs corresponding to the hash values of spd22 are occupied, and the hash values h0, h1, h2, and h3 corresponding to spd23 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated;
[0037] Similarly:
[0038] b0, b1, b2, b3 extract the corresponding hash values of spd24, spd25, spd26, spd27;
[0039] c0, c1, c2, c3 extract the corresponding hash values of spd28, spd29, spd2a, spd2b;
[0040] d0, d1, d2, d3 extract the corresponding hash values of spd2c, spd2d, spd2e, spd2f;
[0041] Conflict Pool:
[0042] The corresponding address RAMs of the three-level table lookup are not empty and are written into the conflict pool.
[0043] Furthermore, the method also includes a query process: N mapping addresses are used as query addresses of the RAM, and M-level table lookups are performed on the RAM. If the matching conditions are met, the lab_data at the current RAM address is taken out to obtain the query results. If the matching conditions are not met, the conflict pool is queried. If the matching conditions are met, the lab_data at the current conflict pool address is taken out to obtain the query results. If the matching conditions are not met, the current query data lab_data is processed on demand according to the requirements of different scenarios.
[0044] (III) Beneficial effects
[0045] The present invention discloses a high-speed accurate matching search method. The method is based on the HASH algorithm and cooperates with efficient arrangement, storage and query methods to achieve high-speed accurate matching search, support configurable table item aging time, and use external DDR or SRAM for storage, saving FPGA internal resources. The method is designed in a modular way and has the advantages of strong scalability, predictable resources, fast table lookup speed, etc., and greatly reduces the hardware cost of the project. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is the table entry storage format of the present invention.
[0047] Figure 2 This is the RAM table and conflict pool storage format of the present invention.
[0048] Figure 3 This is the process of creating a table.
[0049] Figure 4 It is a three-level table lookup process.
[0050] Figure 5 For the query process. DETAILED DESCRIPTION
[0051] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.
[0052] The present invention takes RAM as a memory as an example, the content to be stored or queried (marked as lab_data) is a tuple, lab_data is sent as input to N HASH algorithms (the algorithm is provided by the user), and the N HASH values output by the N HASH algorithms are used as RAM addresses. When the table entry is stored in RAM, the current storage time, current table entry state and valid identifier are recorded. The table entry storage content is as follows: Figure 1 As shown, the explanation of terms is shown in Table 1, and the table item is marked as lab_tab.
[0053] Table 1 Glossary of terms
[0054]
[0055]
[0056] ⑴Create table:
[0057] The address range of the table entry is 0-65535 (2^16-1). Before querying the table entry, you need to create the table first. Figure 2 The storage format for the RAM table and conflict pool, Figure 3 This is the table building process. First, the data lab_data to be stored is obtained, such as a tuple (quintuple\decrypted triplet), and lab_data is input to N HASH algorithm modules in parallel. The output N HASH values (each HASH value is 16-bit data) are used as N mapping addresses of lab_data (each mapping address is 16-bit data). The N mapping addresses are marked as addr0, addr1, ..., addrN-1, and the N mapping addresses are used as the query addresses of the RAM. The RAM is looked up at M levels, and the corresponding data obtained at each mapping address is marked as data0, data1, ..., dataN-1.
[0058] If the vld bit in data0 is invalid, or vld is valid and the lab_data in data0 is equal to the lab_data to be stored, the table entry storage condition is met, and the current table entry is stored in RAM, and the write address is addr0;
[0059] Similarly, if the vld bit in data1, data2, ..., dataN-1 is invalid, or if vld is valid and the lab_data in data0 is equal to the lab_data to be stored, the table entry storage condition is met, and the current table entry is stored in RAM, and the write address is addr1, addr2, ..., addrN-1;
[0060] If two or more addrs meet the storage conditions at the same time, the priority is addr0>addr1>…>addrN-1, and each lab_data only needs to be stored once.
[0061] If none of the N lab_data values meet the entry storage conditions, query the conflict pool and repeat the above comparison query. If the entry storage conditions are met, store the current entry in the conflict pool. If the entry storage conditions are not met, enter the aging update process.
[0062] The relevant process and explanation are as follows:
[0063] ① Explanation of the table entry conditions: At least one of the N HASH values of lab_data matches the RAM / conflict pool address, and the current RAM / conflict pool address is empty and not occupied. The address query priority decreases from hash0 to hashN-1.
[0064] ②Explanation of conflict pool: Reserve storage space as the conflict pool. After the M-level table lookup, lab_data that does not meet the conditions for storage in RAM will be stored in the conflict pool if the conflict pool is not full. The conflict pool address is addressed according to demand.
[0065] ③ Explanation of aging update: When the conflict pool is full, read the timestamps of all table entries, compare them with the current time, delete the original table entry at the address with the longest time, and update it to the current table entry to be written.
[0066] ④M-level table lookup process: The present invention takes lab_data as SPD (Security Policy Database, SPD), 4 groups of HASH algorithms and 3-level table lookup as an example. The 3-level table lookup process is as follows: Figure 3 As shown, the formula for table lookup times is: N^1+N^2+N^3+…+N^M-1, where M is the number of search levels (1, 2, 3…) and N is the number of HASH algorithms.
[0067] Figure 4 It is a three-level table lookup process.
[0068] 1) Level 1 table lookup (4 address lookups):
[0069] Perform four hash algorithms on a lab_data in the spd table entry to obtain four hash values as the RAM first-level table lookup addresses addr0, addr1, addr2, and addr3;
[0070] a) addr0 is mapped to one of the four hash values a0, a1, a2, a3 of spd10, and the current addr0 address RAM is checked to be non-empty (that is, the vld flag bit of data is 1);
[0071] b) addr1 is mapped to one of the four hash values b0, b1, b2, and b3 of spd11, and the current addr1 address RAM is checked to be non-empty (that is, the vld flag bit of data is 1);
[0072] c) addr2 is mapped to one of the four hash values c0, c1, c2, c3 of spd12, and the current addr2 address RAM is checked to be non-empty (that is, the vld flag bit of data is 1);
[0073] d) addr3 is mapped to one of the four hash values d0, d1, d2, and d3 of spd13, and the current addr3 address RAM is checked to be non-empty (that is, the vld flag bit of data is 1);
[0074] 2) Second level table lookup (16 address lookups):
[0075] The RAM corresponding to the first-level table lookup is not empty and the lab_data in data is not equal to the lab_data (spd) to be stored, and the second-level table lookup is entered.
[0076] a) The second-level table lookup is based on the priority. The hash values a0, a1, a2, and a3 corresponding to spd10 are first extracted as the second-level table lookup addresses, and the first-level table lookup process is repeated;
[0077] b) All non-empty address RAMs corresponding to the hash values of Spd10 are occupied, and the hash values b0, b1, b2, and b3 corresponding to spd11 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated;
[0078] c) All non-empty address RAMs corresponding to the hash values of Spd11 are occupied, and the hash values c0, c1, c2, and c3 corresponding to spd12 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated;
[0079] d) All non-empty addresses corresponding to the hash values of Spd12 are occupied, and the hash values d0, d1, d2, and d3 corresponding to spd13 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated;
[0080] 3) Three-level table lookup (64 address lookups)
[0081] The corresponding address RAM of the second-level table lookup is not empty and the lab_data in data is not equal to the lab_data (spd) to be stored, and the third-level table lookup is entered.
[0082] a) The third-level table lookup is based on the priority. The hash values e0, e1, e2, and e3 corresponding to spd20 are first extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated;
[0083] b) All non-empty addresses corresponding to the hash values of Spd20 are occupied, and the hash values f0, f1, f2, and f3 corresponding to spd21 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated;
[0084] c) All non-empty address RAMs corresponding to the hash values of Spd21 are occupied, and the hash values g0, g1, g2, and g3 corresponding to spd22 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated;
[0085] d) All non-empty addresses corresponding to the hash values of Spd22 are occupied, and the hash values h0, h1, h2, and h3 corresponding to spd23 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated;
[0086] e) Similarly
[0087] b0, b1, b2, b3, extract the corresponding hash values of spd24, spd25, spd26, spd27;
[0088] c0, c1, c2, c3, extract the corresponding hash values of spd28, spd29, spd2a, spd2b;
[0089] d0, d1, d2, d3, extract the corresponding hash values of spd2c, spd2d, spd2e, spd2f.
[0090] 4) Conflict Pool
[0091] The corresponding address RAMs of the three-level table lookup are not empty and are written into the conflict pool.
[0092] 5) Aging update
[0093] When the conflict pool is full, read the timestamps of all entries, compare them with the current time, and update the address with the longest time to the SPD entry to be written in the current table lookup.
[0094] 6) Deposit process
[0095] During the table lookup process, it is found that the M-th level table lookup address addrMx meets the storage conditions. Since address addrMx comes from the data in the M-1-th level table lookup address addr(M-1)x; address addr(M-1)x comes from the data in the M-2-th level table lookup address addr(M-2)x; and so on, address addr2x comes from the data in the 1st level table lookup address addr1x. The steps for writing the spd table entry are: the data in address addr(M-1)x is moved to address addrMx; the data in address addr(M-2)x is moved to address addr(M-1)x; and so on, the data in addr2x is moved to addr3x; the data in addr1x is written to addr2x; and the spd table entry to be stored is written to address addr1x.
[0096] like Figure 4 As shown, assuming that when looking up the table at the third level, the e2 address meets the storage conditions, the steps for writing the spd table entry are as follows:
[0097] 1. The data corresponding to address a0 is moved to address e2;
[0098] 2. The data corresponding to the addr0 address is moved to the a0 address;
[0099] 3. The SPD table entry to be stored is written to address addr0;
[0100] ⑵Query:
[0101] Query process as follows Figure 5 As shown, according to the configuration sent by the upper layer or other parsing methods, the data lab_data to be queried is obtained, such as a tuple (quintuple\decrypted triplet), and lab_data is input into N HASH compression algorithm modules in parallel. The output N HASH values are used as N mapping addresses of lab_data. The N mapping addresses are marked as addr0, addr1, ..., addrN-1 respectively. The N mapping addresses are used as the query addresses of the RAM. An M-level table lookup is performed on the RAM. If the matching condition is met, the lab_data at the current RAM address is taken out to obtain the query result. If the matching condition is not met, the conflict pool is queried. If the matching condition is met, the lab_data at the current conflict pool address is taken out to obtain the query result. If the matching condition is not met, the current query data lab_data is processed on demand according to the requirements of different scenarios, such as directly discarding it or returning no query result.
[0102] The relevant process and explanation are as follows:
[0103] ① Explanation of matching conditions: At least one of the N HASH values of lab_data matches the RAM / conflict pool address. The address query priority decreases from hash0 to hashN, and the lab_data at the corresponding address is the same as the current query data.
[0104] ②Other explanations are the same as those for table creation.
[0105] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A high-speed accurate matching search method, characterized in that: The method includes a table building process: data lab_data to be stored is input to N HASH algorithm modules in parallel, the output N HASH values are used as N mapping addresses of lab_data, the N mapping addresses are used as RAM query addresses, and M-level table lookup is performed on the RAM. The corresponding data of the table item obtained by each mapping address is marked as data0, data1, ..., dataN-1 respectively; the table item storage format includes vld, vld represents the valid bit of the table item, If the vld bit in data0 is invalid, or vld is valid and the lab_data in data0 is equal to the lab_data to be stored, the table entry storage condition is met, and the current table entry is stored in RAM, and the write address is addr0; If the vld bit in data1, data2, ..., dataN-1 is invalid, or vld is valid and the lab_data in data0 is equal to the lab_data to be stored, the table entry storage condition is met, and the current table entry is stored in RAM, and the write address is addr1, addr2, ..., addrN-1; in, When N=4, M=3, and the table entry is an SPD table entry, the M-level table lookup on the RAM specifically includes: First level table lookup: Perform four hash algorithms on a certain lab_data and obtain four hash values as the RAM first-level table lookup addresses addr0, addr1, addr2, and addr3; addr0 is mapped to one of the four hash values a0, a1, a2, and a3 of spd10, and the RAM at the current addr0 address is checked to see if it is not empty; addr1 is mapped to one of the four hash values b0, b1, b2, and b3 of spd11, and the RAM address of addr1 is checked to see if it is not empty. addr2 is mapped to one of the four hash values c0, c1, c2, c3 of spd12, and the current addr2 address RAM is checked to see if it is not empty; addr3 is mapped to one of the four hash values d0, d1, d2, and d3 of spd13, and the RAM at the current addr3 address is checked to see if it is not empty; Secondary table lookup: The RAM corresponding to the address of the first-level table lookup is not empty and the lab_data in data is not equal to the lab_data to be stored, and the second-level table lookup is entered; The secondary table lookup is based on the priority. The hash values a0, a1, a2, and a3 corresponding to spd10 are first extracted as the secondary table lookup addresses, and the primary table lookup process is repeated. All non-empty addresses corresponding to the hash values of spd10 are occupied, and the hash values b0, b1, b2, and b3 corresponding to spd11 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated; All non-empty address RAMs corresponding to the hash values of spd11 are occupied, and the hash values c0, c1, c2, and c3 corresponding to spd12 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated; All non-empty addresses corresponding to the hash values of spd12 are occupied, and the hash values d0, d1, d2, and d3 corresponding to spd13 are extracted as the secondary table lookup addresses, and the primary table lookup process is repeated; Three-level lookup table The corresponding address RAM of the second-level table lookup is not empty and the lab_data in data is not equal to the lab_data to be stored, and the third-level table lookup is entered; The third-level table lookup is based on the priority. The hash values e0, e1, e2, and e3 corresponding to spd20 are first extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated. All non-empty addresses corresponding to the hash values of spd20 are occupied, and the hash values f0, f1, f2, and f3 corresponding to spd21 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated; All non-empty address RAMs corresponding to the hash values of spd21 are occupied, and the hash values g0, g1, g2, and g3 corresponding to spd22 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated; All non-empty address RAMs corresponding to the hash values of spd22 are occupied, and the hash values h0, h1, h2, and h3 corresponding to spd23 are extracted as the third-level table lookup addresses, and the first-level table lookup process is repeated; Similarly: b0, b1, b2, b3 extract the corresponding hash values of spd24, spd25, spd26, spd27; c0, c1, c2, c3 extract the corresponding hash values of spd28, spd29, spd2a, spd2b; d0, d1, d2, d3 extract the corresponding hash values of spd2c, spd2d, spd2e, spd2f; Conflict Pool: The corresponding address RAMs of the three-level table lookup are not empty and are written into the conflict pool.
2. The high-speed accurate matching search method according to claim 1, characterized in that: When the entry is stored in RAM, the current storage time, current entry status and validity flag are recorded.
3. The high-speed accurate matching search method according to claim 1, characterized in that: The address range of the table entry is 0-65535.
4. The high-speed accurate matching search method according to claim 1, characterized in that: The data lab_data to be stored is a quintuple or a decrypted triplet.
5. The high-speed accurate matching search method according to claim 1, characterized in that: The HASH value is 16 bits.
6. The high-speed accurate matching search method according to any one of claims 1 to 5, characterized in that: The method also includes: if there are two or more addrs that meet the storage conditions at the same time, the priority is addr0>addr1>...>addrN-1, and each lab_data only needs to be stored once.
7. The high-speed accurate matching search method according to claim 6, characterized in that: At least one of the N HASH values of lab_data matches the RAM / conflict pool address, and the current RAM / conflict pool address is empty and not occupied. The address query priority decreases from hash0 to hashN-1; Reserve storage space as a conflict pool. After the M-level table lookup, the lab_data that does not meet the conditions for storage in RAM is stored in the conflict pool if the conflict pool is not full. The conflict pool address is addressed according to the needs. When the conflict pool is full, read the timestamps of all entries, compare them with the current time, delete the original entry at the address with the longest time, and update it to the entry currently to be written.
8. The high-speed accurate matching search method according to claim 1, characterized in that: The method also includes a query process: N mapping addresses are used as query addresses of the RAM, and an M-level table query is performed on the RAM. If the matching condition is met, the lab_data at the current RAM address is taken out to obtain the query result. If the matching condition is not met, the conflict pool is queried. If the matching condition is met, the lab_data at the current conflict pool address is taken out to obtain the query result. If the matching condition is not met, the current query data lab_data is processed on demand according to the requirements of different scenarios.
