A data storage method, device and equipment

Abstract

The application provides a data storage method, device and equipment. The method comprises the following steps: determining the target bit number P for the newly added to-be-stored data in the bit-cutting hash data structure; the bit-cutting hash data structure comprises a plurality of hash buckets, and the target bit number P is used to make the to-be-stored data and the stored data in the bit-cutting hash data structure not have hash conflicts; determining the BSS value of each bit based on the stored data and the to-be-stored data, and determining P characteristic bits based on the BSS value of each bit; determining the target hash bucket corresponding to the to-be-stored data in the bit-cutting hash data structure based on the value corresponding to the P characteristic bits in the to-be-stored data; and storing the to-be-stored data in the target hash bucket. Through the application scheme, the accurate matching performance of data can be improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data storage method, apparatus and device. Background Technology

[0002] The Forwarding Information Base (FIB) is a crucial data structure in network devices (such as routers or switches) used for efficient forwarding of data packets. The FIB stores the mapping between IP addresses and outgoing interfaces. When a network device receives a data packet, it queries the FIB using the destination address of the data packet to obtain the corresponding outgoing interface, and then sends the data packet through that interface.

[0003] When using IPv6 addresses, the larger address space and longer network segments of IPv6 addresses require a significant amount of memory to store in the File Index (FIB). Furthermore, the performance of updating IPv6 addresses is poor when adding or deleting them from the FIB. Additionally, querying IPv6 addresses from the FIB is also slow and time-consuming. Summary of the Invention

[0004] This application provides a data storage method, the method comprising:

[0005] For newly added data to be stored in the bit-cut hash data structure, a target number of bits P is determined; the bit-cut hash data structure includes multiple hash buckets, and the target number of bits P is used to ensure that there is no hash collision between the data to be stored and the data already stored in the bit-cut hash data structure;

[0006] Based on the stored data and the data to be stored, determine the Bit Separable Set (BSS) value for each bit, and determine P feature bits based on the BSS value for each bit.

[0007] Based on the values ​​of the P feature bits in the data to be stored, determine the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure;

[0008] The data to be stored is stored in the target hash bucket.

[0009] This application provides a data storage device, the device comprising:

[0010] A determination module is used to determine the target number of bits P for newly added data to be stored in a bit-splitting hash data structure; the bit-splitting hash data structure includes multiple hash buckets, and the target number of bits P is used to ensure that there is no hash collision between the data to be stored and the data already stored in the bit-splitting hash data structure; based on the stored data and the data to be stored, the bit separability set (BSS) value of each bit is determined, and P feature bits are determined based on the BSS value of each bit;

[0011] The processing module is used to determine the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure based on the values ​​corresponding to the P feature bits in the data to be stored;

[0012] A storage module is used to store the data to be stored in the target hash bucket.

[0013] This application provides a network device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the data storage method of the example above in this application.

[0014] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements the data storage method described in the above example of this application.

[0015] This application provides a machine-readable storage medium storing machine-executable instructions that can be executed by a processor; wherein the processor is configured to execute the machine-executable instructions to implement the data storage method of the above example of this application when the machine-executable instructions are executed.

[0016] As can be seen from the above technical solutions, in the embodiments of this application, a bit-cut hash data structure can be constructed to store data (such as IPv6 addresses), and then the data can be queried through the bit-cut hash data structure. Thus, this application proposes a perfect hash (i.e., bit-cut hash) that balances query performance, update performance, and memory usage. By implementing the hash function through the bit-cut hash data structure, it solves the problem of accurate data matching in scenarios with high requirements for query, update, and memory usage. It achieves query performance at the nanosecond level, update performance at the microsecond level, and controllable memory usage, effectively improving the accuracy of data matching and providing nanosecond-level query performance, microsecond-level average update performance, and controllable memory usage. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a data storage method in one embodiment of this application;

[0018] Figures 2A-2C This is a schematic diagram of hash switching in one embodiment of this application;

[0019] Figures 3A-3F This is a schematic diagram of hash switching in one embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the structure of a data storage device according to one embodiment of this application;

[0021] Figure 5 This is a hardware structure diagram of a network device according to one embodiment of this application. Detailed Implementation

[0022] This application proposes a data storage method that can be applied to network devices (such as routers or switches).

[0023] See Figure 1 The diagram shown is a flowchart of the method, which may include:

[0024] Step 101: For the newly added data to be stored in the bit-cut hash data structure, determine the target number of bits P; the bit-cut hash data structure includes multiple hash buckets, and the target number of bits P is used to ensure that there is no hash collision between the data to be stored and the data already stored in the bit-cut hash data structure.

[0025] Step 102: Based on the stored data and the data to be stored, determine the BSS value of each bit, and determine P feature bits based on the BSS value of each bit.

[0026] Step 103: Based on the values ​​of the P feature bits in the data to be stored, determine the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure.

[0027] Step 104: Store the data to be stored in the target hash bucket.

[0028] The process of determining the target number of bits P may include: determining the BSS value of each bit based on the stored data and the data to be stored, and sorting all bits in descending order of BSS value; selecting feature bits based on the sorting results; determining hash collisions based on the selected feature bits; and determining the target number of bits P based on the selected feature bits if no hash collisions are found.

[0029] In one example, a bit-splitting hash data structure (i.e., a hash data structure) can be maintained, which may include multiple hash buckets. Based on this, when new data needs to be added to the bit-splitting hash data structure, the target number of bits P can be determined. The BSS (Bit Separability Set) value of each bit is determined based on the already stored data and the data to be stored, and P characteristic bits are determined based on the BSS value of each bit. Based on the values ​​of the P characteristic bits in the data to be stored, the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure can be determined, and then the data to be stored is stored in the target hash bucket.

[0030] For stored data within a bit-splitting hash data structure, if the currently determined P feature bits are the same as the previously determined P feature bits (i.e., the target number of bits P has not changed, and the P feature bits are completely identical), then the hash bucket for the stored data remains unchanged, and there is no need to restore the stored data. If the currently determined P feature bits are different from the previously determined P feature bits (e.g., the target number of bits P has changed, or the P feature bits are not completely identical), then it is necessary to determine the corresponding hash bucket for the stored data in the bit-splitting hash data structure based on the values ​​of the P feature bits in the stored data. Then, the stored data is stored in that hash bucket; in other words, the stored data needs to be restored, and the hash bucket for the stored data needs to be updated.

[0031] In one example, when deleting stored data from a bit-splitting hash data structure, the stored data can be deleted directly without adjusting the P characteristic bits. Alternatively,

[0032] After deleting stored data, the BSS value of each bit can be determined based on the remaining stored data in the bit-splitting hash data structure, and P feature bits can be determined based on the BSS value of each bit. If the P feature bits do not change, the hash bucket of the remaining stored data remains unchanged, and it is not necessary to restore the remaining stored data. If the P feature bits change, the hash bucket corresponding to the remaining stored data in the bit-splitting hash data structure is determined based on the values ​​of the P feature bits in the remaining stored data, and the remaining stored data is stored in this hash bucket. In other words, the remaining stored data needs to be restored, and the hash bucket of the remaining stored data needs to be updated. Alternatively,

[0033] After deleting stored data, the target number of bits P' is redefined (using the same method as the target number of bits). The target number of bits P' may be the same as or different from the target number of bits P. Based on the remaining stored data in the bit-splitting hash data structure, the BSS value of each bit is determined, and P' feature bits are determined based on the BSS value of each bit. If the currently determined P' feature bits are the same as the previously determined P feature bits, the hash bucket for the remaining stored data remains unchanged. If the currently determined P' feature bits are different from the previously determined P feature bits, the hash bucket corresponding to the remaining stored data in the bit-splitting hash data structure is determined based on the values ​​of the P feature bits in the remaining stored data, and the remaining stored data is stored in this hash bucket.

[0034] As can be seen from the above technical solutions, in the embodiments of this application, a bit-cut hash data structure can be constructed to store data (such as IPv6 addresses), and then the data can be queried through the bit-cut hash data structure. Thus, this application proposes a perfect hash (i.e., bit-cut hash) that balances query performance, update performance, and memory usage. By implementing the hash function through the bit-cut hash data structure, it solves the problem of accurate data matching in scenarios with high requirements for query, update, and memory usage. It achieves query performance at the nanosecond level, update performance at the microsecond level, and controllable memory usage, effectively improving the accuracy of data matching and providing nanosecond-level query performance, microsecond-level average update performance, and controllable memory usage.

[0035] The technical solutions described above in the embodiments of this application will be explained below in conjunction with specific application scenarios.

[0036] In data processing scenarios, hash data structures (such as hash tables) can be used to achieve accurate data matching. Hash data structures offer fast lookup speeds while balancing update performance and memory usage. For example, the File Interface Board (FIB) is a critical data structure in network devices, used to store the mapping between IP addresses and outgoing interfaces. A hash data structure can be used to store the IP addresses and outgoing interfaces in the FIB. However, when the IP addresses are IPv6 addresses, using a hash data structure to store IPv6 addresses requires a significant amount of memory. The update performance of IPv6 addresses is poor each time they are added or deleted from the hash data structure. Furthermore, the query performance for IPv6 addresses from the hash data structure is also poor, requiring a longer query time.

[0037] For example, hash data structures can use either traditional hashing or perfect hashing. Traditional hashing, such as JHash, CRCHash, and BiHash, suffers from more hash collisions and poor query performance in certain scenarios or with large amounts of data. Perfect hashing, on the other hand, is collision-free. When adding or deleting data, the hash function needs to be dynamically adjusted, and the hash table needs to be reconstructed entirely, resulting in poor incremental update performance, as seen in BBHash.

[0038] In response to the above findings, this application proposes a perfect hash algorithm that balances query performance, update performance, and memory usage. This perfect hash algorithm is called bitcuts hash. Bitcuts hash can solve the problem of accurate data matching in scenarios with high requirements for query, update, and memory usage, achieving query performance in the nanosecond range, update performance in the microsecond range, and controllable memory usage.

[0039] In one example, a bit-cut hash data structure can be maintained, which includes multiple hash buckets through which data is stored. For ease of distinction, the data in the bit-cut hash data structure is referred to as stored data, and the data not yet stored in the bit-cut hash data structure is referred to as data to be stored. Both stored and unstored data can be IPv6 addresses or other types of data, depending on the data processing scenario. This embodiment does not impose any restrictions and takes IPv6 addresses as an example.

[0040] For ease of description, in subsequent embodiments, both stored data and data to be stored can be collectively referred to as data to be processed; that is, data to be processed may include both stored data and data to be stored. In subsequent embodiments, data to be processed may also be referred to as rules; that is, the rules in subsequent embodiments are data to be processed.

[0041] In one example, this application embodiment proposes a data storage method that can be applied to network devices (such as routers or switches). This data storage method may involve the following processes:

[0042] First, the optimal number of bits for the bitcuts hash algorithm is determined.

[0043] In this embodiment, the bit-cut hash algorithm can be a perfect hash algorithm, and the optimal number of bits can be called the target number of bits P. That is, the target number of bits P is determined for the newly added data to be stored in the bit-cut hash data structure (e.g., if new data needs to be added to the bit-cut hash data structure). The bit-cut hash data structure includes multiple hash buckets, and the target number of bits P is used to ensure that there are no hash collisions between the data to be stored and the data already stored in the bit-cut hash data structure.

[0044] 1. Design concept of bit splitting hash algorithm.

[0045] In one implementation, the data structure with the fewest memory accesses is an array. During the query process, the exact match data is used as the array index for access, and the result can be obtained with just one memory access. However, in the IPv6 FIB query scenario, the address length is 16 bytes (128 bits). To achieve one memory access, an array of length 2^128 bytes is required for storage, which is not feasible due to memory constraints.

[0046] In another implementation, the data structure that enables one memory access is the hash data structure. In the IPv6 FIB query scenario, the hash function needs to be simple and time-efficient, capable of placing all the data to be processed into different hash buckets through calculation, that is, each hash bucket has at most one piece of data to be processed.

[0047] In response to the above findings, this application provides a bit-cut hash algorithm to meet the needs of IPv6 FIB query scenarios. The bit-cut hash algorithm can determine whether the current data matches the corresponding rule with only one memory access and also provides excellent support for CPU pipelines.

[0048] In the bit-splitting hash algorithm, the BSS (Bit Separable Set) value of each bit is calculated. The BSS value is the product of the number of all 0 bits and the number of all 1 bits in the data to be processed. This process identifies the characteristic bits of all the data to be processed, and the data is repeatedly divided until all the partitioned data meets the specified conditions. Then, based on all the characteristic bits, the hash value of each data item is calculated. The hash value is used as the array index, and the data to be processed is stored in a predetermined array (i.e., a hash bucket) within the bit-splitting hash data structure. Regarding how the hash value is calculated, the values ​​corresponding to all the characteristic bits in the data to be processed can be extracted, and a hash operation can be performed on these values ​​using a certain algorithm to obtain the hash value. There are no restrictions on the algorithm used to calculate the hash value.

[0049] For example, the pext value of each piece of data to be processed is calculated based on all feature bits, and the data to be processed is stored in a predetermined array using the pext value as the array index. When performing data matching, the pext value of the corresponding feature bit in the data is retrieved, used as the array index, and matched against the corresponding array element.

[0050] As can be seen from the technical concept of the bit-cut hash algorithm, when a data structure with only one level, where each leaf node in this level has at most one rule (referred to as level 1 rule), is constructed using the bit-cut hash algorithm, this data structure is a hash table capable of completely hashing all rules (named Bitcuts Hash), hence the name bit-cut hash data structure. The above-mentioned partitioning idea, under the condition of only performing exact matching and with sufficient memory, can complete this evolution to obtain a bit-cut hash data structure that completely hashes all rules, provided that the newly added rule is different from any existing rule, provided that no new rule is added.

[0051] When the first piece of data to be processed is added to the bit-cut hash data structure, the number of feature bits P is 0. The bit-cut hash algorithm can completely hash all the data to be processed onto 2^0 = 1 hash buckets.

[0052] When adding a second piece of data to be processed to a bit-cut hash data structure, the number of feature bits P is 1. The bit-cut hash algorithm can completely hash all the data to be processed onto 2^1 = 2 hash buckets.

[0053] When a third piece of data to be processed is added to the bit-cut hash data structure, the number of feature bits P is 2. The bit-cut hash algorithm can completely hash all the data to be processed onto 2^2 = 4 hash buckets.

[0054] When a fourth piece of data to be processed is added to the bit-cut hash data structure, the number of feature bits P is 2. The bit-cut hash algorithm can completely hash all the data to be processed onto 2^2 = 4 hash buckets.

[0055] When the fifth piece of data to be processed is added to the bit split hash data structure, the number of feature bits P is 3. The bit split hash algorithm can completely hash all the data to be processed onto 2^3 = 8 hash buckets.

[0056] Similarly, when adding the 'a'-th piece of data to the bit-cut hash data structure, the number of feature bits P is b. The bit-cut hash algorithm can completely hash all the data to be processed across 2^b hash buckets, with each hash bucket containing at most one piece of data. For example, when adding the 'a'-th piece of data, if the hash bucket for the 'a'-th piece of data is empty (i.e., there is no collision with the 'a'-th piece of data), then the 'a'-th piece of data can be placed into that hash bucket. Alternatively, if the hash bucket for the 'a'-th piece of data is occupied (i.e., there is a collision), then one feature bit is added, bringing the total number of feature bits to b, thus completely hashing all the data to be processed across 2^b hash buckets.

[0057] In summary, each time a piece of data to be processed is added to the bit-splitting hash data structure, the number of feature bits can remain constant or increase. If the number of feature bits remains constant, the hash buckets for data already in place do not need to be updated; only the hash buckets for the newly added data need to be determined. If the number of feature bits increases, then the hash buckets for each piece of data to be processed need to be redefined, meaning that the hash buckets for all data to be processed need to be updated.

[0058] 2. Preprocessing for the bit splitting hash algorithm.

[0059] In one example, the key to the bit-cut hash algorithm is to determine the number of feature bits. After the number of feature bits is determined, feature bits corresponding to that number are selected, and all data to be processed are hashed into different hash buckets based on the feature bits, ensuring that each hash bucket contains at most one piece of data to be processed.

[0060] The number of feature bits determines the number of hash buckets; for example, if the number of hash buckets is 2... P P represents the number of feature bits. Obviously, the number of feature bits P cannot be infinitely large. To support millions to tens of millions of IPv6 prefixes, the maximum number of feature bits P can be set to 24. This results in 16,777,216 hash buckets, which can hold 16 million routes, with an estimated memory usage of 80MB. Of course, the maximum number of 24 is just an example; there is no limit to the maximum number of feature bits P. The number of feature bits P can vary depending on the device's memory size and can be configured according to actual needs.

[0061] In one example, the required feature bits can be selected through a calculation process, with two termination conditions: 1) reaching the maximum number of feature bits, such as 24; 2) all the data to be processed can be completely flattened into the bit-cut hash data structure, and each hash bucket of the bit-cut hash data structure contains at most one piece of data to be processed. As long as either of the above two conditions is met, the feature bits can be obtained.

[0062] In one example, all rules used for exact matching (here, "rule" is a general term, equivalent to the IPv6 address data awaiting processing) need to have a consistent length and match the data to be matched. In IPv6 FIB query scenarios, this means the length of the IPv6 address, such as 16 bytes.

[0063] 3. The selection of the optimal feature bits for the bit-cut hash algorithm, i.e., determining the target number of bits P. For example, for newly added data to be stored in the bit-cut hash data structure (e.g., needing to add new data to be stored in the bit-cut hash data structure), the target number of bits P can be determined.

[0064] In one example, to determine the target number of bits P, the following steps can be taken:

[0065] Step 1: Determine the BSS value for each bit based on the data to be processed (i.e., all stored data and newly added data to be stored in the bit-cut hash data structure).

[0066] In one example, each time new data is added to be stored, the BSS value for each bit needs to be recalculated incrementally. For an IPv6 address with 128 bits, the BSS value for all 128 bits needs to be calculated.

[0067] When adding a new piece of data to be stored, the BSS value of the i-th bit can be calculated as follows: if the i-th bit of the newly added data is 0, then the BSS... i =(Num0 i +1)×Num1 i If the i-th bit of the newly added data to be stored is 1, then the BSS i =Num0 i ×(Num1 i +1).

[0068] When deleting one piece of stored data, the BSS value of the i-th bit can be calculated as follows: if the i-th bit of the deleted stored data is 0, then the BSS... i =(Num0 i -1)×Num1 i If the i-th bit of the deleted stored data is 1, then the BSS i =Num0 i ×(Num1 i -1).

[0069] For example, in the formula above, i can represent the i-th bit, BSS i It can represent the BSS value of the i-th bit, Num0 i It can represent the number / quantity of data to be processed where the value of the i-th bit is 0, Num1 i It can represent the number / quantity of data to be processed where the value of the i-th bit is 1.

[0070] Regarding the performance estimation in step 1, the number of calculations equals the bit length of the data to be processed. For example, if the bit length of the data to be processed is 128 bits, then the number of calculations is 128, which is time-efficient. Regarding the memory estimation in step 1, each bit of the hash data structure requires an array of 128 elements. Each array element is used to record the bit number (1 byte), BSS value (8 bytes), number of 0 bits (4 bytes), and number of 1 bits (4 bytes) for each bit, occupying a total of 2176 bytes of memory, which is relatively small.

[0071] Step 2: Sort all bits in descending order of BSS value.

[0072] For example, after calculating the BSS value of all bits, sort all bits in descending order of BSS value. Since only 128 bits need to be sorted, it is time-saving.

[0073] Step 3: Determine the target number of bits P based on the sorting results of all bits. The target number of bits P is used to ensure that there are no hash collisions between the data to be stored and the data already stored in the bit-cut hash data structure.

[0074] In one example, method 1 for determining the target number of bits P:

[0075] The configured starting value is set as the current value. The first few bits of the current value are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision. If so, the sum between the current value and the fixed value is updated to the current value, and the operation of selecting the first few bits of the current value as feature bits is returned. If not, the current value is set as the target number of bits P.

[0076] For example, the initial value can be 1, and the fixed value can also be 1. Let 1 be the current value. Select the first bit in the sorted sequence as the feature bit, and determine if a hash collision exists based on the feature bit. If not, the target number of bits P is 1. If yes, update 2 to the current value, select the first two bits in the sorted sequence as the feature bits, and determine if a hash collision exists based on the feature bit. If not, the target number of bits P is 2. If yes, update 3 to the current value, and so on, until it is determined that no hash collision exists based on the feature bit, and the current value is then set as the target number of bits P.

[0077] In one example, method 2 for determining the target number of bits P:

[0078] The configured starting value is determined as the current value, and the first few bits of the current value are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision.

[0079] If so, update the current value to the first type of value; if twice the current value is less than the configured fixed value, update twice the current value to the current value; otherwise, update the fixed value to the current value and return to perform the operation of selecting the first few bits of the current value as the feature bits.

[0080] If not, update the current value to the second type of value and determine whether the difference between the second type of value and the first type of value is 1; if it is 1, determine the second type of value as the target number of bits P; if it is not 1, determine a reference value based on half of the sum of the first type of value and the second type of value (e.g., the floor value of half of the sum of the first type of value and the second type of value can be used as a reference value), and select the first few bits of the reference value as feature bits, and determine whether there is a hash collision based on the feature bits; if there is, update the reference value to the first type of value and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1; if there is no collision, update the reference value to the second type of value and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1.

[0081] For example, in method 2 for determining the target number of bits P, instead of trying them one by one in a linear increment, the optimal feature bits are tried using the "double-halving method" to obtain the target number of bits P. For example, see Table 1, which shows an example of obtaining the target number of bits P using the "double-halving method". In Table 1, the maximum number of feature bits P is 24, as an example.

[0082] Table 1

[0083]

[0084]

[0085] In Table 1, the first column represents the actual number of feature bits required to spread out all rules (i.e., all data to be processed), which is also the number of attempts needed to divide all rules. The second column is the feature bit selection algorithm, which determines the target number of bits P. Since step 2 sorted the data in descending order of BSS value, we can try to spread out all rules using more feature bits in descending order of BSS value. The third column represents the number of attempts, indicating how many attempts are needed to determine the target number of bits P. The fourth column represents the maximum number of rules that can be stored, i.e., the maximum number of rules that can be stored based on the optimal number of feature bits, indicating the maximum amount of data to be processed that can be stored based on the optimal number of feature bits.

[0086] Regarding method 2 for determining the target number of bits P, the basic idea is: the more bits selected, the better it is guaranteed that all rules (i.e., the data to be processed) can be completely tiled on the array; that is, selecting more points will not affect the tiling result. The processing logic is: after trying N feature bits and failing to tile all rules, the next attempt involves (2×N) feature bits. If successful, the N and (2×N) feature bits are halved, resulting in (1.5×N) bits, and the attempt is repeated. If unsuccessful, another 2×(2×N) feature bits are tried, and so on. A special handling is used: when the number of feature bits (2×N) is greater than the preset maximum value (i.e., the maximum number of feature bits P, 24), the maximum value is used, and subsequent attempts are based on the maximum value and N, involving halving. The termination condition is: after a certain attempt, the number of feature bits that can tile all rules is exactly greater than the number of feature bits that could not tile all rules in the last attempt by exactly one greater than the number of feature bits that could not tile all rules in the last attempt.

[0087] The time complexity of method 2 for determining the target number of bits P is O(n). B represents the actual number of feature bits, R represents the total number of rules (i.e., the total amount of data to be processed), and +1 is because all rules need to be traversed again at the end for the final partitioning operation. For example, selecting a suitable number of feature bits from an IPv6 address requires a minimum of 1 attempt and a maximum of 9 attempts.

[0088] In this example, during algorithm initialization, a block of size (2) is allocated. MaxBitNumEight bytes of memory are allocated, which is used as a bitmap for subsequent pext value calculations. While traversing all rules, for each rule, the corresponding pext value is calculated using the attempted feature bits (pext is a high-level instruction that only consumes 3 CPU cycles). After calculation, the value is set into the aforementioned bitmap. If the same bitmap position is repeatedly set during the calculation process, it indicates that the current number of feature bits is being used for tiling, resulting in one bucket containing two rules within the bitcuts hash buckets. This violates the "one rule per layer" requirement, and the current attempt is terminated. Based on this, more feature bits can be tried. Assuming the maximum number of feature bits is 24, the allocated bitmap memory size is 2MB, eliminating the need to allocate one bitmap for each bitcuts hash.

[0089] The following explanation, based on the optimal number of feature bits 23 in Table 1, illustrates method 2 for determining the target number of bits P. In this example, the initial value is configured as 1, and the fixed value is 24.

[0090] First round of attempts: The configured starting value 1 is set as the current value. The first bit in the sorted sequence is selected as the feature bit. Based on the feature bit, it is determined whether there is a hash collision. If there is a hash collision, the current value 1 is updated to the first type of value, and the second round of attempts is executed.

[0091] Second round of attempts: If twice the current value 1 is less than the configured fixed value 24, update twice the current value (i.e., value 2) to the current value. Select the top two bits as feature bits. Determine whether there is a hash collision based on the feature bits. If there is a hash collision, update the current value 2 to the first type of value (i.e., the first type of value has changed from 1 to 2) and execute the third round of attempts.

[0092] Third attempt: If twice the current value 2 is less than the fixed value 24, update twice the current value to the current value. Select the top 4 bits as feature bits. If a hash collision is determined based on the feature bits, update the current value 4 to the first type of value and execute the fourth attempt.

[0093] Fourth round of attempts: If twice the current value 4 is less than the fixed value 24, update twice the current value to the current value. Select the top 8 bits as feature bits. If a hash collision is determined based on the feature bits, update the current value 8 to the first type of value and execute the fifth round of attempts.

[0094] Fifth attempt: If twice the current value 8 is less than the fixed value 24, update twice the current value to the current value. Select the top 16 bits as feature bits. If a hash collision is determined based on the feature bits, update the current value 16 to the first type of value and execute the sixth attempt.

[0095] Sixth round of attempts: If twice the current value 16 is greater than the fixed value 24, update the fixed value 24 to the current value. Select the top 24 bits as feature bits. If it is determined based on the feature bits that there is no hash collision, then update the current value 24 to the second type of value. Then, determine whether the difference between the second type of value (e.g., 24) and the first type of value (e.g., 16) is 1. If not, execute the seventh round of attempts.

[0096] The seventh attempt: Take the floor value of half the sum of the first type of value 16 and the second type of value 24 as a reference value (e.g., the reference value is 20), and select the first 20 bits as feature bits. Then, based on the feature bits, determine if there is a hash collision. If there is a hash collision, the reference value 20 can be updated to the first type of value. Then, determine if the difference between the second type of value (e.g., 24) and the first type of value (e.g., 20) is 1. If not, the eighth attempt can be performed.

[0097] Eighth round of attempts: The floor value of half the sum of the first type of value 20 and the second type of value 24 is taken as a reference value (e.g., the reference value is 22), and the first 22 bits are selected as feature bits. Then, based on the feature bits, it is determined whether a hash collision exists. If a hash collision exists, the reference value 22 can be updated to the first type of value. Then, it is determined whether the difference between the second type of value (e.g., 24) and the first type of value (e.g., 22) is 1. If not, the ninth round of attempts can be executed.

[0098] Ninth round of attempts: The floor value of half the sum of the first type of value 22 and the second type of value 24 is taken as a reference value (e.g., the reference value is 23), and the first 23 bits are selected as feature bits. Then, it can be determined whether a hash collision exists based on the feature bits.

[0099] If a hash collision occurs, the reference value 23 can be updated to the first type of value. Then, it is determined whether the difference between the second type of value (e.g., 24) and the first type of value (e.g., 23) is 1. If so, the second type of value 24 is determined as the target number of bits P, that is, the target number of bits P is 24.

[0100] If no hash collision occurs, the reference value 23 can be updated to a second-type value. Then, it is determined whether the difference between the second-type value (e.g., 23) and the first-type value (e.g., 22) is 1. If so, the second-type value 23 is determined as the target number of bits P, i.e., the target number of bits P is 23. In Table 1, the reference value 23 is updated to a second-type value, and the target number of bits P is 23.

[0101] In one example, method 3 for determining the target number of bits P:

[0102] Select the top-ranked configured target value bits as feature bits. The target value can be greater than 1. Determine whether there is a hash collision based on the feature bits.

[0103] If not, the target value can be determined as the target number of bits P.

[0104] If so, the target value can be updated to the first type of value. Based on the target value, the current value is determined, and the top-ranked bits of the current value are selected as feature bits. A hash collision is then determined based on the feature bits. If so, the current value is updated to the first type of value. If twice the current value is less than the configured fixed value, twice the current value is updated to the current value; otherwise, the fixed value is updated to the current value, and the process of selecting the top-ranked bits of the current value as feature bits is returned. If not, the current value is updated to the second type of value, and the difference between the second type of value and the first type of value is determined. If the difference between the two types of values ​​is 1, then the second type of value is determined as the target number of bits P. If it is not 1, then a reference value is determined based on half the sum of the first and second types of values. The first few bits of the reference value are selected as feature bits, and a hash collision is determined based on the feature bits. If a collision exists, then the reference value is updated to the first type of value, and the operation of determining whether the difference between the second and first types of values ​​is 1 is performed. If no collision exists, then the reference value is updated to the second type of value, and the operation of determining whether the difference between the second and first types of values ​​is 1 is performed.

[0105] For example, to improve the performance of selecting feature bits and reduce memory usage, in method 3 for determining the target number of bits P, it is not necessary to start trying the tiling rule from the first feature bit. Depending on the device memory size, the tiling rule can be tried from the Nth feature bit, where N represents the target value, which can be configured empirically. Thus, the top N feature bits sorted by BSS value are selected as the starting value for the trial. Taking a target value N of 8 as an example, see Table 2 for an example of obtaining the target number of bits P. In Table 2, the maximum number of feature bits P is 24, as an example.

[0106] Table 2

[0107]

[0108]

[0109] As can be seen from Table 2, compared with method 2 for determining the target number of bits P, only one attempt is needed when 8 or fewer (including 8) feature bits are actually required. When 9 to 16 feature bits are required, only 5 attempts are needed, which is 3 fewer attempts than method 2; when 17 to 24 feature bits are required, only 6 attempts are needed, which is 3 fewer attempts than method 2.

[0110] The following explanation, based on the optimal number of feature bits 23 in Table 2, details method 3 for determining the target number of bits P. The configured starting value is 1, the target value is 8, and the fixed value is 24.

[0111] First round of attempts: The configured target value of 8 is determined as the current value. The top 8 bits are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision. If there is a hash collision, the current value of 8 is updated to the first type of value, and the second round of attempts is executed.

[0112] Second round of attempts: If twice the current value 8 is less than the fixed value 24, update twice the current value to the current value. Select the top 16 bits as feature bits. If a hash collision is determined based on the feature bits, update the current value 16 to the first type of value and execute the third round of attempts.

[0113] Third round of attempts: If twice the current value 16 is greater than the fixed value 24, update the fixed value 24 to the current value. Select the top 24 bits as feature bits. If it is determined based on the feature bits that there is no hash collision, then update the current value 24 to the second type of value. Then, determine whether the difference between the second type of value (e.g., 24) and the first type of value (e.g., 16) is 1. If not, execute the fourth round of attempts.

[0114] The fourth attempt: Take the floor value of half the sum of the first type of value 16 and the second type of value 24 as a reference value (e.g., the reference value is 20), and select the first 20 bits as feature bits. Then, based on the feature bits, determine if there is a hash collision. If there is a hash collision, the reference value 20 can be updated to the first type of value. Then, determine if the difference between the second type of value (e.g., 24) and the first type of value (e.g., 20) is 1. If not, the fifth attempt can be performed.

[0115] Fifth round of attempts: Take the floor value of half the sum of the first type of value 20 and the second type of value 24 as a reference value (e.g., the reference value is 22), and select the first 22 bits as feature bits. Then, based on the feature bits, determine whether there is a hash collision. If there is a hash collision, the reference value 22 can be updated to the first type of value. Then, determine whether the difference between the second type of value (e.g., 24) and the first type of value (e.g., 22) is 1. If not, the sixth round of attempts can be executed.

[0116] The sixth attempt: Take the floor value of half the sum of the first type of value 22 and the second type of value 24 as a reference value (e.g., the reference value is 23), and select the first 23 bits as the feature bits. Then, it can be determined whether a hash collision exists based on the feature bits.

[0117] If a hash collision occurs, the reference value 23 can be updated to the first type of value. Then, it is determined whether the difference between the second type of value (e.g., 24) and the first type of value (e.g., 23) is 1. If so, the second type of value 24 is determined as the target number of bits P, that is, the target number of bits P is 24.

[0118] If there is no hash collision, the reference value 23 can be updated to the second type of value. Then, determine whether the difference between the second type of value (e.g., 23) and the first type of value (e.g., 22) is 1. If so, the second type of value 23 is determined as the target number of bits P, that is, the target number of bits P is 23.

[0119] In summary, for method 3 of determining the target number of bits P, first determine 8 bits of the target value as feature bits. If it is determined based on the feature bits that there is no hash collision, then directly use the target value of 8 as the target number of bits P. If it is determined based on the feature bits that there is a hash collision, then the "double halving method" of method 2 is used to determine the target number of bits P. This process will not be elaborated further.

[0120] In one example, method 4 for determining the target number of bits P:

[0121] Select the top-ranked configured target value bits as feature bits. The target value can be greater than 1. Determine whether there is a hash collision based on the feature bits.

[0122] If not, the target value can be determined as the target number of bits P.

[0123] If so, the target value is updated to the first type of value, and the top-ranked configured performance boundary bits are selected as feature bits. Based on the feature bits, it is determined whether a hash collision exists.

[0124] If no hash collision occurs, update the performance boundary value to the second type of value and determine if the difference between the second type of value and the first type of value is 1. If it is 1, determine the second type of value as the target number of bits P. If it is not 1, determine a reference value based on half the sum of the first type of value and the second type of value, select the first few bits of the reference value as feature bits, and determine if a hash collision occurs based on the feature bits. If a collision occurs, update the reference value to the first type of value and return to execute the operation to determine if the difference between the second type of value and the first type of value is 1. If a collision does not occur, update the reference value to the second type of value and return to execute the operation to determine if the difference between the second type of value and the first type of value is 1.

[0125] If a hash collision occurs, the top-ranked configured memory threshold bits are selected as feature bits. The existence of a hash collision is determined based on these feature bits. For example, the memory threshold can be greater than the performance boundary value. If not, the memory threshold can be determined as the target number of bits P.

[0126] For example, in order to improve the performance of selecting feature bits and reduce memory usage, in method 4 for determining the number of target bits P, it is not necessary to start trying the tiling rule from the first feature bit. Depending on the device memory size, the tiling rule can be tried from the Nth feature bit, where N represents the target value, such as 8.

[0127] In method 4 for determining the target number of bits P, two more concepts can be introduced: the performance boundary point and the memory critical point of the number of characteristic bits, which can also be called the performance boundary value and the memory critical value.

[0128] Regarding performance boundary values: The performance of selecting feature bits depends not only on the number of selections but also on the actual number of rules. The more rules there are, the more feature bits are needed, and the fewer selection attempts should be attempted. When the number of feature bits is small, a few more attempts have little impact on performance. Based on this, a performance boundary value can be set for the number of feature bits, such as 16. Of course, the performance boundary value can be set according to actual needs, such as based on CPU processing performance. With a performance boundary value of 16, a maximum of 65536 rules can be completely spread across the hash bucket.

[0129] Regarding memory thresholds: In CPU-centric computer architectures, memory access performance is a crucial factor affecting software performance. For example, the access performance of CPU caches such as L1, L2, and L3 is more than 10 times higher than that of DRAM memory. Fully utilizing the CPU cache can significantly improve software processing performance. However, CPU cache size is relatively small; for example, the L3 cache is only 8MB. Therefore, whether bitcut hashes can be fully placed in the CPU cache is a critical factor in improving lookup performance.

[0130] In this embodiment, the bitcuts hash bucket stores relative indexes pointing to the actual data, with a size of 4 bytes. Therefore, the memory threshold can be 20, resulting in a memory footprint of 4 × 2 bytes for the bitcuts hash bucket. 20 =4MB, which can be completely placed in the L3 cache. Of course, the memory threshold can also be set according to actual needs, such as based on the CPU cache size, and there is no restriction on it.

[0131] After obtaining the performance boundary value and the memory threshold, the feature bits can be selected in the following way: when the number of attempts to select the feature bits exceeds the performance boundary value, the "double-half method" is no longer used. Instead, the memory threshold is directly tried. If it is satisfied, it can be used directly. If it is not satisfied, that is, after exceeding the memory threshold, considering the possibility of "a small number of rules requiring many feature bits", the number of attempts is based on the actual number of rules until the maximum value of the feature bits is reached.

[0132] See Table 3 for an example of obtaining the target number of bits P.

[0133] Table 3

[0134]

[0135]

[0136] As can be seen from Table 3, compared with method 3 for determining the number of target bits P, when 17 to 24 feature bits are needed, only 2 attempts are required, which is 4 fewer attempts than method 3.

[0137] The following explanation, in conjunction with Table 3, details method 4 for determining the target number of bits P. The configured initial value is 1, the target value is 8, the fixed value is 24, the performance boundary value is 16, and the memory critical value is 20.

[0138] First round of attempts: The configured target value of 8 is set as the current value. The top 8 bits are selected as feature bits. Based on the feature bits, it is determined whether a hash collision exists. If no hash collision exists, the target value of 8 can be directly set as the target number of bits P. If a hash collision exists, the current value (target value) 8 can be updated to the first type of value, and the second round of attempts can be executed.

[0139] The second round of testing involves selecting the top 16 performance boundary values ​​as feature bits and determining whether a hash collision exists based on these feature bits. If no hash collision occurs, the performance boundary value 16 is updated to a second-type value, and the difference between the second-type value 16 and the first-type value 8 is checked to see if it is 1. If it is not 1, the "double-halving method" is used for testing, which will be explained below.

[0140] For example, the floor value of half the sum of the first type of value 8 and the second type of value 16 is taken as the reference value (12). The first 12 bits are selected as the feature bits, and the existence of a hash collision is determined based on the feature bits. If a collision exists, the reference value is updated to the first type of value, and the operation of determining whether the difference between the second type of value and the first type of value is 1 continues. If a collision does not exist, the reference value is updated to the second type of value, and the operation of determining whether the difference between the second type of value and the first type of value is 1 continues.

[0141] This process continues until the difference between the second type of value and the first type of value is 1. The second type of value is then determined as the target number of bits P. This process is described in Method 2 and Method 3, and will not be repeated here.

[0142] In the second round of attempts, after selecting the top 16 performance boundary values ​​as feature bits, if a hash collision is determined based on the feature bits, a third round of attempts is executed.

[0143] The third round of testing involves selecting the top 20 memory threshold bits as feature bits and determining whether hash collisions exist based on these feature bits. If no hash collisions occur, this memory threshold can be set as the target number of bits, P. For example, if hash collisions occur at 16 feature bits but not at 20, then the memory threshold of 20 can be set as the target number of bits, P.

[0144] If a hash collision occurs, the upper limit of the number of bits, 24, can be used as the target number of bits P. Alternatively, the memory threshold of 20 can be used as the first type of value, and the upper limit of the number of bits, 24, as the second type of value. The "double halving method" can be used for testing. The process of the "double halving method" will not be described in detail here. The target number of bits P can also be determined in the following way (see Table 3):

[0145] The first 21 bits (i.e., the memory threshold 20 as the current value, incremented by 1) are selected as feature bits. Based on these feature bits, it is determined whether a hash collision exists. If no hash collision exists, the current value 21 is set as the target number of bits P. If a hash collision exists, the first 22 bits (i.e., the current value incremented by 1) are selected as feature bits. Based on these feature bits, it is determined whether a hash collision exists. If no hash collision exists, the current value 22 is set as the target number of bits P. If a hash collision exists, the first 23 bits (i.e., the current value incremented by 1) are selected as feature bits. Based on these feature bits, it is determined whether a hash collision exists. If no hash collision exists, the current value 23 is set as the target number of bits P. If a hash collision exists, the upper limit of the number of bits, 24, is set as the target number of bits P.

[0146] In one example, method 5 for determining the target number of bits P: The target number of bits P can be determined based on the total number of data to be stored and all stored data within the bit-splitting hash data structure (i.e., the total number of data to be processed). For instance, based on the stored data and the data to be stored, determine the BSS value of each bit, and sort all bits in descending order of BSS values; determine the numerical range of the target total number and the corresponding candidate number; select the top-ranked candidate bits as feature bits; if it is determined based on the feature bits that there are no hash collisions, then the target number of bits P is determined as that candidate number. For example, the numerical range can be from 256K to 1M, with 20 candidates for this range; the numerical range can be from 1M to 2M, with 21 candidates for this range; the numerical range can be from 2M to 4M, with 22 candidates for this range; the numerical range can be from 4M to 8M, with 23 candidates for this range; and the numerical range can be no less than 8M, with 24 candidates for this range. Of course, the above are just a few examples of numerical ranges and candidate numbers, and there are no restrictions on the numerical ranges and candidate numbers.

[0147] For example, the target number of bits P can be determined in the following way:

[0148] If the total number of targets is less than 256K, the top-ranked configured target value bits are selected as feature bits, the target value is greater than 1, and a hash collision is determined based on the feature bits; if not, the target value can be determined as the number of target bits P. If the total target quantity is not less than 256K and less than 1M, then the first 20 bits are selected as feature bits. If no hash collision is determined based on the feature bits, the target bit quantity P is determined to be 20. If the total target quantity is not less than 1M and less than 2M, then the first 21 bits are selected as feature bits. If no hash collision is determined based on the feature bits, the target bit quantity P is determined to be 21. If the total target quantity is not less than 2M and less than 4M, then the first 22 bits are selected as feature bits. If no hash collision is determined based on the feature bits, the target bit quantity P is determined to be 22. If the total target quantity is not less than 4M and less than 8M, then the first 23 bits are selected as feature bits. If no hash collision is determined based on the feature bits, the target bit quantity P is determined to be 23. If the total target quantity is not less than 8M, then the first 24 bits are selected as feature bits. If no hash collision is determined based on the feature bits, the target bit quantity P is determined to be 24.

[0149] For example, a special rule set might lead to a situation where "a small number of rules require a large number of feature bits," which would waste memory resources. Taking a diagonal rule set as an example, a diagonal rule set could include:

[0150] Rule 1: 1000……0000

[0151] Rule 2: 0100……0000

[0152] Rule 3: 0010……0000

[0153] Rule 4: 0001……0000

[0154] ...

[0155] Rule 125: 0000……1000

[0156] Rule 126: 0000……0100

[0157] Rule 127: 0000……0010

[0158] Rule 128: 0000……0001

[0159] If all 128 rules (i.e., the data to be processed) were laid out flat, 128 feature bits would be needed, far exceeding the maximum number of bits of 24. However, as more rules are added, the required number of feature bits can be reduced. Therefore, considering memory usage, the feature bit selection algorithm is as follows:

[0160] If the total number of data to be processed is less than 256K, and the memory threshold (e.g., 20) bits cannot completely cover all rules, then the "double-halving method" is used, such as method 2, method 3, or method 4 for determining the number of target bits P. For example, select the top-ranked target value (8 bits) as feature bits, and determine whether there is a hash collision based on the feature bits. If not, determine the target value as the number of target bits P. If yes, the processing procedure is the same as method 2, method 3, or method 4.

[0161] If the total number of target bits is not less than 256K and less than 1M, then try the top 20 bits, that is, select the top 20 bits as feature bits. If it is determined that there is no hash collision based on the feature bits, then the target number of bits P is determined to be 20. If there is a hash collision, then continue to try 21 bits, and so on, until it is determined that there is no hash collision based on the feature bits, and the target number of bits P is obtained.

[0162] If the total number of target bits is not less than 1M and less than 2M, then try the top 21 bits, that is, select the first 21 bits in the sorted list as feature bits. If it is determined that there is no hash collision based on the feature bits, then the target number of bits P is determined to be 21. If there is a hash collision, then continue to try 22 bits, and so on, until it is determined that there is no hash collision based on the feature bits, and the target number of bits P is obtained.

[0163] If the total number of target bits is not less than 2M and less than 4M, then try the top 22 bits, that is, select the first 22 bits in the sorted list as feature bits. If it is determined that there is no hash collision based on the feature bits, then the target number of bits P is determined to be 22. If there is a hash collision, then continue to try 23 bits, and so on, until it is determined that there is no hash collision based on the feature bits, and the target number of bits P is obtained.

[0164] If the total number of target bits is not less than 4M and less than 8M, then try the top 23 bits, that is, select the top 23 bits as feature bits. If it is determined that there is no hash collision based on the feature bits, then the target number of bits P is determined to be 23. If there is a hash collision, continue to try 23 bits, and so on, until it is determined that there is no hash collision based on the feature bits, and the target number of bits P is obtained.

[0165] If the total number of target bits is not less than 8M, then try the top 24 bits, that is, select the first 24 bits in the sorted list as feature bits. If it is determined that there is no hash collision based on the feature bits, then the number of target bits P is determined to be 24. If a hash collision exists, it means that the bit split hash algorithm cannot be used.

[0166] In summary, if hash collisions still occur when trying 24 bits, the bit split hash algorithm will not be used. Instead, other hash algorithms will be used to store the data. See the following examples for details.

[0167] Second, the characteristic bits of the bitcuts hash algorithm are determined.

[0168] In one example, based on all the data to be processed (such as stored data and data to be stored), the BSS value of each bit is determined, and P feature bits are determined based on the BSS value of each bit. For example, after the number of feature bits (i.e., the target number of bits P) is determined, feature bits corresponding to that target number of bits can be selected. For example, P feature bits can be selected.

[0169] For example, when adding one piece of data to a bit-splitting hash data structure, the BSS value of all data to be processed is calculated. This involves counting the number of times the first bit of all data is 0 and the number of times the first bit is 1; the product of these two counts is the BSS value of the first bit. Similarly, counting the number of times the second bit is 0 and the number of times the second bit is 1, the product of these two counts is the BSS value of the second bit. This process continues until the BSS value of each bit is obtained, resulting in the BSS value for 128 bits.

[0170] Then, all bits are sorted in descending order of BSS value. Based on the sorted bits, the top P bits are selected as feature bits. For example, when bits 5, 8, 9, 4, 2, etc. are selected as feature bits, "5" indicates that the 5th value in the data to be processed is selected as the feature bit.

[0171] For example, when adding one piece of data to a bit-splitting hash data structure, the BSS value of the i-th bit can be calculated as follows: if the i-th bit of the newly added data is 0, then the BSS... i =(Num0 i +1)×Num1 i If the i-th bit of the newly added data to be processed is 1, then the BSS i =Num0 i ×(Num1 i +1). Furthermore, when deleting one piece of data from a bit-splitting hash data structure, the BSS value of the i-th bit can be calculated as follows: if the i-th bit of the deleted data is 0, then the BSS... i =(Num0 i -1)×Num1 i If the i-th bit of the data to be deleted is 1, then BSS i =Num0 i ×(Num1 i -1).

[0172] In one example, when determining P feature bits based on the BSS value of each bit, all bits can be sorted in descending order of BSS value, and the top P bits in the sorted order can be selected as the P feature bits. Alternatively, all bits can be sorted in descending order of BSS value, and the P feature bits can be selected based on the sorting result. For example, each bit can be iterated through sequentially based on the sorting result as the current bit; if the current bit is a redundant bit, it is prohibited from being selected as a feature bit; if the current bit is not a redundant bit, it is allowed to be selected as a feature bit. In this way, none of the selected P feature bits are redundant bits, meaning that the top P non-redundant bits in the sorted order can be selected as the P feature bits.

[0173] For example, when selecting P feature bits, to find a suitable number of feature bits, one type of feature bit needs to be excluded: redundant feature bits. These redundant feature bits must be eliminated. For any two feature bits, partitioning the rule set by combining these two feature bits yields the same result as partitioning the rule set using the two feature bits individually. Therefore, these two feature bits are called pairwise redundant feature bits. Based on this, only one of these redundant feature bits needs to be selected. In the process of selecting the optimal feature bits (P feature bits), the remaining feature bits need to be excluded to avoid occupying the maximum number of feature bits.

[0174] In one example, the characteristics of redundant feature bits may include:

[0175] Feature 1: In terms of the redundancy feature bits, from a vertical perspective, 01 and 10 can coexist (referred to as "01 redundancy"), and 00 and 11 can coexist (referred to as "00 redundancy"), but (01 or 10) and (00 or 11) cannot coexist. See Table 4 for an example of the redundancy feature bits.

[0176] Table 4

[0177] Rule 1 0 0 0 0 1 0 0 0 Rule 2 0 0 0 0 1 0 0 1 Rule 3 1 1 0 1 0 0 1 1

[0178] In Table 4, bits 4 and 5 are redundant feature bits; they are "01 redundant," meaning both 01 and 10 exist simultaneously. Bits 1 and 2 are redundant feature bits; they are "00 redundant," meaning both 00 and 11 exist simultaneously. Bits 7 and 8 are not redundant feature bits; they do not simultaneously contain either 01 and 10 or 00 and 11. Bits 5 and 8 are also not redundant feature bits; they do not simultaneously contain either 01 and 10 or 00 and 11, but their BSS values ​​are equal. Bits 3 and 6 are not redundant feature bits, or even feature bits, because their BSS values ​​are 0.

[0179] Feature 2: The BSS value of a redundant feature bit cannot be 0, but the BSS values ​​of two redundant feature bits are the same. That is, the number and position of 1 bits in one redundant feature bit are equal to the number and position of 1 bits in another redundant feature bit, or the number and position of 1 bits in one redundant feature bit are equal to the number and position of 0 bits in another redundant feature bit.

[0180] Feature 3: The redundant feature bits have the same BSS value, and after sorting them in descending order of BSS value, the redundant feature bits can be compacted together by assigning redundancy numbers to each pair of redundant feature bits.

[0181] Feature 4: When deleting rules, the impact on redundancy is as follows: If pairs of bits are redundant, deleting a rule does not affect the redundancy of these two bits. If pairs of bits are not redundant, deleting a rule may maintain the non-redundancy of these two bits, or it may make them redundant.

[0182] Feature 5: When adding rules, the impact on redundancy is as follows: If two pairs of bits are not redundant, then adding a new rule will not make these two bits redundant.

[0183] If pairs of bits are redundant, adding a new rule may maintain or destroy this redundancy, rendering the two bits non-redundant. For example, maintaining redundancy hinges on whether the addition of the new rule causes either bit changes simultaneously or remains unchanged for both bits compared to the previous rule. If one bit changes while the other remains unchanged, the redundancy is destroyed, and the two bits become non-redundant. For instance, this change could mean transitioning from 0 to 1 or from 1 to 0.

[0184] For the new redundant feature bits, they can only come from bits with a BSS value of 0, and these bits with a BSS value of 0 are redundant in pairs, because these bits have changed in the new rules.

[0185] Feature 6: The positions of each rule can be arbitrarily changed without affecting the redundancy of each feature bit.

[0186] Feature 7: Redundancy is transitive. For example, if bit1 and bit2 are redundant, and bit2 and bit3 are redundant, then bit1 and bit3 are also redundant. If bit1 and bit2 are redundant, but bit1 and bit3 are not redundant, then bit2 and bit3 are also not redundant. In other words, redundant bits also have clustering properties; sets of different redundant bits are disjoint.

[0187] In summary, when iterating through each bit based on the sorting result as the current bit, if the following conditions are met, the current bit is a redundant bit and it is prohibited from being selected as a feature bit; otherwise, the current bit is not a redundant bit and it is allowed to be selected as a feature bit. For example, if the number and position of 1s corresponding to the current bit in all the data to be processed are equal to the number and position of 1s corresponding to the first bit (i.e., the first bit is sorted first and has already been used as a feature bit) in all the data to be processed; or, if the number and position of 1s corresponding to the current bit in all the data to be processed are equal to the number and position of 0s corresponding to the first bit in all the data to be processed; or, if the number and position of 0s corresponding to the current bit in all the data to be processed are equal to the number and position of 1s corresponding to the first bit in all the data to be processed. Clearly, a redundant bit is defined as follows: any two feature bits, whether combined or used individually, produce the same result when dividing the rule; then these two feature bits are pairwise redundant. Specifically, looking at the vertical axis, the 0s and 1s of two bits are either exactly the same or completely opposite. For example, in the first rule, Bit1, Bit2, and Bit3 are 1, 1, and 0 respectively; in the second rule, they are 1, 1, and 1 respectively; and in the third rule, they are 0, 0, and 1 respectively. Bit1 and Bit2 have the same number of 1 bits (i.e., two bits each) and are in the same position (i.e., both 1s are in the first and second rules; being in the same rule means being in the same position), so they are redundant. However, Bit2 and Bit3 have the same number of 1 bits, but are in different positions (the two 1s of Bit2 are in the first and second rules, while the two 1s of Bit3 are in the second and third rules; being in different rules means being in different positions), so they are not redundant.

[0188] When the above conditions are met, the current bit and the first bit are redundant bits. Since the first bit is ordered first and has already been used as the feature bit, the current bit is not selected as the feature bit, thus avoiding the simultaneous selection of two redundant bits as feature bits.

[0189] When deleting stored data from a bit-splitting hash data structure, the BSS value of each bit is determined based on the remaining stored data in the bit-splitting hash data structure. All bits are sorted in descending order of BSS value, and P feature bits are selected based on the sorting result. Based on the values ​​of the P feature bits in the remaining stored data, the hash bucket corresponding to the remaining stored data in the bit-splitting hash data structure is determined, and the remaining stored data is stored in the hash bucket.

[0190] Based on this, when iterating through each bit according to the sorting result as the current bit, if the current bit is a redundant bit, it is prohibited from being selected as the feature bit; if the current bit is not a redundant bit, it is allowed to be selected as the feature bit. Furthermore, after deleting stored data from the bit-splitting hash data structure, according to characteristic 4, for each deleted stored data, then:

[0191] When the BSS value is not 0, each pair of bits is redundant, and deleting a single stored data does not change the redundancy. For example, if the BSS value of the current bit is not 0, and the current bit is redundant with another bit that has been used as a feature bit, deleting a stored data will still leave the current bit as a redundant bit, meaning the redundancy will not change.

[0192] When the BSS value is not 0, for two non-redundant bits, after deleting the last stored data that caused these two bits to be non-redundant, these two bits will become redundant; otherwise, these two bits will remain non-redundant, meaning that the deleted stored data is not the last stored data that caused these two bits to be non-redundant. For example, if the BSS value of the current bit is not 0, and the current bit and another bit that is already a feature bit are not redundant, after deleting a stored data, if this deleted data is the last data that caused the current bit and another bit to be non-redundant, then the current bit is a redundant bit, meaning these two bits will become redundant. If this deleted data is not the last data that caused the current bit and another bit to be non-redundant, then the current bit is not a redundant bit. If this deleted data does not cause the current bit and another bit to be non-redundant, then the current bit is not a redundant bit, and these two bits remain non-redundant.

[0193] In one example, if the second and third bits of the data to be processed are determined to be either Type I or Type II redundancy based on the previous W data to be processed, the redundant bits of the W+1 data to be processed can be updated as follows when adding a new data to be processed: For Type I redundancy, if the second and third bits of the data to be processed are 00 or 11, then the second and third bits are Type I redundancy; if the second and third bits of the data to be processed are 01 or 10, then the second and third bits are not redundant. For Type II redundancy, if the second and third bits of the data to be processed are 00 or 11, then the second and third bits are not redundant; if the second and third bits of the data to be processed are 01 or 10, then the second and third bits are Type II redundancy. If the second and third bits contain both 01 and 10, then the second and third bits are of type II redundancy; if the second and third bits contain both 00 and 11, then the second and third bits are of type I redundancy.

[0194] For example, see Table 5 for an example of determining redundant bits.

[0195] Table 5

[0196]

[0197] Referring to Table 5, when adding data to be processed 1, since each bit is either 0 or 1, the BSS value is always 0, therefore it is not a redundant bit. When adding data to be processed 2, if the value of each bit in data to be processed 2 is different from that in data to be processed 1, the BSS value is 1, and the maximum BSS value can only be 1. Therefore, all bits with a BSS value of 1 are paired redundant. When adding data to be processed 3, for bits that are already mutually redundant, it is only necessary to confirm whether the pairs are "00 redundant" or "01 redundant" (refer to feature 1) and compare them with the bits of the newly added data to be processed, and so on. Referring to Table 5, if based on the previous W data to be processed, the second and third bits are determined to be either type I redundancy (e.g., 00 redundancy) or type II redundancy (01 redundancy), and both 01 and 10 exist in the second and third bits, then the second and third bits are 01 redundancy; if both 00 and 11 exist in the second and third bits, then the second and third bits are 00 redundancy. Therefore:

[0198] Scenario 1: If the second and third bits are of type I redundancy (e.g., 00 redundancy), when adding a piece of data to be processed, if the second and third bits of that data are 00, then type I redundancy is maintained. If the second and third bits of that data are 11, then type I redundancy is maintained. If the second and third bits of that data are 01, then the second and third bits are not redundant, i.e., the redundancy relationship changes. If the second and third bits of that data are 10, then the second and third bits are not redundant, i.e., the redundancy relationship changes.

[0199] Scenario 2: If the second and third bits are of type II redundancy (e.g., 0 / 1 redundancy), when adding a piece of data to be processed, if the second and third bits of that data are 00, then the second and third bits are not redundant, i.e., the redundancy relationship changes. If the second and third bits of that data are 11, then the second and third bits are not redundant, i.e., the redundancy relationship changes. If the second and third bits of that data are 01, then the second and third bits maintain type II redundancy, remaining type II redundancy. If the second and third bits of that data are 10, then the second and third bits maintain type II redundancy.

[0200] In one example, if the redundancy of the fourth and fifth bits is determined based on the previous W data to be processed, and if the BSS value of the fourth bit is 0 and the BSS value of the fifth bit is 0, then when adding a new data to be processed, the redundancy bits of the W+1 subnet routes are updated as follows: If the fourth and fifth bits of the previous W data to be processed are 00, and the fourth and fifth bits of the current data to be processed are also 00, then the fourth and fifth bits are determined to be Type I redundancy; if the fourth and fifth bits of the previous W data to be processed are 00, and the fourth and fifth bits of the current data to be processed are 11, then the fourth and fifth bits are determined to be Type I redundancy; if the fourth and fifth bits of the previous W data to be processed are 01, and the fourth and fifth bits of the current data to be processed are also 01, then the fourth and fifth bits are determined to be Type II ... also 00, then the redundancy bits of the current data to be processed are determined to be Type II redundancy; if the fourth and fifth bits of the previous W data to be processed are also 00, then the redundancy bits of the current data to be processed are determined to be Type II redundancy; if the fourth and fifth bits of the previous W data to be processed are also 00, then the redundancy bits of the current data to be processed are determined to be Type I redundancy; if the fourth and fifth bits of the previous W data to be processed are also 00, then the redundancy bits of the current data to be processed are determined to be Type II redundancy; if the fourth and fifth bits of the previous W data to be processed are also 01, then the redundancy bits of the current data to be processed are determined to be Type II redundancy; If the fifth bit is 01, and the fourth and fifth bits of the data to be processed are 10, then the fourth and fifth bits are determined to be of type II redundancy. If the fourth and fifth bits of the preceding W data to be processed are 10, and the fourth and fifth bits of the data to be processed are 10, then the fourth and fifth bits are determined to be of type II redundancy. If the fourth and fifth bits of the preceding W data to be processed are 10, and the fourth and fifth bits of the data to be processed are 01, then the fourth and fifth bits are determined to be of type II redundancy. If the fourth and fifth bits of the preceding W data to be processed are 11, and the fourth and fifth bits of the data to be processed are 11, then the fourth and fifth bits are determined to be of type I redundancy. If the fourth and fifth bits of the preceding W data to be processed are 11, and the fourth and fifth bits of the data to be processed are 00, then the fourth and fifth bits are determined to be of type I redundancy.

[0201] For example, see Table 6 for an example of determining redundant bits.

[0202] Table 6

[0203]

[0204]

[0205] Referring to Table 6, for redundant non-feature bits with a BSS value of 0, according to characteristic 5, it is possible to add new redundant feature bits. The redundancy relationship can be confirmed according to Table 6. Similarly, for each piece of data to be processed, for already mutually redundant bits, it is determined whether redundancy will be compromised; for bits with a BSS of 0, it is determined whether new redundant bits will be added. After identifying all redundant feature bits, when selecting feature bits, one can be randomly selected from the multiple mutually redundant feature bits as the feature bits (i.e., P feature bits).

[0206] Referring to Table 6, in the case of redundancy in the fourth and fifth bits, if the BSS value of the fourth bit is 0 and the BSS value of the fifth bit is 0, then:

[0207] Scenario 1: The fourth and fifth bits of the preceding W data entries are 00. When adding a new data entry, if the fourth and fifth bits are 00, then the fourth and fifth bits are determined to be Type I redundancy (00 redundancy), with a BSS value of 0 for both the fourth and fifth bits. If the fourth and fifth bits are 01, then the fourth and fifth bits are determined to be non-redundant. If the fourth and fifth bits are 10, then the fourth and fifth bits are determined to be non-redundant. If the fourth and fifth bits are 11, then the fourth and fifth bits are determined to be Type I redundancy (00 redundancy).

[0208] Scenario 2: If the fourth and fifth bits of the previous W data to be processed are 0 or 1, when adding a new data to be processed, if the fourth and fifth bits of that new data are 00, then the fourth and fifth bits are determined to be non-redundant. If the fourth and fifth bits of that new data are 01, then the fourth and fifth bits are determined to be Type II redundancy (01 redundancy), with a BSS value of 0 for both the fourth and fifth bits. If the fourth and fifth bits of that new data are 10, then the fourth and fifth bits are determined to be Type II redundancy (01 redundancy). If the fourth and fifth bits of that new data are 11, then the fourth and fifth bits are determined to be non-redundant.

[0209] Scenario 3: If the fourth and fifth bits of the previous W data to be processed are 10, when adding a new data to be processed, if the fourth and fifth bits of that new data are 00, then the fourth and fifth bits are determined to be non-redundant. If the fourth and fifth bits of that new data are 01, then the fourth and fifth bits are determined to be Type II redundancy (01 redundancy). If the fourth and fifth bits of that new data are 10, then the fourth and fifth bits are determined to be Type II redundancy (01 redundancy), with a BSS value of 0 for both the fourth and fifth bits. If the fourth and fifth bits of that new data are 11, then the fourth and fifth bits are determined to be non-redundant.

[0210] Scenario 4: If the fourth and fifth bits of the previous W data to be processed are 11, when adding a new data to be processed, if the fourth and fifth bits of that new data are 00, then the fourth and fifth bits are determined to be Type I redundancy (00 redundancy). If the fourth and fifth bits of that new data are 01, then the fourth and fifth bits are determined to be non-redundant. If the fourth and fifth bits of that new data are 10, then the fourth and fifth bits are determined to be non-redundant. If the fourth and fifth bits of that new data are 11, then the fourth and fifth bits are determined to be Type I redundancy (00 redundancy), with both the fourth and fifth bits having a BSS value of 0.

[0211] In one example, regarding the redundancy determination process, the performance estimation is as follows: the time complexity of determining the redundant feature bits is O(N×(N-1) / 2), where N is the bit length of the data to be processed, which is 128 in this case. In the worst case, it requires 8128 comparisons, which is not time-consuming. Regarding memory estimation: each bitcut hash requires an array of 128 elements to record the bit number (1 byte), BSS value (8 bytes), number of 0 bits (4 bytes), number of 1 bits (4 bytes), redundancy number (1 byte), and redundancy count (1 byte) for each bit. An additional 16 bytes are needed to allocate the redundancy number, totaling 2448 bytes of memory, which is relatively small.

[0212] Third, data storage using the bitcuts hash algorithm.

[0213] In one example, after obtaining P feature bits, for newly added data to be stored in the bit-splitting hash data structure, based on the values ​​corresponding to the P feature bits in the data to be stored, the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure is determined, and the data to be stored is stored in the target hash bucket. For example, assuming the P feature bits are 10 feature bits and the data length to be stored is 64, then 10 values ​​corresponding to the 10 feature bits are selected from the 64 values ​​of the data to be stored. A hash algorithm is used to perform a hash operation on these 10 values ​​to obtain a hash value; the hash operation process is not restricted. The hash bucket corresponding to this hash value is used as the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure, and the data to be stored is stored in the target hash bucket.

[0214] In one example, when determining the target hash bucket for the data to be stored, candidate hash buckets corresponding to the data to be stored in the bit-splitting hash data structure can be determined based on the values ​​of P characteristic bits in the data to be stored. If there is no hash collision between the candidate hash bucket and the hash bucket corresponding to the already stored data, the candidate hash bucket is determined as the target hash bucket for the data to be stored. If there is a hash collision between the candidate hash bucket and the hash bucket corresponding to the already stored data, then: if the number of target bits P has not reached the configured upper limit (e.g., the upper limit of the number of bits is 24), the number of target bits P is re-determined. The determination method is the same as in the above embodiment and will not be repeated here. Based on the re-determined number of target bits P, the candidate hash bucket corresponding to the data to be stored in the bit-splitting hash data structure is determined, and so on.

[0215] If the target number of bits P is the configured upper limit, i.e. the upper limit of 24 bits has been reached, and there is a hash collision between the candidate hash bucket and the hash bucket corresponding to the stored data, it means that the bit-cut hash algorithm cannot be used to complete the data storage. In this case, the algorithm can be degraded to the transition hash algorithm to determine the target hash bucket corresponding to the data to be stored in the hash data structure (which is no longer the bit-cut hash data structure, but the hash data structure corresponding to the transition hash algorithm). The content of the transition hash algorithm can be found in the following embodiments.

[0216] For example, when implementing a bitcut hash data structure using the bitcut hash algorithm, the bitcut hash data structure includes multiple hash buckets, and the bitcut hash data structure corresponds to P feature bits. For the data to be stored, based on the values ​​corresponding to the P feature bits in the data to be stored, a candidate hash bucket corresponding to the data to be stored in the bitcut hash data structure is determined. If the candidate hash bucket is idle, it is used as the target hash bucket, and the data to be stored is stored in the target hash bucket. That is, all the data to be stored can be completely spread across the hash buckets, and each hash bucket contains at most one data to be stored. If the candidate hash bucket is occupied, and the number of target bits P has not reached the configured upper limit, a feature bit is added. Based on the added feature bit, all the data to be stored is completely hashed into the hash buckets, and each hash bucket contains at most one data to be stored. Alternatively, if the candidate hash bucket is occupied and the number of target bits P is the configured upper limit, it means that the bitcuts hash algorithm cannot be used to implement the bit-cut hash data structure. In this case, the transition hash algorithm can be used to determine the corresponding hash bucket in the hash data structure for the data to be processed.

[0217] In one example, the bitcuts hash algorithm supports incremental addition and deletion of data to be processed. For incremental deletion of data to be processed, the pext value of the data to be deleted is calculated using the characteristic bits, the hash bucket storing the data to be deleted is located, the data to be deleted is removed from this hash bucket, and the BSS array information of the bitcuts hash data structure is updated. This operation is time-efficient and can meet the update requirements.

[0218] For incremental addition of data to be processed, Case 1: Calculate the pext value of the data to be added using the optimal feature bits. Then, use this pext value as the index of the bitcuts hash bucket array to find the hash bucket where the data to be added should be stored. If this hash bucket does not already contain data, the data to be added is directly stored in this hash bucket. This operation is time-efficient and does not affect query performance. Case 2: Calculate the pext value of the data to be added using the optimal feature bits. Then, use this pext value as the index of the bitcuts hash bucket array to find the hash bucket where the data to be added should be stored. If this hash bucket already contains data, more feature bits need to be selected and all the data to be processed needs to be re-flattened.

[0219] Case 3: Based on Case 2, if the maximum number of feature bits is reached (the configured maximum number) or if all the data to be processed cannot be completely spread out due to memory utilization considerations, a transitional hashing algorithm can be used to determine the corresponding hash bucket in the hash data structure for the data to be processed.

[0220] Fourth, data retrieval using the bitcuts hash algorithm.

[0221] In one example, after selecting the characteristic bits, a Bitcuts Hash query can be performed based on the characteristic bits (e.g., P characteristic bits). For instance, for the data to be queried, based on the values ​​corresponding to the P characteristic bits in the data, the hash bucket corresponding to the data in the bitcuts hash data structure can be determined, and the query operation can be completed by reading the stored data from this hash bucket.

[0222] For example, using the CPU advanced instruction `pext` (which consumes only 3 CPU cycles and takes only single-digit nanoseconds), the values ​​corresponding to P characteristic bits in the data to be queried can be extracted and used as indices in the `bitcuts` hash bucket array to access the corresponding hash bucket index. Then, the corresponding stored data can be found through the bucket index, and an exact match can be performed between the stored data and the data to be queried. If the bucket index is invalid or an exact match cannot be found, the corresponding data will not be found (e.g., the corresponding route cannot be found); otherwise, the route in the hash bucket can be hit, and the forwarding path of the route can be obtained.

[0223] In summary, this embodiment proposes a bitcuts hash algorithm. When new data to be processed is added, the BSS value of the bits of the already stored data is saved, and the BSS value of the bits of the new data to be processed is incrementally calculated. Then, the new BSS values ​​of each bit are sorted in descending order, and the optimal feature bit is selected according to the "double halving method + top N" and other methods, combined with the total amount of data, while redundant feature bits are eliminated. Finally, the optimal feature bit is used to completely spread all the data to be processed into the hash bucket of the bitcuts hash, forming a "1-layer 1-rule" bitcuts hash data structure, that is, a bit-cut hash data structure.

[0224] Fifth, the hash conversion mechanism, which uses different hash algorithms to store data.

[0225] In one example, for memory utilization considerations, with a limited number of rules, when selecting the optimal feature bit, even if it's impossible to completely spread all rules across the bitcuts hash, more feature bits won't be selected to completely spread all rules. Therefore, a hash conversion mechanism is needed to switch between different hash algorithms, thus delaying the reselection of the optimal feature bit due to rule collisions. If collisions are unavoidable, it can degenerate into a higher-performance hash algorithm. At the same time, the overhead of the hash conversion itself (which affects the update performance of the data to be processed) must be fully considered. The hash conversion mechanism mainly switches between three hash algorithms: perfect hash, transitional hash, and regular hash.

[0226] In this embodiment, a perfect hash algorithm can be used to store data, such as the bit-cut hash algorithm, which is a perfect hash algorithm, and the data can be stored using a bit-cut hash data structure. Alternatively, a transitional hash algorithm can be used to store data, such as storing data using a transitional hash algorithm data structure. Or, a conventional hash algorithm can be used to store data, such as storing data using a conventional hash algorithm data structure.

[0227] For example, see Figure 2A The diagram illustrates hash switching. If no hash collisions occur when using the perfect hash algorithm, then the perfect hash algorithm is used to store the data. If hash collisions occur when using the perfect hash algorithm (e.g., when the target number of bits P reaches the configured upper limit of 24), and no hash collisions occur when using the transitional hash algorithm, then the transitional hash algorithm is used to store the data. If hash collisions occur when using both the perfect and transitional hash algorithms, and no hash collisions occur when using the regular hash algorithm, then the regular hash algorithm is used to store the data.

[0228] See Figure 2BThe diagram illustrates hash switching. If no hash collisions occur when using the bitcuts hash algorithm, then the bitcuts hash algorithm can be used to store the data. If hash collisions occur when using the bitcuts hash algorithm (i.e., hash collisions still exist when the target number of bits P reaches the configured upper limit of 24), and no hash collisions occur when using the bitcuts cuckoo hash algorithm, then the bitcuts cuckoo hash algorithm can be used to store the data. If hash collisions occur when using both the bitcuts hash and bitcuts cuckoo hash algorithms, and no hash collisions occur when using the bitcuts cuckoo filter algorithm, then the bitcuts cuckoo filter algorithm can be used to store the data. If hash collisions occur when using both the bitcuts hash and bitcuts cuckoo hash algorithms, and no hash collisions occur when using the bitcuts cuckoo filter algorithm, and no hash collisions occur when using the CRC32 hash algorithm, then the CRC32 hash algorithm can be used to store the data.

[0229] See Figure 2C The diagram shown is a schematic of hash switching. Figure 2C and Figure 2B In contrast, the multi-plybitcuts hash algorithm can be used to replace the bitcuts cuckoo filter algorithm, and the rest are similar.

[0230] For example, in Figure 2B and Figure 2C In this context, the bitcuts hash algorithm can represent a perfect hash algorithm, the bitcuts cuckoo hash algorithm, the bitcuts cuckoo filter algorithm, and the multi-ply bitcuts hash algorithm can represent transitional hash algorithms, and the CRC32 hash algorithm can represent a regular hash algorithm.

[0231] In one example, if the number of feature bits P reaches the maximum of 24 and the bitcuts hash algorithm still experiences collisions (e.g., a hash collision occurs between the candidate hash bucket corresponding to the data to be stored and the hash bucket corresponding to the already stored data), then it degenerates into a transitional hash algorithm to implement the data structure. The transitional hash algorithm can be the bitcuts cuckoo hash algorithm, which will be explained below.

[0232] For example, if a candidate hash bucket corresponding to the data to be stored has a hash collision with the hash bucket corresponding to the already stored data, and the number of target bits P is the configured upper limit (e.g., 24), then the first hash value and the second hash value are determined based on the values ​​corresponding to the P feature bits in the data to be stored. If the first hash value corresponds to the first hash bucket in the bit-splitting hash data structure, and the second hash value corresponds to the second hash bucket in the bit-splitting hash data structure, then the first hash bucket can be determined as the target hash bucket corresponding to the data to be stored, and the data to be stored is stored in the target hash bucket. Alternatively, the second hash bucket can be determined as the target hash bucket corresponding to the data to be stored, and the data to be stored is stored in the target hash bucket.

[0233] For example, when adding one feature bit, the number of hash buckets grows exponentially (2 to the power of P), while the amount of data to be processed grows linearly (one bit at a time). This leads to low memory utilization of Bitcuts Hash, resulting in many empty hash buckets. Although this reduces the probability of hash collisions, a small amount of data to be stored can consume a large amount of memory, reducing query performance. Therefore, the Bitcuts Hash algorithm can be converted to the Bitcuts Cuckoo Hash algorithm. The Bitcuts Cuckoo Hash algorithm improves the space utilization of Bitcuts Hash with minimal impact on query performance.

[0234] When using the bitcuts cuckoo hash algorithm, the hash data structure of the bitcuts cuckoo hash algorithm (bitcuts cuckoo hash is an optimization of bitcuts hash, also known as bit-cut hash data structure) includes multiple hash buckets, and this bit-cut hash data structure can correspond to P feature bits.

[0235] For each piece of data to be processed, a first hash value and a second hash value are determined based on the values ​​corresponding to P feature bits (e.g., 24 bits) in the data. For example, P values ​​corresponding to the P feature bits are selected from all the values ​​in the data to be processed, and a hash algorithm is used to perform a hash operation on these P values ​​to obtain the first hash value. The first hash value is then added to a preset value (e.g., 1) to obtain the second hash value.

[0236] If the first hash value is free in the first hash bucket corresponding to the bit-splitting hash data structure, then the data to be processed (hereinafter referred to as data to be processed 1 for ease of distinction) is determined to correspond to the first hash bucket in the bit-splitting hash data structure, and data to be processed 1 is stored in the first hash bucket. In this case, the first hash bucket is determined as the target hash bucket corresponding to data to be processed 1.

[0237] If the first hash bucket is occupied, and the data to be processed 2 already stored in the first hash bucket has not been moved (i.e., the data to be processed 2 was not moved from another hash bucket to the first hash bucket), then the data to be processed 2 already stored in the first hash bucket is moved to another hash bucket (the two hash values ​​of the data to be processed 2 also correspond to two hash buckets, one hash bucket is the first hash bucket, and the other hash bucket is the third hash bucket, so the data to be processed 2 is moved to the third hash bucket). It is then determined that the data to be processed 1 corresponds to the first hash bucket in the bit-splitting hash data structure, and the data to be processed 1 is stored in the first hash bucket. In this case, the first hash bucket is determined as the target hash bucket corresponding to the data to be processed 1.

[0238] If the first hash bucket is occupied, and the data to be processed 2 stored in the first hash bucket has been moved (i.e., the data to be processed 2 has been moved from another hash bucket to the first hash bucket), and the second hash value is in a free second hash bucket in the bit-cut hash data structure, then the data to be processed 1 corresponds to the second hash bucket in the bit-cut hash data structure, and the data to be processed 1 is stored in the second hash bucket. In this case, the second hash bucket is determined as the target hash bucket corresponding to the data to be processed 1.

[0239] If the first hash bucket is occupied, and the data to be processed 2 stored in the first hash bucket has been moved, and the second hash bucket is occupied, and the data to be processed stored in the second hash bucket has not been moved, then the data to be processed 3 stored in the second hash bucket is moved to another hash bucket. The data to be processed 1 is then determined to correspond to the second hash bucket in the bit-splitting hash data structure, and this data to be processed 1 is stored in the second hash bucket. In this case, the second hash bucket is determined as the target hash bucket corresponding to the data to be processed 1.

[0240] If the first hash bucket is occupied, and the data to be processed 2 stored in the first hash bucket has been moved, and the second hash bucket is occupied, and the data to be processed 3 stored in the second hash bucket has been moved, then the hash bucket corresponding to the data to be processed 1 cannot be determined, that is, the bitcuts cuckoo hash algorithm fails.

[0241] For example, the first hash value is determined based on P characteristic bits, that is, the hash value determined by the bitcuts hash algorithm. The second hash value is the first hash value with an offset plus 1.

[0242] Considering that the first hash value and the second hash value correspond to two adjacent hash buckets, the probability of them existing in a cacheline (the smallest cache unit in the cache) is the highest. This can greatly reduce the probability of CPU cache misses and improve query performance.

[0243] When using the Bitcuts Cuckoo Hash algorithm for querying, two hash values ​​need to be calculated for the same data to be queried. These two hash values ​​can be a first hash value and a second hash value. Two hash buckets are then identified using these two hash values: the first hash bucket corresponding to the first hash value and the second hash bucket corresponding to the second hash value. If at least one of these two hash buckets matches the data to be queried, the query is considered successfully matched; otherwise, the query fails to match.

[0244] In one example, the transitional hash algorithm could be the bitcuts cuckoo filter algorithm, as explained below. If the number of feature bits P reaches the maximum of 24 and the bitcuts hash algorithm still experiences collisions, then it directly degenerates to the bitcuts cuckoo filter algorithm to implement the data structure. Alternatively, the bitcuts hash algorithm can be degenerated to the bitcuts cuckoo hash algorithm first. Even after degenerating to the bitcuts cuckoo hash algorithm, there may still be conflicting data to be processed. For example, if three consecutive data items with the same hash value are added to the bitcuts hash, the addition of the third data item will fail because the data item in the second hash bucket cannot be retrieved. In this case, it degenerates to the bitcuts cuckoo filter algorithm to implement the data structure. The bitcuts cuckoo filter algorithm is explained below.

[0245] For example, if there is a hash collision between the candidate hash bucket corresponding to the data to be stored and the hash bucket corresponding to the stored data, and the number of target bits P is the configured upper limit (e.g., 24), then the hash value is determined based on the values ​​of the P feature bits in the data to be stored; if there is already stored data in the hash bucket corresponding to the hash value in the bit-slicing hash data structure, then the hash bucket can be determined as the target hash bucket corresponding to the data to be stored, and the data to be stored is stored in the target hash bucket.

[0246] For example, by using the Bitcuts Cuckoo Filter algorithm, the depth of the hash bucket can be changed from 1 to 2, meaning that one hash bucket can store two pieces of data instead of just one. For instance, a Bitcuts Cuckoo Filter algorithm can be implemented based on the Bitcuts Hash algorithm, allowing the simultaneous storage of two pieces of data with the same hash value. Alternatively, a Bitcuts Cuckoo Filter algorithm can be implemented based on the Bitcuts Cuckoo Hash algorithm, allowing the simultaneous storage of four pieces of data with the same hash value.

[0247] When using the bitcuts cuckoo filter algorithm, the hash data structure of the bitcuts cuckoo filter algorithm (bitcuts cuckoo filter is an optimization of bitcuts hash, also known as bit-cut hash data structure) includes multiple hash buckets, and this bit-cut hash data structure can correspond to P feature bits.

[0248] For each piece of data to be stored, a hash value is determined based on the values ​​corresponding to P characteristic bits (e.g., 24 characteristic bits) within that data. For example, P values ​​corresponding to the P characteristic bits are selected from all values ​​in the data to be stored, and a hash algorithm is used to perform a hash operation on these P values ​​to obtain the hash value. If the hash bucket corresponding to the hash value in the bit-splitting hash data structure is empty, or if the hash bucket corresponding to the hash value in the bit-splitting hash data structure already stores one piece of data, then it is determined that the data to be stored corresponds to that hash bucket in the bit-splitting hash data structure, and that hash bucket is designated as the target hash bucket for the data to be stored. The data to be stored is then stored in the target hash bucket.

[0249] Alternatively, for each piece of data to be stored, a first hash value and a second hash value are determined based on the values ​​corresponding to P characteristic bits in the data. If the first hash value corresponds to an empty first hash bucket in the bit-splitting hash data structure, then the data to be stored corresponds to the first hash bucket in the bit-splitting hash data structure, and the data to be stored is stored in the first hash bucket.

[0250] If the first hash value has already stored one piece of data in the first hash bucket corresponding to the bit-cut hash data structure (i.e., a stored piece of data), then it can be determined that the data to be stored corresponds to the first hash bucket in the bit-cut hash data structure, and the data to be stored is stored in the first hash bucket.

[0251] If the first hash value has two data stored in the first hash bucket corresponding to the bit split hash data structure (i.e., two stored data), and at least one stored data has not been migrated, then the unmigrated stored data can be migrated to determine the first hash bucket corresponding to the data to be stored in the bit split hash data structure, and the data to be stored can be stored in the first hash bucket.

[0252] If the first hash bucket already stores two pieces of data, and both pieces of data have been migrated, and the second hash value in the bit-cut hash data structure is either free or has already stored one piece of data (i.e., one piece of stored data), then it is determined that the data to be stored corresponds to the second hash bucket in the bit-cut hash data structure, and the data to be stored is stored in the second hash bucket, and so on.

[0253] For example, based on the bitcuts cuckoo filter, the depth of the hash bucket can be changed from 1 to N. In this way, the 2N hash buckets calculated by the two cuckoo hash functions have a total of 2N positions to store data with the same hash value. N can be 2 or greater than 2. The above process takes 2 as an example.

[0254] In one example, the transitional hash algorithm could be a multi-ply bitcuts hash algorithm, as explained below. If a bitcuts hash collision occurs, it can directly degenerate to a multi-ply bitcuts hash algorithm. Alternatively, it can first degenerate to a bitcuts cuckoo hash algorithm; if a bitcuts cuckoo hash collision occurs, it can degenerate to a multi-ply bitcuts hash algorithm. Or, it can first degenerate to a bitcuts cuckoo hash algorithm; if a bitcuts cuckoo hash collision occurs, it can degenerate to a bitcuts cuckoo filter algorithm; if a bitcuts cuckoo filter collision occurs, it can degenerate to a multi-ply bitcuts hash algorithm.

[0255] Therefore, a multi-ply bitcuts hash algorithm can be used. This algorithm calculates multiple feature bit masks (e.g., N masks) based on the bitcuts hash algorithm, allowing for N layers in the multi-layer bitcuts hash. Based on this, if a candidate hash bucket corresponding to the data to be stored has a hash collision with the hash bucket corresponding to the already stored data, and the target bit count P is the configured upper limit (e.g., 24), a next-layer hash bucket set is created. Each hash bucket set includes multiple hash buckets; the feature bits corresponding to the next-layer hash bucket set are different from those corresponding to the current-layer hash bucket set. The address hash bucket within the current-layer hash bucket set is determined based on the feature bits, and this address hash bucket stores the location of the next-layer hash bucket set (meaning the next-layer hash bucket set is found through this location). Based on the value of the feature bits corresponding to the next-layer hash bucket set in the data to be stored, the hash bucket corresponding to the data to be stored in the next-layer hash bucket set is determined; this hash bucket is then designated as the target hash bucket for the data to be stored.

[0256] For example, when implementing a data structure using the multi-ply bitcuts hash algorithm, a first-level hash bucket set is first created. This first-level hash bucket set includes multiple hash buckets. During the process of storing data into the first-level hash bucket set, as the amount of data increases, the target number of bits P corresponding to the first-level hash bucket set increases. For how to determine the target number of bits P, please refer to the above embodiment. For instance, each time data is stored into the first-level hash bucket set, if a hash collision occurs, the target number of bits P is increased.

[0257] This process continues until the number of target bits P reaches the configured upper limit (e.g., 24). When storing data in the first-level hash bucket set again, if there is no hash collision, the data can be stored in the first-level hash bucket set. If there is a hash collision, a next-level hash bucket set (denoted as the second-level hash bucket set) is created below the current level hash bucket set (i.e., the first-level hash bucket set). The address hash bucket in the first-level hash bucket set is determined based on the characteristic bits (e.g., 24 characteristic bits) corresponding to the first-level hash bucket set, and the location of the second-level hash bucket set is stored in the address hash bucket. This location is used to maintain the second-level hash bucket set, and the second-level hash bucket set can be found through this location.

[0258] After creating the second-level hash bucket set, the target number of bits P corresponding to the second-level hash bucket set can be determined. For details on how to determine the target number of bits P, please refer to the above embodiment. Then, the feature bits corresponding to the second-level hash bucket set (i.e., the target number of feature bits P) are determined, and the feature bits corresponding to the second-level hash bucket set are different from the feature bits corresponding to the first-level hash bucket set. For example, if the feature bits corresponding to the first-level hash bucket set are the 1st to 24th feature bits after sorting by BSS value, and assuming the second-level hash bucket set corresponds to 8 feature bits, then the feature bits corresponding to the second-level hash bucket set are the 25th to 32nd feature bits after sorting by BSS value.

[0259] After creating the second-level hash bucket set, the hash bucket corresponding to the data to be stored is determined based on the value of the characteristic bit corresponding to the second-level hash bucket set in the data to be stored. In this way, the data to be stored can be written into the hash bucket in the second-level hash bucket set.

[0260] Then, data continues to be stored in the second-level hash bucket set. During this process, as the amount of data increases, the target bit count P corresponding to the second-level hash bucket set increases. That is, each time data is stored in the second-level hash bucket set, if a hash collision occurs, the target bit count P is increased. This continues until the target bit count P reaches the configured upper limit (e.g., 24). When data is stored in the second-level hash bucket set again, if there is no hash collision, data can be stored. If a hash collision occurs, a new hash bucket set (denoted as the third-level hash bucket set) is created below the current hash bucket set (i.e., the second-level hash bucket set). Based on the characteristic bits corresponding to the second-level hash bucket set (e.g., 24 characteristic bits), the address hash bucket within the second-level hash bucket set is determined, and the position of the third-level hash bucket set is stored in that address hash bucket. This process continues.

[0261] In summary, when implementing a bit-cut hash data structure, the bit-cut hash data structure can include N layers of hash bucket sets, and each layer of hash bucket sets can include multiple hash buckets of the bitcuts hash algorithm, thereby greatly increasing the number of hash buckets.

[0262] In one example, if collisions still occur with the transitional hash algorithm, it degenerates into a regular hash algorithm, which could be a CRC32 hash algorithm. For instance, even after degenerating into the transitional hash algorithm, there may still be conflicting data to be processed. For example, if five consecutive data entries with the same hash value are added to the bitcuts cuckoo filter, the addition of the fifth entry will fail. Therefore, a CRC32 hash (Cyclic Redundancy Check hash) algorithm can be used for data storage. This embodiment does not restrict the implementation process of this CRC32 hash algorithm.

[0263] For example, while conventional hash algorithms offer excellent update performance, their query performance bottleneck lies in hash calculation and hash collisions. Since hardware can support conventional hash algorithms, hardware acceleration methods can be used to speed up the calculation of conventional hashes, resulting in a more uniform distribution of rules after hashing, thus ensuring minimum query performance.

[0264] In summary, to ensure query and update performance while minimizing the overhead of the transformation itself, the hash transformation process can include: adding data to be processed into a bitcuts hash; if adding fails, converting to a bitcuts cuckoo hash and adding the data to the bitcuts cuckoo hash (adding data to the bitcuts cuckoo hash will succeed). This transformation process only modifies the hash type, having minimal impact on update performance. Then, adding data to be processed into a bitcuts cuckoo hash; if adding fails, first attempting to convert to a bitcuts hash and adding the data to the bitcuts hash (adding data to the bitcuts cuckoo hash will succeed). If adding fails, converting to a bitcuts cuckoo filter and adding the data to the bitcuts cuckoo filter (adding data to the bitcuts cuckoo filter will succeed). This transformation process only modifies the hash type. Then, add data to be processed to the bitcuts cuckoofilter. If adding fails, first try converting it to a bitcuts hash and adding the data to the bitcuts hash. If adding succeeds, convert it to a bitcuts hash. If adding fails, convert it to a CRC32 hash and add the data to the CRC32 hash. Adding data to the CRC32 hash will always succeed; the conversion process only modifies the hash type. When converting CRC32 hash to bitcuts hash, since CRC32 hash always succeeds, the conversion is initiated by starting a timer on the control plane to periodically attempt the conversion. When the ratio of the total amount of data to be processed to the number of CRC32 hash buckets reaches a threshold (i.e., the hash bucket space utilization is low), when incrementally adding data to be processed, attempt to convert to bitcuts hash.

[0265] For the process of deleting pending data, hash conversion is not required. However, considering memory utilization, a timer can be started on the control plane to periodically perform bitcuts hash conversion, thereby reclaiming memory. Alternatively, a periodic attempt can be made to convert the CRC32 hash to a bitcuts hash.

[0266] In summary, by converting between bitcuts hash, bitcuts cuckoo hash, bitcuts cuckoo filter, and CRC32 hash, the limitation of bitcuts hash itself in not being able to completely flatten all the data to be processed in a limited memory space can be solved, balancing the query and update performance of the new hash and the overhead of the conversion itself.

[0267] In one example, we will discuss a variant of the bitcuts hash algorithm, namely the multi-ply bitcuts hash algorithm. The multi-ply bitcuts hash algorithm will be explained below.

[0268] For hash construction using the multi-ply bicuts hash algorithm, compared to the bitcuts hash algorithm, the multi-ply bicuts hash algorithm has multiple feature bit masks (one feature bit mask corresponds to P feature bits in the above embodiment), and each feature bit mask corresponds to one layer of hash buckets. For example, with two feature bit masks, there are two layers of hash buckets. The selection method for multiple feature bit masks is the same as that for the bitcuts hash algorithm.

[0269] For example, after calculating all feature bits and eliminating redundant feature bits, the current feature bits are divided into segments according to their BSS values ​​in descending order (excluding bits with a BSS value of 0). Each segment contains at most P (e.g., 24) feature bits. Each segment of feature bits serves as the feature bit mask for different levels of hash buckets. For example, feature bits 1-24 serve as the feature bit mask for the first-level hash bucket, feature bits 25-48 serve as the feature bit mask for the second-level hash bucket, and so on.

[0270] When adding new data to be processed, if the first feature bitmask is used to search the first-level hash bucket and data is already present, the second feature bitmask is used to search the second-level hash bucket. If data is also present in the second-level hash bucket, the third feature bitmask is used, and so on, until a hash bucket at level N is found to be empty. The data to be processed is then placed into that hash bucket. Furthermore, when deleting stored data, multiple feature bitmasks can be used simultaneously to calculate multiple hash values, find multiple hash buckets at corresponding levels, and search for the data to be deleted within those hash buckets.

[0271] This section discusses hash lookups using the multi-ply bicuts hash algorithm. When querying a bit-cut hash data structure, multiple feature bit masks can be used to calculate multiple hash values ​​for the query data. Then, the system searches for potentially matching data in the hash buckets corresponding to each feature bit mask.

[0272] For the hash variant of the multi-ply bicuts hash algorithm, the multi-ply bicuts hash algorithm is compatible with the bitcuts cuckoo hash. That is, a cuckoo shift operation can be performed on each level of the multi-level hash buckets of this hash to form a new variant hash, the multi-ply bicuts cuckoo hash.

[0273] The multi-ply bicuts hash algorithm is compatible with the bitcuts cuckoo filter. The bitcuts cuckoo filter expands the capacity of a single hash bucket from one piece of data to multiple pieces of data, while the multi-ply bicuts hash algorithm increases the number of layers in the overall bucket. In this way, the capacity of each hash bucket in each layer of the multi-ply bicuts hash algorithm can be changed to multiple (such as two), forming a new variant hash, the multi-ply bicuts cuckoo filter.

[0274] In summary, perfect hash algorithms may include, but are not limited to, bitcuts hash (BH), transitional hash algorithms may include, but are not limited to, bitcuts cuckoo hash (BCH), bitcuts cuckoo filter (BCF), multi-ply bitcuts hash (MPBH), multi-ply bitcuts cuckoo hash (MPBCH), and multi-ply bitcuts cuckoo filter (MPBCF), and conventional hash algorithms may include CRC32 hash (C32H).

[0275] Based on this, see Figure 3A The diagram illustrates hash switching. It shows the ability to switch between the BH, MPBH, and C32H algorithms. See also... Figure 3B The diagram illustrates hash switching. It shows the ability to switch between the BH, MPBH, MPBCH, and C32H algorithms. See also... Figure 3CThe diagram illustrates hash switching. It shows the ability to switch between the BH, MPBH, MPBCH, MPBCF, and C32H algorithms. See also... Figure 3D The diagram illustrates hash switching. It shows the ability to switch between the BH, BCH, MPBCH, and C32H algorithms. See also... Figure 3E The diagram illustrates hash switching. It shows the ability to switch between the BH, BCH, MPBCH, MPBCF, and C32H algorithms. See also... Figure 3F The diagram illustrates hash switching. It shows the ability to switch between the BH, BCH, BCF, MPBCF, and C32H algorithms. These are just examples and are not limitations. The perfect hash algorithm can be BH, the standard hash algorithm can be C32H, and the transitional hash algorithm can be any combination of BCH, BCF, MPBH, MPBCH, and MPBCF algorithms.

[0276] As can be seen from the above technical solutions, this application proposes a perfect hash (i.e., bit-cut hash) that balances query performance, update performance, and memory usage. It implements hashing functionality through a bit-cut hash data structure, thereby solving the problem of precise data matching in scenarios with high requirements for query, update, and memory. It achieves query performance at the nanosecond level, update performance at the microsecond level, and controllable memory usage, effectively improving the precise data matching performance and providing nanosecond-level query performance, microsecond-level average update performance, and controllable memory usage. For example, in scenarios with millions to tens of millions of routes, it can significantly improve the query performance of IPv6 addresses. The bit-cut hash algorithm is a precise data matching algorithm, suitable for any scenario requiring precise data matching. With the same amount of data to be processed, the throughput of the bit-cut hash algorithm is significantly higher than other hash algorithms, and as the amount of data to be processed increases, the throughput of the bit-cut hash algorithm remains at a high level without significant performance degradation. The update performance of the bit-cut hash algorithm is superior to other hash algorithms, with an average update performance between 1 and 20 microseconds, meeting the requirements of production environments. When the amount of data to be processed is small, the memory usage of the bit-cut hash algorithm is comparable to that of other hash algorithms. However, as the amount of data to be processed increases, the memory usage of the bit-cut hash algorithm is significantly less than that of other hash algorithms. Furthermore, as the principle of this bit-cut hash algorithm shows, its memory consumption mainly comes from the number of feature bits P, while the number of hash buckets is 2 to the power of P; therefore, the memory usage is controllable.

[0277] Based on the same application concept as the above method, this application proposes a data storage device, see [link to relevant documentation]. Figure 4 The diagram shown is a structural schematic of the data storage device, which may include:

[0278] The determination module 41 is used to determine the target number of bits P for newly added data to be stored in the bit-splitting hash data structure; the bit-splitting hash data structure includes multiple hash buckets, and the target number of bits P is used to ensure that there is no hash collision between the data to be stored and the data already stored in the bit-splitting hash data structure; based on the stored data and the data to be stored, the bit separable set (BSS) value of each bit is determined, and P feature bits are determined based on the BSS value of each bit;

[0279] Processing module 42 is used to determine the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure based on the values ​​corresponding to the P feature bits in the data to be stored;

[0280] Storage module 43 is used to store the data to be stored in the target hash bucket.

[0281] In one example, when determining the target number of bits P, the determining module 41 is specifically used for:

[0282] Based on the stored data and the data to be stored, determine the BSS value of each bit, and sort all bits in descending order of BSS value;

[0283] Feature bits are selected based on the sorting results. Hash collision determination is performed based on the selected feature bits. If there is no hash collision, the target number of bits P is determined based on the selected feature bits.

[0284] In one example, the determining module 41 selects feature bits based on the sorting result, performs hash collision determination based on the selected feature bits, and determines the target number of bits P based on the selected feature bits when no hash collision occurs. Specifically, this is used for:

[0285] The configured starting value is determined as the current value, and the first few bits of the current value are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision.

[0286] If so, update the sum between the current value and the fixed value to the current value, and return to perform the operation of selecting the first few bits of the current value as the feature bits;

[0287] If not, then the current value is determined as the target number of bits P.

[0288] In one example, the determining module 41 selects feature bits based on the sorting result, performs hash collision determination based on the selected feature bits, and determines the target number of bits P based on the selected feature bits when no hash collision occurs. Specifically, this is used for:

[0289] The configured starting value is determined as the current value, and the first few bits of the current value are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision.

[0290] If so, update the current value to the first type of value; if twice the current value is less than the configured fixed value, update twice the current value to the current value; otherwise, update the fixed value to the current value and return to perform the operation of selecting the first few bits of the current value as the feature bits.

[0291] If not, update the current value to the second type of value, and determine whether the difference between the second type of value and the first type of value is 1; if it is 1, determine the second type of value as the target number of bits P; if it is not 1, determine a reference value based on half of the sum of the first type of value and the second type of value, select the first few bits of the reference value as feature bits, and determine whether there is a hash collision based on the feature bits; if there is, update the reference value to the first type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1; if not, update the reference value to the second type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1.

[0292] In one example, the determining module 41 selects feature bits based on the sorting result, performs hash collision determination based on the selected feature bits, and determines the target number of bits P based on the selected feature bits when no hash collision occurs. Specifically, this is used for:

[0293] Select the top-ranked configured target value bits as feature bits, where the target value is greater than 1, and determine whether a hash collision exists based on the feature bits.

[0294] If not, the target value is determined as the target number of bits P.

[0295] In one example, after determining whether a hash collision exists based on the characteristic bits, if a hash collision exists, the determining module 41 specifically determines the target number of bits P by:

[0296] Update the target value to the first type of value, determine the current value based on the target value, and select the top-ranked bits of the current value as feature bits;

[0297] Determine whether a hash collision exists based on the aforementioned characteristic bits;

[0298] If so, update the current value to the first type of value; if twice the current value is less than the configured fixed value, update twice the current value to the current value; otherwise, update the fixed value to the current value and return to perform the operation of selecting the first few bits of the current value as the feature bits.

[0299] If not, update the current value to the second type of value, and determine whether the difference between the second type of value and the first type of value is 1; if it is 1, determine the second type of value as the target number of bits P; if it is not 1, determine a reference value based on half of the sum of the first type of value and the second type of value, select the first few bits of the reference value as feature bits, and determine whether there is a hash collision based on the feature bits; if there is, update the reference value to the first type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1; if there is no collision, update the reference value to the second type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1.

[0300] In one example, after determining whether a hash collision exists based on the characteristic bits, if a hash collision exists, the determining module 41 specifically determines the target number of bits P by:

[0301] The target value is updated to the first type of value, and the top-ranked configured performance boundary value bits are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision.

[0302] If no hash collision occurs, the performance boundary value is updated to the second type of value, and it is determined whether the difference between the second type of value and the first type of value is 1. If it is 1, the second type of value is determined as the target number of bits P. If it is not 1, a reference value is determined based on half of the sum of the first type of value and the second type of value. The first few bits of the reference value are selected as feature bits, and it is determined whether a hash collision occurs based on the feature bits. If a hash collision occurs, the reference value is updated to the first type of value, and the operation of determining whether the difference between the second type of value and the first type of value is 1 is returned. If no hash collision occurs, the reference value is updated to the second type of value, and the operation of determining whether the difference between the second type of value and the first type of value is 1 is returned.

[0303] If a hash collision occurs, the top-ranked configured memory threshold bits are selected as feature bits. Based on these feature bits, it is determined whether a hash collision exists. The memory threshold is greater than the performance boundary value. If not, the memory threshold is determined as the target number of bits P.

[0304] In one example, the determining module 41 selects feature bits based on the sorting result, performs hash collision determination based on the selected feature bits, and determines the target number of bits P based on the selected feature bits when no hash collision occurs. Specifically, this is used for:

[0305] Based on the target total number of the data to be stored and all stored data in the bit-splitting hash data structure, determine the numerical range of the target total number and the candidate number corresponding to the numerical range; select the top candidate number of bits as feature bits; if it is determined that there is no hash collision based on the feature bits, then determine the target number of bits P as the candidate number.

[0306] In one example, when the processing module 42 determines the target hash bucket corresponding to the data to be stored in the bit-cut hash data structure based on the values ​​corresponding to the P feature bits in the data to be stored, it is specifically used to: determine the candidate hash bucket corresponding to the data to be stored in the bit-cut hash data structure based on the values ​​corresponding to the P feature bits in the data to be stored.

[0307] If the candidate hash bucket does not have a hash collision with the hash bucket corresponding to the stored data, then the candidate hash bucket is determined as the target hash bucket corresponding to the data to be stored.

[0308] The processing module 42, based on the values ​​corresponding to the P feature bits in the data to be stored, determines the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure. Specifically, if the candidate hash bucket has a hash collision with the hash bucket corresponding to the stored data, and the number of target bits P is a configured upper limit, then a first hash value and a second hash value are determined based on the values ​​corresponding to the P feature bits in the data to be stored; if the first hash value corresponds to an idle first hash bucket in the bit-splitting hash data structure, then the first hash bucket is determined as the target hash bucket corresponding to the data to be stored; if the first hash bucket is occupied, and the stored data in the first hash bucket has not been migrated... If the first hash bucket is occupied, and the stored data in the first hash bucket has been migrated, and the second hash value is in the second hash bucket corresponding to the bit-cut hash data structure, then the second hash bucket is determined as the target hash bucket corresponding to the data to be stored; if the first hash bucket is occupied, and the stored data in the first hash bucket has been migrated, and the second hash bucket is occupied, and the stored data in the second hash bucket has not been migrated, then the stored data in the second hash bucket is migrated, and the second hash bucket is determined as the target hash bucket corresponding to the data to be stored.

[0309] The processing module 42, based on the values ​​corresponding to the P feature bits in the data to be stored, determines the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure. Specifically, if the candidate hash bucket has a hash collision with the hash bucket corresponding to the stored data, and the number of target bits P is the configured upper limit, then the hash value is determined based on the values ​​corresponding to the P feature bits in the data to be stored; if the hash value has a stored data in the hash bucket corresponding to the bit-splitting hash data structure, then the hash bucket is determined as the target hash bucket corresponding to the data to be stored.

[0310] The processing module 42 is further configured to, if a hash collision occurs between the candidate hash bucket and the hash bucket corresponding to the stored data, and the target bit count P is a configured upper limit, create a next-layer hash bucket set of the current layer hash bucket set, each hash bucket set including multiple hash buckets; wherein the feature bits corresponding to the next-layer hash bucket set are different from the feature bits corresponding to the current layer hash bucket set; wherein, based on the feature bits corresponding to the current layer hash bucket set, an address hash bucket is determined within the current layer hash bucket set, and the address hash bucket is used to store the location of the next-layer hash bucket set; based on the value of the feature bits corresponding to the next-layer hash bucket set in the data to be stored, the hash bucket corresponding to the data to be stored in the next-layer hash bucket set is determined; and this hash bucket is determined as the target hash bucket corresponding to the data to be stored.

[0311] In one example, when the determining module 41 determines P feature bits based on the BSS value of each bit, it specifically performs the following steps: sorting all bits in descending order of BSS values, and selecting P feature bits based on the sorting result; wherein, each bit is traversed sequentially based on the sorting result as the current bit; if the current bit is a redundant bit, it is prohibited from selecting the current bit as a feature bit; if the current bit is not a redundant bit, it is allowed to select the current bit as a feature bit.

[0312] The current bit is considered a redundant bit if the following conditions are met:

[0313] The number and position of the current bit corresponding to 1 in all the data to be processed are equal to the number and position of the first bit corresponding to 1 in all the data to be processed; or, the number and position of the current bit corresponding to 1 in all the data to be processed are equal to the number and position of the first bit corresponding to 0 in all the data to be processed; or, the number and position of the current bit corresponding to 0 in all the data to be processed are equal to the number and position of the first bit corresponding to 1 in all the data to be processed.

[0314] In one example, if the second and third bits are determined to be either Type I or Type II redundancy based on the previous W pieces of data to be processed, the determining module 41 updates the redundant bits of the W+1 pieces of data to be processed each time a piece of data to be processed is added, in the following manner:

[0315] For the first type of redundancy, if the second and third bits of the data to be processed are 00 or 11, then the second and third bits are determined to be the first type of redundancy; if the second and third bits of the data to be processed are 01 or 10, then the second and third bits are determined to be non-redundant.

[0316] For the second type of redundancy, if the second and third bits of the data to be processed are 00 or 11, then the second and third bits are determined to be non-redundant; if the second and third bits of the data to be processed are 01 or 10, then the second and third bits are determined to be the second type of redundancy.

[0317] Specifically, if the second bit and the third bit both contain 01 and 10, then the second bit and the third bit are of the second type of redundancy; if the second bit and the third bit both contain 00 and 11, then the second bit and the third bit are of the first type of redundancy.

[0318] If the redundancy of the fourth and fifth bits is determined based on the previous W data to be processed, and the BSS value of the fourth bit is 0 and the BSS value of the fifth bit is 0, then when adding a new data to be processed, the determining module 41 updates the redundancy bits of the W+1 subnet routes in the following manner: If the fourth and fifth bits of the previous W data to be processed are 00, and the fourth and fifth bits of the current data to be processed are 00, then the fourth and fifth bits are determined to be of the first type of redundancy; if the fourth and fifth bits of the previous W data to be processed are 00, and the fourth and fifth bits of the current data to be processed are 11, then the fourth and fifth bits are determined to be of the first type of redundancy.

[0319] If the fourth and fifth bits of the preceding W data to be processed are 0 or 1, and the fourth and fifth bits of this data to be processed are 0 or 1, then the fourth and fifth bits are determined to be type 2 redundancy; if the fourth and fifth bits of the preceding W data to be processed are 0 or 1, and the fourth and fifth bits of this data to be processed are 10, then the fourth and fifth bits are determined to be type 2 redundancy.

[0320] If the fourth and fifth bits of the preceding W data to be processed are 10, and the fourth and fifth bits of this data to be processed are also 10, then the fourth and fifth bits are determined to be type II redundancy; if the fourth and fifth bits of the preceding W data to be processed are 10, and the fourth and fifth bits of this data to be processed are 01, then the fourth and fifth bits are determined to be type II redundancy.

[0321] If the fourth and fifth bits of the preceding W data to be processed are 11, and the fourth and fifth bits of this data to be processed are also 11, then the fourth and fifth bits are determined to be of type I redundancy; if the fourth and fifth bits of the preceding W data to be processed are 11, and the fourth and fifth bits of this data to be processed are 00, then the fourth and fifth bits are determined to be of type I redundancy.

[0322] In one example, the determining module 41 is further configured to, when deleting stored data from the bit-cut hash data structure, determine the BSS value of each bit based on the remaining stored data in the bit-cut hash data structure, sort all bits in descending order of BSS values, and select P feature bits based on the sorting result; determine the hash bucket corresponding to the remaining stored data in the bit-cut hash data structure based on the values ​​of the P feature bits in the remaining stored data; the storage module 43 is further configured to store the remaining stored data in the hash bucket;

[0323] The determining module 41 is further configured to sequentially traverse each bit as the current bit based on the sorting result; if the current bit is a redundant bit, then it is prohibited to select the current bit as the feature bit; if the current bit is not a redundant bit and the BSS value is not 0, then it is allowed to select the current bit as the feature bit.

[0324] Specifically, after deleting stored data from the bit-splitting hash data structure, if the BSS value of the current bit is not 0, and the current bit is redundant with another bit that has been used as a feature bit, then the current bit is a redundant bit; or, if the BSS value of the current bit is not 0, and the current bit is not redundant with another bit that has been used as a feature bit, then if the deleted data is the last data that causes the current bit to not be redundant with the other bit, then the current bit is a redundant bit; if the deleted data is not the last data that causes the current bit to not be redundant with the other bit, then the current bit is not a redundant bit.

[0325] Based on the same application concept as the above method, this application proposes a network device, see [link to application]. Figure 5 As shown, the network device includes a processor 51 and a machine-readable storage medium 52, wherein the machine-readable storage medium 52 stores machine-executable instructions that can be executed by the processor 51; the processor 51 is used to execute the machine-executable instructions to implement the data storage method disclosed in the above example of this application.

[0326] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the data storage method disclosed in the above examples of this application.

[0327] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0328] Based on the same concept as the methods described above, this application also provides a computer program product, which may include a computer program. When executed by a processor, the computer program implements the data storage method disclosed in the examples above.

[0329] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0330] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data storage method, characterized in that, The method includes: For newly added data to be stored in the bit-cut hash data structure, a target number of bits P is determined; the bit-cut hash data structure includes multiple hash buckets, and the target number of bits P is used to ensure that there is no hash collision between the data to be stored and the data already stored in the bit-cut hash data structure; Based on the stored data and the data to be stored, determine the Bit Separable Set (BSS) value for each bit, and determine P feature bits based on the BSS value for each bit. Based on the values ​​of the P feature bits in the data to be stored, determine the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure; The data to be stored is stored in the target hash bucket.

2. The method according to claim 1, characterized in that, Determining the target number of bits P includes: Based on the stored data and the data to be stored, determine the BSS value of each bit, and sort all bits in descending order of BSS value; Feature bits are selected based on the sorting results. Hash collision determination is performed based on the selected feature bits. If there is no hash collision, the target number of bits P is determined based on the selected feature bits.

3. The method according to claim 2, characterized in that, The process of selecting feature bits based on the sorting result, determining hash collisions based on the selected feature bits, and determining the target number of bits P based on the selected feature bits when no hash collisions are found includes: The configured starting value is determined as the current value, and the first few bits of the current value are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision. If so, update the sum between the current value and the fixed value to the current value, and return to perform the operation of selecting the first few bits of the current value as the feature bits; If not, then the current value is determined as the target number of bits P.

4. The method according to claim 2, characterized in that, The process of selecting feature bits based on the sorting result, determining hash collisions based on the selected feature bits, and determining the target number of bits P based on the selected feature bits when no hash collisions are found includes: The configured starting value is determined as the current value, and the first few bits of the current value are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision. If so, update the current value to the first type of value; if twice the current value is less than the configured fixed value, update twice the current value to the current value; otherwise, update the fixed value to the current value and return to perform the operation of selecting the first few bits of the current value as the feature bits. If not, update the current value to the second type of value, and determine whether the difference between the second type of value and the first type of value is 1; if it is 1, determine the second type of value as the target number of bits P; if it is not 1, determine a reference value based on half of the sum of the first type of value and the second type of value, select the first few bits of the reference value as feature bits, and determine whether there is a hash collision based on the feature bits; if there is, update the reference value to the first type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1; if not, update the reference value to the second type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1.

5. The method according to claim 2, characterized in that, The process of selecting feature bits based on the sorting result, determining hash collisions based on the selected feature bits, and determining the target number of bits P based on the selected feature bits when no hash collisions are found includes: Select the top-ranked configured target value bits as feature bits, where the target value is greater than 1, and determine whether a hash collision exists based on the feature bits. If not, the target value is determined as the target number of bits P.

6. The method according to claim 5, characterized in that, After determining whether a hash collision exists based on the characteristic bits, determining the target number of bits P further includes: If so, update the target value to the first type of value, determine the current value based on the target value, and select the top-ranked bits of the current value as feature bits; Determine whether a hash collision exists based on the aforementioned characteristic bits; If so, update the current value to the first type of value; if twice the current value is less than the configured fixed value, update twice the current value to the current value; otherwise, update the fixed value to the current value and return to perform the operation of selecting the first few bits of the current value as the feature bits. If not, update the current value to the second type of value, and determine whether the difference between the second type of value and the first type of value is 1; if it is 1, determine the second type of value as the target number of bits P; if it is not 1, determine a reference value based on half of the sum of the first type of value and the second type of value, select the first few bits of the reference value as feature bits, and determine whether there is a hash collision based on the feature bits; if there is, update the reference value to the first type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1; if there is no collision, update the reference value to the second type of value, and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1.

7. The method according to claim 5, characterized in that, After determining whether a hash collision exists based on the characteristic bits, determining the target number of bits P further includes: If so, the target value is updated to the first type of value, and the top-ranked configured performance boundary value bits are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision. If no hash collision occurs, the performance boundary value is updated to the second type of value, and it is determined whether the difference between the second type of value and the first type of value is 1. If it is 1, the second type of value is determined as the target number of bits P. If it is not 1, a reference value is determined based on half of the sum of the first type of value and the second type of value. The first few bits of the reference value are selected as feature bits, and it is determined whether a hash collision occurs based on the feature bits. If a hash collision occurs, the reference value is updated to the first type of value, and the operation of determining whether the difference between the second type of value and the first type of value is 1 is returned. If no hash collision occurs, the reference value is updated to the second type of value, and the operation of determining whether the difference between the second type of value and the first type of value is 1 is returned. If a hash collision occurs, the top-ranked configured memory threshold bits are selected as feature bits. Based on these feature bits, it is determined whether a hash collision exists. The memory threshold is greater than the performance boundary value. If not, the memory threshold is determined as the target number of bits P.

8. The method according to claim 2, characterized in that, The process of selecting feature bits based on the sorting result, determining hash collisions based on the selected feature bits, and determining the target number of bits P based on the selected feature bits when no hash collisions are found includes: Based on the target total number of the data to be stored and all stored data in the bit-splitting hash data structure, determine the numerical range of the target total number and the candidate number corresponding to the numerical range; select the top candidate number of bits as feature bits; if it is determined that there is no hash collision based on the feature bits, then determine the target number of bits P as the candidate number.

9. The method according to any one of claims 1-8, characterized in that, Based on the values ​​corresponding to the P feature bits in the data to be stored, the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure is determined, including: Based on the values ​​of the P feature bits in the data to be stored, determine the candidate hash buckets corresponding to the data to be stored in the bit-splitting hash data structure. If the candidate hash bucket does not have a hash collision with the hash bucket corresponding to the stored data, then the candidate hash bucket is determined as the target hash bucket corresponding to the data to be stored.

10. The method according to claim 9, characterized in that, If the candidate hash bucket has a hash collision with the hash bucket corresponding to the stored data, the method further includes: If the target number of bits P is the configured upper limit value, the first hash value and the second hash value are determined based on the values ​​corresponding to the P feature bits in the data to be stored. If the first hash value is free in the first hash bucket corresponding to the bit split hash data structure, then the first hash bucket is determined as the target hash bucket corresponding to the data to be stored. If the first hash bucket is occupied and the stored data in the first hash bucket has not been migrated, then the stored data in the first hash bucket is migrated, and the first hash bucket is determined as the target hash bucket corresponding to the data to be stored. If the first hash bucket is occupied, and the stored data in the first hash bucket has been migrated, and the second hash value is free in the second hash bucket corresponding to the bit split hash data structure, then the second hash bucket is determined as the target hash bucket corresponding to the data to be stored. If the first hash bucket is occupied and the stored data in the first hash bucket has been migrated, and the second hash bucket is occupied and the stored data in the second hash bucket has not been migrated, then the stored data in the second hash bucket is migrated, and the second hash bucket is determined as the target hash bucket corresponding to the data to be stored.

11. The method according to claim 9, characterized in that, If the candidate hash bucket has a hash collision with the hash bucket corresponding to the stored data, the method further includes: If the number of target bits P is the configured upper limit, then the hash value is determined based on the values ​​of the P feature bits in the data to be stored. If the hash value has a stored data in the corresponding hash bucket in the bit-cut hash data structure, then that hash bucket is determined as the target hash bucket corresponding to the data to be stored.

12. The method according to claim 9, characterized in that, If the candidate hash bucket has a hash collision with the hash bucket corresponding to the stored data, the method further includes: If the target number of bits P is the configured upper limit, then a next-level hash bucket set is created for the current layer hash bucket set, and each hash bucket set includes multiple hash buckets; wherein the feature bits corresponding to the next-level hash bucket set are different from the feature bits corresponding to the current layer hash bucket set; wherein, the address hash buckets in the current layer hash bucket set are determined based on the feature bits corresponding to the current layer hash bucket set, and the address hash buckets are used to store the positions of the next-level hash bucket set; Based on the value of the feature bit corresponding to the next-level hash bucket set in the data to be stored, determine the hash bucket corresponding to the data to be stored in the next-level hash bucket set; The hash bucket is determined as the target hash bucket corresponding to the data to be stored.

13. The method according to any one of claims 1-8, characterized in that, The determination of P characteristic bits based on the BSS value of each bit includes: Sort all bits in descending order of BSS value, and select P feature bits based on the sorting result. Each bit is traversed sequentially based on the sorting result as the current bit. If the current bit is a redundant bit, it is prohibited from being selected as a feature bit. If the current bit is not a redundant bit, it is allowed to be selected as a feature bit. The current bit is considered a redundant bit if the following conditions are met: The number and position of the current bit corresponding to 1 in all the data to be processed are equal to the number and position of the first bit corresponding to 1 in all the data to be processed; or, the number and position of the current bit corresponding to 1 in all the data to be processed are equal to the number and position of the first bit corresponding to 0 in all the data to be processed; or, the number and position of the current bit corresponding to 0 in all the data to be processed are equal to the number and position of the first bit corresponding to 1 in all the data to be processed.

14. The method according to claim 13, characterized in that, If, based on the previous W data to be processed, it is determined that the second and third bits are either type I or type II redundancy, then when adding a new data to be processed, the redundant bits of the W+1 data to be processed are updated in the following manner: For the first type of redundancy, if the second and third bits of the data to be processed are 00 or 11, then the second and third bits are determined to be the first type of redundancy; if the second and third bits of the data to be processed are 01 or 10, then the second and third bits are determined to be non-redundant. For the second type of redundancy, if the second and third bits of the data to be processed are 00 or 11, then the second and third bits are determined to be non-redundant; if the second and third bits of the data to be processed are 01 or 10, then the second and third bits are determined to be the second type of redundancy. Specifically, if the second bit and the third bit both contain 01 and 10, then the second bit and the third bit are of the second type of redundancy; if the second bit and the third bit both contain 00 and 11, then the second bit and the third bit are of the first type of redundancy.

15. The method according to claim 13, characterized in that, If the redundancy of the fourth and fifth bits is determined based on the previous W data to be processed, and the BSS value of the fourth bit is 0 and the BSS value of the fifth bit is 0, the redundant bits of the W+1 subnet routes are updated in the following way each time a data to be processed is added. If the fourth and fifth bits of the preceding W data to be processed are 00, and the fourth and fifth bits of this data to be processed are also 00, then the fourth and fifth bits are determined to be a type I redundancy; if the fourth and fifth bits of the preceding W data to be processed are 00, and the fourth and fifth bits of this data to be processed are 11, then the fourth and fifth bits are determined to be a type I redundancy. If the fourth and fifth bits of the preceding W data to be processed are 0 or 1, and the fourth and fifth bits of this data to be processed are 0 or 1, then the fourth and fifth bits are determined to be type 2 redundancy; if the fourth and fifth bits of the preceding W data to be processed are 0 or 1, and the fourth and fifth bits of this data to be processed are 10, then the fourth and fifth bits are determined to be type 2 redundancy. If the fourth and fifth bits of the preceding W data to be processed are 10, and the fourth and fifth bits of this data to be processed are also 10, then the fourth and fifth bits are determined to be type II redundancy; if the fourth and fifth bits of the preceding W data to be processed are 10, and the fourth and fifth bits of this data to be processed are 01, then the fourth and fifth bits are determined to be type II redundancy. If the fourth and fifth bits of the preceding W data to be processed are 11, and the fourth and fifth bits of this data to be processed are also 11, then the fourth and fifth bits are determined to be of type I redundancy; if the fourth and fifth bits of the preceding W data to be processed are 11, and the fourth and fifth bits of this data to be processed are 00, then the fourth and fifth bits are determined to be of type I redundancy.

16. The method according to any one of claims 1-8, characterized in that, The method further includes: when deleting stored data from the bit-splitting hash data structure, determining the BSS value of each bit based on the remaining stored data in the bit-splitting hash data structure, sorting all bits in descending order of BSS values, and selecting P feature bits based on the sorting result; determining the hash bucket corresponding to the remaining stored data in the bit-splitting hash data structure based on the values ​​of the P feature bits in the remaining stored data, and storing the remaining stored data in the hash bucket; Specifically, each bit is traversed sequentially based on the sorting result as the current bit; if the current bit is a redundant bit, it is prohibited to select the current bit as the feature bit; if the current bit is not a redundant bit and the BSS value is not 0, it is allowed to select the current bit as the feature bit. In this context, after deleting stored data from the bit-splitting hash data structure, if the current bit is redundant with another bit that has been used as a feature bit, then the current bit is a redundant bit. Alternatively, if there is no redundancy between the current bit and another bit that has been used as a feature bit, then the current bit is a redundant bit when the deleted data is the last data that causes the current bit to have no redundancy with the other bit; and the current bit is not a redundant bit when the deleted data is not the last data that causes the current bit to have no redundancy with the other bit.

17. A data storage device, characterized in that, The device includes: A determination module is used to determine the target number of bits P for newly added data to be stored in a bit-splitting hash data structure; the bit-splitting hash data structure includes multiple hash buckets, and the target number of bits P is used to ensure that there is no hash collision between the data to be stored and the data already stored in the bit-splitting hash data structure; based on the stored data and the data to be stored, the bit separability set (BSS) value of each bit is determined, and P feature bits are determined based on the BSS value of each bit; The processing module is used to determine the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure based on the values ​​of the P feature bits in the data to be stored. A storage module is used to store the data to be stored in the target hash bucket.

18. The apparatus according to claim 17, characterized in that, When determining the target number of bits P, the determining module is specifically used for: Based on the stored data and the data to be stored, determine the BSS value of each bit, and sort all bits in descending order of BSS value; The configured starting value is determined as the current value, and the first few bits of the current value are selected as feature bits. Based on the feature bits, it is determined whether there is a hash collision. If so, update the current value to the first type of value; if twice the current value is less than the configured fixed value, update twice the current value to the current value; otherwise, update the fixed value to the current value and return to perform the operation of selecting the first few bits of the current value as the feature bits. If not, update the current value to the second type of value and determine whether the difference between the second type of value and the first type of value is 1; if it is 1, determine the second type of value as the target number of bits P; if it is not 1, determine a reference value based on half of the sum of the first type of value and the second type of value, select the first reference value bits as feature bits, and determine whether there is a hash collision based on the feature bits. If it exists, update the reference value to the first type of value and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1; if it does not exist, update the reference value to the second type of value and return to execute the operation of determining whether the difference between the second type of value and the first type of value is 1.

19. The apparatus according to claim 17, characterized in that, The processing module, based on the values ​​corresponding to the P feature bits in the data to be stored, determines the target hash bucket corresponding to the data to be stored in the bit-splitting hash data structure, specifically for the following purposes: Based on the values ​​of the P feature bits in the data to be stored, determine the candidate hash buckets corresponding to the data to be stored in the bit-splitting hash data structure. If the candidate hash bucket does not have a hash collision with the hash bucket corresponding to the stored data, then the candidate hash bucket is determined as the target hash bucket corresponding to the data to be stored. If the candidate hash bucket has a hash collision with the hash bucket corresponding to the stored data, and the number of target bits P is the configured upper limit, then the first hash value and the second hash value are determined based on the values ​​of the P feature bits in the data to be stored; if the first hash value is free in the first hash bucket corresponding to the bit-splitting hash data structure, then the first hash bucket is determined as the target hash bucket corresponding to the data to be stored. If the first hash bucket is occupied and the stored data in the first hash bucket has not been migrated, then the stored data in the first hash bucket is migrated, and the first hash bucket is determined as the target hash bucket corresponding to the data to be stored. If the first hash bucket is occupied, and the stored data in the first hash bucket has been migrated, and the second hash value is in the second hash bucket corresponding to the bit-cut hash data structure, then the second hash bucket is determined as the target hash bucket corresponding to the data to be stored; if the first hash bucket is occupied, and the stored data in the first hash bucket has been migrated, and the second hash bucket is occupied, and the stored data in the second hash bucket has not been migrated, then the stored data in the second hash bucket is migrated, and the second hash bucket is determined as the target hash bucket corresponding to the data to be stored.

20. The apparatus according to claim 17, characterized in that, When the determining module determines P feature bits based on the BSS value of each bit, it is specifically used for: Sort all bits in descending order of BSS value, and select P feature bits based on the sorting result. Each bit is traversed sequentially based on the sorting result as the current bit. If the current bit is a redundant bit, it is prohibited from being selected as a feature bit. If the current bit is not a redundant bit, it is allowed to be selected as a feature bit. The current bit is considered a redundant bit if the following conditions are met: The number and position of the current bit corresponding to 1 in all the data to be processed are equal to the number and position of the first bit corresponding to 1 in all the data to be processed; or, the number and position of the current bit corresponding to 1 in all the data to be processed are equal to the number and position of the first bit corresponding to 0 in all the data to be processed; or, the number and position of the current bit corresponding to 0 in all the data to be processed are equal to the number and position of the first bit corresponding to 1 in all the data to be processed.

21. A network device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-16.

Images