Preprocessing circuit, hash storage device and hash storage method

Abstract

Provided are a preprocessing circuit, a hash storage device and a hash storage method. An input end of the preprocessing circuit is connected to an output end of a hash calculation module; the preprocessing circuit comprises multiple stages of multiplexers connected in cascade; the multiplexer of each stage comprises a first input end, a second input end and an output end; the first input end of the multiplexer of each stage is separately connected to output ends of preset N first bits corresponding to the hash calculation module; the first input end of a first-stage multiplexer is connected to output ends of preset N second bits corresponding to the hash calculation module; the first bit is a bit used for determining a block number; the N second bits are N first bits having the lowest bit order among a plurality of preset first bits; the output end of the first-stage multiplexer outputs target hash values of the N second bits; the second input end of the multiplexer of the previous stage is connected to the output end of the multiplexer of the next stage; and a signal having a preset size is input to the second input end of the multiplexer of the last stage.

Description

Preprocessing circuit, hash storage device and hash storage method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on January 15, 2024, with application number 202410057421.1 and invention name “Preprocessing circuit, hash storage device and hash storage method”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The embodiments of the present disclosure relate to the technical field of programmable chip design, and in particular to a preprocessing circuit, a hash storage device, and a hash storage method. Background Art

[0004] A network processor (NP) is a microprocessor that provides the logic for sending and receiving data (including voice and video) across a communications network. Traditional network chips have fixed functions. Upper-layer network applications using traditional network chips must be designed based on the fixed functions of the chip, and new network functions cannot be added at will.

[0005] A programmable network chip refers to a chip that can flexibly add new network protocols or new network functions to the network chip without redesigning the hardware structure.

[0006] Programmable network chips primarily consist of two components: a protocol parser and a pipeline of processing modules. The parser extracts various data from network packets, while the pipeline performs operations based on this data and updates the chip's memory. This data and the corresponding operations are stored in the chip's memory as tables. The pipeline performs table lookups based on the data and retrieves the corresponding operations; this is known as the match-action mechanism.

[0007] Hardware table lookups are generally divided into two types: ternary lookups and hash lookups. Ternary lookups obtain addresses by looking up data, using six times the memory resources of regular memory. One row of data corresponds to one row of addresses, eliminating conflicts and fallbacks. Hash lookups use the hash value of the data as the address, consuming only logic resources. However, hash lookups are subject to hash collisions, meaning that different data may have the same hash value. Therefore, hash algorithms have a high hit rate requirement. Due to area constraints, programmable network chips typically use a small number of ternary lookups and a large number of hash lookups.

[0008] As business needs constantly evolve, table formats and sizes vary significantly. In programmable processing, due to the limited storage resources of programmable network chips, high-hit-rate table lookups are required to achieve higher resource utilization. Therefore, reducing the hash address rollback rate is a high-value and urgent issue that needs to be addressed. Summary of the Invention

[0009] The embodiments of the present disclosure provide a preprocessing circuit, a hash storage device, and a hash storage method, so as to reduce the rollback rate of hash storage and improve the utilization rate of hash storage.

[0010] In a first aspect, an embodiment of the present disclosure provides a preprocessing circuit, wherein the input end of the preprocessing circuit is connected to the output end of a hash calculation module, and the preprocessing circuit includes: a cascaded multi-stage multiplexer; each stage of the multiplexer includes a first input end, a second input end, and an output end; the first input end of each stage of the multiplexer is respectively connected to the output end of the preset N first bits corresponding to the hash calculation module; wherein the first input end of the first-stage multiplexer is connected to the output end of the preset N second bits corresponding to the hash calculation module; the first bit is a bit used to determine a block number; the block number is used to determine the table depth of a hash table; the N second bits are the N first bits with the lowest bit order among the multiple preset first bits; the output end of the first-stage multiplexer outputs the target hash value of the N second bits; the second input end of the previous stage multiplexer is connected to the output end of the next stage multiplexer; the second input end of the last stage multiplexer inputs a signal of a preset size.

[0011] In a second aspect, an embodiment of the present disclosure provides a hash storage device, comprising a hash calculation module, a preprocessing circuit as described in the first aspect, a hash distribution module, and a storage module, wherein the output end of the hash calculation module is connected to the input end of the preprocessing circuit, the input end of the hash distribution module is connected to the output end of the preprocessing circuit, and the output end of the hash distribution module is connected to the input end of the storage module; wherein a hash matrix is ​​provided in the hash calculation module for calculating the hash value of input data and outputting a hash value of a preset number of bits; the preprocessing circuit receives the hash value of the preset number of bits output by the hash calculation module, and preprocesses the hash value of the preset N second bits in the hash value of the preset number of bits for determining the block number, to obtain the N the preprocessing circuit transparently transmits the hash values ​​of the other bits in the preset number of bits except the N second bits to the hash distribution module, and transmits the target hash value obtained after the hash values ​​of the N second bits are preprocessed by the preprocessing circuit to the hash distribution module; the hash distribution module receives the hash value of the preset number of bits at the output end of the preprocessing circuit, and determines a block number according to the hash values ​​of multiple first bits including the preprocessed N second bits, and determines multiple entries according to the hash values ​​of multiple bits used to determine the entries; and determines a target address for storing the data according to the block number and the multiple entries, and stores the data in the storage module according to the target address.

[0012] In a third aspect, an embodiment of the present disclosure provides a hash storage method, including a hash storage device for use in the second aspect, the method comprising: a hash calculation module, in response to detecting receipt of input data to be stored, performing hash value calculation on the input data, and outputting a hash value of a preset number of bits to a preprocessing circuit; the preprocessing circuit, in response to receiving the hash value of the preset number of bits, preprocesses a plurality of preset first bits in the preset number of bits for determining a block number, and obtains target hash values ​​corresponding to N second bits, where the second bits belong to the first bit; sending the target hash value to a hash distribution module, and dividing the N second bits by the hash value of the preset number of bits; The hash values ​​of other bits except the second bit are transparently transmitted to the hash distribution module; in response to receiving the hash value of a preset number of bits output by the preprocessing circuit; the hash distribution module determines, from the preset number of bits, multiple groups of first bits for determining block numbers and multiple groups of third bits for determining entries; for each hash storage, the hash distribution module determines, according to a group of first bits corresponding to the hash storage, a target block number corresponding to the input data, and determines, according to a corresponding group of third bits, a target entry corresponding to the input data; according to the determined target block number and target entry of each hash storage, the hash distribution module stores the input data into the storage module according to the hash storage rule.

[0013] The preprocessing circuit, hash storage device and hash storage method provided by the present disclosure preprocess the hash value of N second bits used to determine the block number by using a multi-stage multiplexer including a cascade, and the target hash value corresponding to the N second bits obtained can achieve (2 n )k. Compared with the related art, which can only achieve (2 n )k table depth, the table depth range of hash storage is expanded, which is conducive to realizing the flexibility of hash paths, can reduce the fallback of hash storage, and improve the utilization rate of hash storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0015] FIG1 is a schematic structural diagram of a preprocessing circuit provided by an embodiment of the present disclosure;

[0016] FIG2 is a schematic diagram of the structure of a hash storage device provided by an embodiment of the present disclosure;

[0017] FIG3 is a flow chart of a hash storage method in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0019] Hash storage takes the keyword (English: Key) as the independent variable. Through a certain function mapping relationship (such as a hash function), the corresponding function value (hash address) is calculated. This function value is used as the address of the data element, and the data element is stored in the storage unit corresponding to the address. When searching, the hash address is calculated using the same function according to the keyword to be searched, and then the data element to be found is directly retrieved from the storage unit corresponding to the hash address. When storing data, the keyword is mapped to an address through a hash function. The advantage of this is that when searching for this data, the address can be directly calculated through the hash function, and the data can be directly found. However, due to the many-to-one nature of the hash function, it is possible for multiple data to map to the same address (the probability of conflict depends on the selection of the hash function, and an appropriate hash function can reduce the probability of conflict).

[0020] Cuckoo Hash is a relatively effective way to handle hash conflicts. Cuckoo Hash can guarantee a worst-case query complexity of O(1) and has extremely high space utilization. Among them, O(f(n)) is the general expression of complexity. O(f(n)) means that the complexity is linearly related to f(n), that is, there exists a constant C such that O(f(n)) < C|f(n)|. O(1) refers to constant-order complexity, that is, the complexity is not affected by the length of the hash table. In fact, this is also the characteristic of the hash function. Regardless of the depth of the table, the address can be located through only one operation. Cuckoo Hash uses multiple hash functions to handle conflicts: each input data corresponds to multiple positions. When a corresponding position is empty, any position is selected for insertion; when all positions are not empty, the data in one position is randomly kicked out for insertion, and the kicked-out data then executes the same algorithm until all data is inserted, or when the number of times of kicking out data reaches the specified threshold (usually 4 or 8), it ends. When the number of times of kicking out data reaches the threshold, the kicked-out data has a conflict back-off, and the proportion of the remaining stored data is the hash hit rate.

[0021] In a hardware storage scheme for implementing Cuckoo Hash storage in related technologies, a hash value (such as a 52-bit hash value) is first uniformly calculated. Then, 6 hash values are extracted bit by bit from this hash value to form a 6-way Cuckoo Hash storage. Since the hash values are obtained bit by bit, the depth of each table in the 6-way Cuckoo Hash storage is a power function of 2. Here, the depth of the table refers to the length of the hash table, and the depth of the table can also be called the table length. At this time, the decomposition of the storage table with a pre-given size is extremely inflexible. For example, a 63k table can only represent 6-way Cuckoo Hash storage with the following table depths: 32k, 16k, 4k, 2k, 1k. In addition, since the depths of each table vary greatly, and the high utilization rate of Cuckoo Hash is obtained under the condition that the depths of each table are the same. Therefore, when the depths of each table vary greatly, the high utilization rate of Cuckoo Hash storage cannot be achieved.

[0022] Furthermore, the table depth of each cuckoo hash storage path in the above scheme is a power-of-two function, which limits the choice of table depth for each path. For tables of a given size (such as a 63KB storage table), arbitrary-way storage is not possible. For example, a 63KB table must have 6 paths, making 4-way storage difficult. This scheme results in significant variations in the table depth of each cuckoo hash path, with some paths having smaller table depths, which increases hash conflicts.

[0023] In order to solve the above problems, the present invention proposes a hash bit preprocessing circuit that can express the bit on the hardware as only 2 n Hash value, adjusted to express 2 n and 2 n ×p kinds of values, so that (2 n )k, which makes the selection of table depth more flexible, and is conducive to reducing the difference of table depth of multi-way cuckoo storage, thereby helping to reduce the hash conflict rate, reduce the fallback of hash storage, and thus improve the high utilization of cuckoo hash storage.

[0024] Please refer to Figure 1, which shows a schematic diagram of the structure of a preprocessing circuit. The input end of the preprocessing circuit is connected to the output end of a hash calculation module. The hash calculation module can perform a hash calculation on the input data and output a hash value of a preset number of bits.

[0025] Each level of the multiplexer includes a first input terminal, a second input terminal, and an output terminal; the first input terminal of each level of the multiplexer is respectively connected to the output terminal of the preset N first bits corresponding to the hash calculation module; wherein the first input terminal of the first-level multiplexer is connected to the output terminal of the preset N second bits corresponding to the hash calculation module; the first bit is a bit used to determine the block number; the N second bits are the N first bits with the lowest bit order among the multiple preset first bits; the output terminal of the first-level multiplexer outputs the target hash value of the N second bits;

[0026] The second input terminal of the previous stage multiplexer is connected to the output terminal of the next stage multiplexer. For example, in FIG1 , the second input terminal of multiplexer 101 is connected to the output terminal of multiplexer 102, and the second input terminal of multiplexer 102 is connected to the output terminal of multiplexer 103.

[0027] The second input terminal of the last-stage multiplexer inputs a signal of a preset magnitude, for example, the second input terminal of the multiplexer 103 in FIG1 inputs a preset signal 00.

[0028] In this embodiment, the number of stages of the cascaded multiplexers may be three or more.

[0029] The hash calculation module can output an m-bit hash value. The m-bit hash value, starting from the 0th hash bit to the j-1th hash bit, can be used to calculate the entry bits. The entry is the storage location on this path when the hash is stored. The jth to m-1th hash bits are used as the first bit to determine the block number. The block number is used to instruct the hardware which blocks to assign to the hash of this path. Here, m is an integer greater than 2. j is an integer greater than 1 and less than m. The combination of block number and entry can achieve a specified hash table depth. The hash table depth can be expressed as {block number, entry value}.

[0030] The size of m is related to the hash matrix set in the hash calculation module. The hash matrix in the hash calculation module can be obtained from a hash polynomial. The first input terminal of each multiplexer is connected to the corresponding output terminal of the hash matrix for outputting the preset N first bit hash values. The above N is an integer greater than or equal to 2 and less than (mj).

[0031] The first-stage multiplexer is connected to output ends of N bits with the lowest bit order among the plurality of preset first bits.

[0032] In some embodiments, the hash calculation module outputs a 52-bit hash value.

[0033] Furthermore, the first input end of the first-stage multiplexer is connected to the output end of N consecutive bits from the 40th to the 51st bits of the 52 bits of the hash calculation module.

[0034] In some application scenarios, N is equal to 2, and the above-mentioned first-stage multiplexer is connected to the output end of the above-mentioned hash calculation module for outputting the hash values ​​of the jth and j+1th bits in m bits, where the jth and j+1th bits are two second bits.

[0035] Take a 4-way hash table with m = 52 as an example. Bits 0-39 of bits 0-51 can be divided into four parts: 0-9, 10-19, 20-29, and 30-39, each corresponding to a set of entry bits. Bits 40-51 are the first bits, and appropriate bits can be selected from these first bits to represent the block number. From the appropriate bits selected to represent the block number, the two lowest-order bits are selected as the second bits. As you can see, one hash table corresponds to one way, and when looking up the table, each way is searched in parallel based on the corresponding hash value.

[0036] For example, bits 40 and 41 are selected as the second bit. The first input of the first-stage multiplexer is connected to the output of the hash calculation module for outputting the bit values ​​of bits 40 and 41. The second input is connected to the output of the second-stage multiplexer. The output of the first-stage multiplexer outputs the preprocessed target hash value of bits 40 and 41. The signal of the preset size input to the second input of the last-stage multiplexer can be 00.

[0037] In some embodiments, the cascaded multiple stages of multiplexers include three stages of cascaded multiplexers.

[0038] Each level of the multiplexer includes a selection signal input terminal (also called a control signal input terminal), which can input a selection signal (also called a control signal). The multiplexer outputs different input signals under the control of different selection signals.

[0039] Take the above-mentioned multiplexer as a two-to-one multiplexer as an example. The selection signal of the multiplexer has two values: 0 and 1. When the selection signal is 1, the signal input from the first input terminal can be selected as the output. When the selection signal is 0, the signal input from the second input terminal can be selected as the output. For each level of the multiplexer, when the selection signal is 1, 2 of the N first bits can be selected. N For example, if we select the hash value whose sum of the first bit values ​​of N is 0 as the output, we can have 2 N -1 target hash value.

[0040] The following description takes N equal to 2 as an example.

[0041] As shown in FIG1 , a three-stage cascaded multiplexer is provided. The first input of the first-stage multiplexer 101 is connected to the output of the hash calculation module for outputting the hash values ​​of the 41st and 40th bits. The second input is connected to the output of the second-stage multiplexer 102. The first input of the second-stage multiplexer 102 is connected to the output of the hash calculation module for outputting the hash values ​​of the 51st and 50th bits; the second input is connected to the output of the third-stage multiplexer 103.

[0042] The first input terminal of the third-stage multiplexer 103 is connected to the output terminal of the hash calculation module for outputting the hash values ​​of the 48th and 47th bits; the second input terminal inputs 00.

[0043] When the signals outputted by the 41st and 40th bit positions are 00, 01, or 10, the selection signal of the first-stage multiplexer 101 is 1, thereby selecting the signals of the 41st and 40th bit positions as the outputs of the first-stage multiplexer 101 .

[0044] When the signal outputted by the 41st:40th bit positions is 11, the selection signal of the first-stage multiplexer 101 is 0, and the output of the second-stage multiplexer 102 is selected as the output of the first-stage multiplexer 101 .

[0045] Similarly, for the second-stage multiplexer 102, when the signals output by the 51st:50th bits are 00, 01, or 10, the selection signal of the second-stage multiplexer 102 is 1, thereby selecting the signals of the 51st:50th bits inputted at the first input terminal as the output of the second-stage multiplexer 102.

[0046] When the signals outputted by the 51st and 50th bit positions are 11, the selection signal of the second-stage multiplexer 102 is 0, and the output of the third-stage multiplexer 103 is selected as the output of the second-stage multiplexer 102 .

[0047] For the third-stage multiplexer 103 , when the signals output by the 48th:47th bits are 00, 01, or 10, the selection signal of the third-stage multiplexer 103 is 1, thereby selecting the signals of the 48th:47th bits inputted at the first input terminal as the output of the third-stage multiplexer 103 .

[0048] When the signal outputted by the 48th and 47th bits is 11, the selection signal of the third-stage multiplexer 103 is 0, and 00 inputted at the second input terminal is selected as the output of the third-stage multiplexer.

[0049] In this way, when the 41st and 40th bits are 11, the values ​​00, 01, and 10 of the 51st and 50th bits or the 48th and 47th bits are used as the output of the 41st and 40th bits. For the 41st and 40th bits, there are only three values ​​00, 01, and 10.

[0050] It is understandable that the multiplexer in the above pre-processing circuit can have different implementation schemes to select any three values ​​from 00, 01, 10, and 11.

[0051] It is understandable that the above-mentioned 51st:50th bit or 48th:47th bit may also be the value of any two adjacent bits used to represent the block number.

[0052] Here, the 51st:50th bit or the 48th:47th bit is used as the input of the second input end of the second-stage multiplexer and the third-stage multiplexer, so that the input of the two-stage multiplexer is more evenly distributed on the 42nd to 51st bits, which is conducive to achieving the balance of the output results of the 41st:40th bit.

[0053] Here, since the signal (hash value) of the lowest 2 bits used to represent the hash table passes through the above preprocessing circuit, the original 4 values ​​corresponding to the 2 bits are processed into 3 values, which is 1 / 4 less than the original 4 values ​​that can be represented by the 2 bits. The above 2 bits can be combined with the first bit of other bits used to determine the block number to achieve 2 n ×3 hash values, based on 2 n ×3 hash values ​​to determine the block number and entry can be achieved (2 n ×3)k, where n is an integer greater than or equal to 0. This allows for a variety of table depths. This makes the selection of table depths and the number of cuckoo hash storage paths more flexible, helping to reduce the differences in table depths of multi-way cuckoo hash storage, thereby reducing the hash collision rate and improving the utilization of cuckoo hash storage.

[0054] It should be understood that the first bits corresponding to the first input terminals of each multiplexer shown in Figure 1 are merely illustrative and should not be construed as limiting the bits corresponding to each multiplexer. These three groups of first bits may vary depending on the application scenario, such as in order or in the selected bits.

[0055] The output end of the hash calculation module can output a hash value of a preset number of bits, and the multiple preset number of bits may include multiple first bits. The multiple first bits are used to determine the block number of the multi-way hash table. The above-mentioned multiple first bits may include two preset first bits corresponding to the first-level multiplexer, two preset first bits corresponding to the second-level multiplexer, and two preset first bits corresponding to the third-level multiplexer. Among them, the two preset first bits corresponding to the first-level multiplexer can be the two first bits with the lowest order (regarded as two second bits) among the multiple preset first bits corresponding to the above-mentioned first-level multiplexer, the second-level multiplexer and the third-level multiplexer respectively.

[0056] The two second bits corresponding to the first-stage multiplexer may be two first bits arbitrarily selected from a plurality of first bits, and are not limited to the two bits 41:40 shown in FIG. 1 .

[0057] The two first bits corresponding to the second-stage multiplexer may be two consecutive first bits arbitrarily selected from a plurality of first bits having a higher order than the second bit, and are not limited to the two bits 51:50 shown in FIG1 .

[0058] The two first bits corresponding to the third-stage multiplexer may be any two first bits selected from a plurality of first bits having a higher order than the second bit, and are not limited to the two bits 48:47 shown in FIG1 .

[0059] The preprocessing circuit provided in this embodiment preprocesses the hash values ​​of the N second bits used to determine the block number by using a cascaded multi-stage multiplexer to obtain a selected plurality (for example, p) of target hash values, which can be implemented in conjunction with other bits (2 n ×p)k size table depth. Compared with the related art which can only achieve (2 n )k-sized table depth, which expands the range of hash storage table depth, facilitates the flexibility of hash paths, reduces hash storage rollback, and improves hash storage utilization. As you can understand, this example uses a block capacity of 1024 (1k), that is, a 10-bit entry value range. If the block capacity is other values, corresponding adjustments can be made. The overall concept is consistent and will not be repeated here.

[0060] Please refer to Figure 2, which shows the structure of a hash storage device. The hash storage device includes: a hash calculation module 201, a preprocessing circuit 202, a hash distribution module 203, and a storage module 204. The input of the hash calculation module 201 receives input data. The output of the hash calculation module 201 is connected to the input of the preprocessing circuit 202; the input of the hash distribution module 203 is connected to the output of the preprocessing circuit 202, and the output of the hash distribution module 203 is connected to the input of the storage module. The hash calculation module 201 is provided with a hash matrix for calculating the hash value of the input data. Specifically, a hash polynomial for calculating the hash value can be mapped to the hash matrix through a compilation algorithm. The hash calculation module receives input data. After receiving the input data, the hash calculation module calculates the input data with the hash matrix to obtain a hash value (hash-bits) of a preset number of bits for the input data. The hash value here is a binary signal. The preset number of bits is related to the hash matrix or hash polynomial. Using different hash polynomials or hash matrices can produce hash values ​​with different bit counts.

[0061] The preprocessing circuit 202 receives the hash value of the preset number of bits output by the hash calculation module 201, and preprocesses the hash value of the preset N second bits in the hash value of the preset number of bits for determining the block number to obtain a target hash value of the N second bits; the preprocessing circuit 202 transparently transmits the hash values ​​of the other bits in the preset number of bits except the N second bits to the hash distribution module 203, and transmits the target hash value obtained after the hash value of the N second bits is preprocessed by the preprocessing circuit 202 to the hash distribution module 203.

[0062] The hash distribution module 203 receives a hash value of a preset number of bits outputted by the preprocessing circuit 202, determines a block number based on the hash values ​​of a plurality of first bits including the preprocessed N second bits, and determines a plurality of entries based on the hash values ​​of a plurality of bits used to determine the entries; and

[0063] A target address for storing data is determined according to the block number and the plurality of entries, and the data is stored in the storage module according to the target address.

[0064] The output terminal of the hash calculation module 201 is connected to the input terminal of the preprocessing circuit 202. The hash calculation module can output a hash value of a preset number of bits, such as a 52-bit hash value. For example, different output terminals can be used to output the hash value of the preset number of bits in parallel.

[0065] Taking N equal to 2 as an example, after the preprocessing circuit 202 receives the hash value of the preset number of bits (e.g., 52 bits), it can preprocess the values ​​of some bits. The purpose of preprocessing the values ​​of some bits output by the hash calculation module is to adapt to the size of (2 n ×3)k surface depth.

[0066] The pre-processing circuit 202 receives the preset digit hash value output by the hash calculation module, and pre-processes the hash values ​​of the two second bits of the preset digit hash value for determining the block number to obtain the target hash values ​​of the two second bits. The target hash value has 2 n ×3 values, where n is an integer greater than or equal to 0; the preprocessing circuit transparently transmits the hash values ​​of the other bits in the preset number of bits except the two second bits to the hash distribution module, and transmits the target hash value obtained after preprocessing the hash values ​​of the two second bits by the preprocessing circuit to the hash distribution module.

[0067] The detailed description of the pre-processing circuit 202 can refer to the description of the embodiment shown in FIG1 , which will not be repeated here.

[0068] The hash distribution module 203 receives a hash value of a preset number of bits outputted from the preprocessing circuit, determines a block number based on the hash values ​​of a plurality of first bits including the two preprocessed second bits, and determines a plurality of entries based on the hash values ​​of a plurality of bits used to determine the entries; and

[0069] A target address for storing data is determined according to the block number and the plurality of entries, and the data is stored in the storage module according to the target address.

[0070] The input of the hash distribution module 203 receives the hash value of the preset number of bits output by the preprocessing circuit 202. The hash value output for the second bit of the preset number of bits is the target hash value obtained after preprocessing by the preprocessing circuit of the embodiment shown in FIG1 . The hash values ​​for the remaining bits remain the same as the hash values ​​output by the hash calculation module.

[0071] The hash allocation module can determine the number of hash paths and the corresponding table depth for a table of a preset size based on balance. Taking a 9k-sized table as an example, a relatively balanced allocation is 4-way hash storage, with table depths corresponding to each path being 3k, 2k, 2k, and 2k, respectively. Taking the 52-bit hash value output by the preprocessing circuit as an example, bits 0 to 39 are divided into four parts to determine the entry, with one part allocated to each path (10 bits = 1k, so 1k corresponds to 10 bits). The number of bits used to determine the block number for each table depth is allocated as 2 bits, 1 bit, 1 bit, and 1 bit. 3k needs to be represented by a designated second bit processed by the preprocessing circuit. Still using the preprocessing circuit shown in Figure 1 as an example, the bits used to determine the block number for each table depth are [41:40], [42:42], [45:45], and [48:48], respectively.

[0072] The table depth of the hash path in the related art is (2 n )k, since this scheme introduces (2 n ×p)k, so the selection of table depth is more flexible. When making a specific selection, a heuristic method can be used to seek the most balanced distribution, or a recursive selection can be used. Taking 63k table storage as an example, after recursive selection, a 6-way hash storage is determined, of which 5 ways have a table depth of 12k and 1 way has a table depth of 3k, that is, 63k=60k+3k=12k×5+3k. Compared with the 6-way hash storage in the related art, which corresponds to table depths of 32k, 16k, 8k, 4k, 2k, and 1k respectively, the table depths of the 6-way hash storage schemes of 5 ways with a table depth of 12k and 1 way with a table depth of 3k are relatively balanced. Under this relatively balanced table depth selection, the conflict rate is increased from 92% to 99%. Reducing the conflict rate of cuckoo hash storage and reducing hash storage fallback are conducive to improving the utilization rate of hash storage.

[0073] Taking a 63k table stored in a 4-way hash table as an example, the 4-way table depths correspond to 24k, 24k, 12k, and 3k, respectively. Taking a 52-bit hash value as an example, bits 0 to 9 from bits 0 to 39 of the 52 bits are selected as the first group of entry bits and assigned to the first hash storage path. Bits 10 to 19 are the second group of entry bits and assigned to the second hash storage path. Bits 20 to 29 are the third group of entry bits and assigned to the third hash storage path. Bits 30 to 39 are the fourth group of entry bits and assigned to the fourth hash storage path. Since the output of bits 40 to 41 is processed by the preprocessing circuit shown in Figure 1, these two bits can achieve three different values, which is 1 / 4 fewer than the four different values ​​that a 2-bit value can achieve. Therefore, the values ​​of the N bits, including the two bits processed by the preprocessing circuit, are also 1 / 4 fewer than the original number of values ​​of the N bits.

[0074] Bits 40-44 can take 24 values ​​to determine the block number of the first path, and together with the first group of entry bits, achieve a table depth of 24k. Bits 40-44 can take 24 values ​​to determine the block number of the second path, and together with the second group of entry bits, achieve a table depth of 24k. Although the block numbers of the first and second paths are the same, because the two paths correspond to different groups of entry bits, the entry values ​​obtained from the two groups of entry bits are also different, which can achieve corresponding different hash values. Bits 40-43 can take 12 values ​​to determine the block number of the third path, and together with the third group of entry bits, achieve a table depth of 12k. Bits 40-41 can take 3 values ​​to determine the block number of the fourth path, and together with the fourth group of entry bits, achieve a table depth of 3k.

[0075] After determining the number of hash ways and the table depth of each hash storage, the corresponding hash bits for each way are also determined. Using the 63k, 4-way storage example above, the first hash storage uses bits 0-9 of the 52 bits output by the preprocessing circuit to generate the entry value for the first hash storage, and bits 40-44 are used to calculate the block number.

[0076] When storing data, the hash allocation module can determine the block number based on the first bit including the two preset second bits after preprocessing, and determine multiple entries based on multiple bits used to determine the entries, determine the target address for storing the data based on the block number and the multiple entries, and store the data in the storage module based on the target address.

[0077] Specifically, for a piece of input data, a 52-bit hash value is generated after passing through the hash calculation module and preprocessing circuit. The hash value of bits 0 to 9 is used to calculate the entry value, and the hash value of bits 40 to 44 is used to calculate the block number. The target storage address is determined based on the block number and entry value. The input data is stored at this target storage address.

[0078] The aforementioned hash storage device, by utilizing the preprocessing circuit shown in Figure 1, expands the hash storage table depth from the original power-of-two function, achieving a diverse range of table depths. This allows for more flexible selection of the table depths of each hash storage channel and the number of cuckoo hash storage channels, helping to reduce variations in the table depths of multi-channel cuckoo hash storage, thereby reducing hash collision rates and ultimately improving the utilization of cuckoo hash storage.

[0079] Please refer to Figure 3, which shows a schematic flow chart of the hash storage method provided by the present disclosure. The hash storage method is applied to the hash storage device shown in Figure 2. As shown in Figure 3, the method includes the following steps:

[0080] S301: In response to detecting and receiving input data to be stored, the hash calculation module performs hash value calculation on the input data and outputs a hash value of a preset number of bits to the preprocessing circuit.

[0081] The hash calculation module stores a hash matrix. The hash calculation module receives input data. After receiving the input data, the hash calculation module calculates the input data against the hash matrix to obtain a hash value for a preset number of hash bits. This preset number of bits is related to the hash matrix or hash polynomial. Using different hash polynomials or hash matrices can produce hash values ​​with different number of bits.

[0082] S302: In response to receiving a hash value of a preset number of bits, the preprocessing circuit preprocesses multiple preset first bits in the preset number of bits for determining the block number to obtain a target hash value corresponding to N second bits, where the second bits belong to the first bits; sends the target hash value to the hash distribution module, and transparently transmits the hash values ​​of other bits in the hash value of the preset number of bits except the N second bits to the hash distribution module.

[0083] After the preprocessing circuit receives the hash value of the preset number of bits (e.g., 52 bits) output by the hash calculation module, it can preprocess the local hash bits of the hash value of the preset number of bits. The purpose of processing the local hash bits output by the hash calculation module is to adapt the size of (2 n ×p)k tables are used to implement flexible hash table depth. For details, please refer to the description of the embodiment shown in Figure 1.

[0084] S303: In response to receiving the hash value of the preset number of bits output by the preprocessing circuit, the hash distribution module determines multiple groups of first bits for determining block numbers and multiple groups of third bits for determining entries from the preset number of bits.

[0085] S304: For each hash storage, the hash allocation module determines a target block number corresponding to the input data according to a group of first bits corresponding to the hash storage, and determines a target entry corresponding to the input data according to a corresponding group of third bits.

[0086] S305: Based on the determined target block numbers and target entries of each hash storage, the hash allocation module stores the input data into the storage module according to the hash storage rule.

[0087] In some embodiments, the hash value of the preset number of bits is a 52-bit hash value, and the N second bits are N consecutive bits selected from bits 40 to 51 of the 52 bits.

[0088] In some embodiments, the above N is equal to 2.

[0089] Taking a 63k table stored in a 4-way hash table as an example, the 4-way table depths correspond to 24k, 24k, 12k, and 3k, respectively. Taking a 52-bit hash value as an example, bits 0 to 9 from bits 0 to 39 of the 52 bits are selected as the first group of entry bits and assigned to the first hash storage way. Bits 10 to 19 are the second group of entry bits and assigned to the second hash storage way. Bits 20 to 29 are the third group of entry bits and assigned to the third hash storage way. Bits 30 to 39 are the fourth group of entry bits and assigned to the fourth hash storage way. The values ​​of bits 40 to 44 determine the block number of the first hash storage way. The values ​​of bits 40 to 44 determine the block number of the second hash storage way. The values ​​of bits 40 to 43 determine the block number of the third hash storage way. The values ​​of bits 40 to 41 determine the block number of the fourth hash storage way.

[0090] After receiving the 52-bit hash value output by the preprocessing circuit, the entry number can be calculated using the binary values ​​of bits 0 to 9, and the block number can be calculated using the binary values ​​of bits 40 to 44, thereby determining the target address of the first hash storage. Similarly, the target address of each hash storage path is determined.

[0091] The above-mentioned hash storage rule can be a cuckoo hash storage rule. Starting from the first hash storage, it can be determined whether the target address of the path is free. If it is free, it is stored. If it is not free, the target addresses of other hash storage paths are checked to see if they are free. If one of the hash storage paths is free, the above-mentioned input data is stored. If none of the hash storage paths are free, a hash storage path is selected, the value in the target address corresponding to the hash storage path is removed, and the input data is stored in the target address of the hash storage path. The removed data is then recalculated according to the above-mentioned hash storage rule and stored in the target address of each hash storage path.

[0092] The hash storage method provided in this embodiment passes the input data through a preprocessing circuit, and determines the target address corresponding to the multi-way hash storage based on the hash value of a preset number of bits passed by the preprocessing circuit, and stores the input data according to the target address of the multi-way hash storage. Since the preprocessing circuit expands the table depth, the size difference of each table depth is reduced, the hash conflict rate is reduced, and the high utilization rate of the cuckoo hash storage is improved.

[0093] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0094] Although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0095] Although the subject matter has been described in language specific to structural features and methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A preprocessing circuit, wherein, The input end of the preprocessing circuit is connected to the output end of the hash calculation module. The preprocessing circuit includes: cascaded multi-stage multiplexers; Each stage of multiplexer includes a first input end, a second input end, and an output end; The first input ends of each stage of multiplexer are respectively connected to the output ends of the corresponding preset N first bit positions of the hash calculation module; among them, the first input end of the first-stage multiplexer is connected to the output ends of the corresponding preset N second bit positions of the hash calculation module; the first bit position is the bit position for determining the block number; the N second bit positions are the lowest N first bit positions among the multiple preset first bit positions; the output end of the first-stage multiplexer outputs the target hash value of the N second bit positions; The second input end of the previous-stage multiplexer is connected to the output end of the next-stage multiplexer; A signal of a preset size is input to the second input end of the last-stage multiplexer.

2. The preprocessing circuit according to claim 1, wherein, The multiplexer is a two-to-one selector.

3. The preprocessing circuit according to claim 1, wherein The hash calculation module includes a hash matrix compiled from a hash polynomial.

4. The preprocessing circuit according to claim 1, wherein, N is equal to 2.

5. The preprocessing circuit according to claim 1, wherein, The hash calculation module outputs a hash value of 52 bits.

6. The preprocessing circuit according to claim 5, wherein, The first input end of the first-stage multiplexer is connected to the output ends of consecutive N bit positions among the 40th to 51st bit positions of the 52-bit positions.

7. The preprocessing circuit according to any one of claims 1-6, wherein, The cascaded multi-stage multiplexers include cascaded three-stage multiplexers.

8. A hash storage device, wherein, It includes a hash calculation module, a preprocessing circuit as described in any one of claims 1-7, a hash distribution module, and a storage module. Among them, the output end of the hash calculation module is connected to the input end of the preprocessing circuit, the input end of the hash distribution module is connected to the output end of the preprocessing circuit, and the output end of the hash distribution module is connected to the input end of the storage module; among them, A hash matrix is set in the hash calculation module for calculating the hash value of the input data and outputting a hash value of a preset number of bits; The preprocessing circuit receives the hash value of the preset number of bits output by the hash calculation module, preprocesses the hash value of the preset N second bit positions used for determining the block number in the hash value of the preset number of bits to obtain the target hash value of the N second bit positions; the preprocessing circuit transparently transmits the hash values of other bit positions except the N second bit positions in the preset number of bits to the hash distribution module, and transmits the target hash value obtained by preprocessing the hash value of the N second bit positions through the preprocessing circuit to the hash distribution module; The hash distribution module receives the hash value of the preset number of bits at the output end of the preprocessing circuit, determines the block number according to the hash values of multiple first bit positions including the preprocessed N second bit positions, and determines multiple entries according to the hash values of multiple bit positions used for determining the entries; and Determines the target address for storing the data according to the block number and the multiple entries, and stores the data in the storage module according to the target address.

9. A hash storage method for a hash storage device as described in claim 8, the method includes: The hash calculation module calculates the hash value of the input data in response to detecting the receipt of the input data to be stored, and outputs a hash value with a preset number of bits to the preprocessing circuit; The preprocessing circuit preprocesses a plurality of preset first bit positions for determining the block number in the preset number of bits in response to receiving the hash value with the preset number of bits, and obtains a target hash value corresponding to N second bit positions, where the second bit positions belong to the first bit positions; Send the target hash value to the hash allocation module, and pass through the other bit hash values except the N second bit positions in the hash value with the preset number of bits to the hash allocation module; In response to receiving the hash value with the preset number of bits output by the preprocessing circuit; The hash allocation module respectively determines multiple groups of first bit positions for determining the block number and multiple groups of third bit positions for determining the entry from the preset number of bits; For each path of hash storage, the hash allocation module determines the target block number corresponding to the input data according to a group of first bit positions corresponding to the path of hash storage, and determines the target entry corresponding to the input data according to a corresponding group of third bit positions; According to the determined target block numbers and target entries of each path of hash storage, the hash allocation module stores the input data into the storage module according to the hash storage rule.

10. The method according to claim 9, wherein, The hash storage rule is the cuckoo hash storage rule.

11. The method according to claim 9, wherein, N is equal to 2.

12. The method according to claim 9, wherein, The hash value with the preset number of bits is a 52-bit bit hash value, and the N second bit positions are consecutive N bit positions selected from the 40th to 51st bit positions in the 52-bit bit hash value.

Images