37 results about "Perfect hash function" patented technology

Generating and using a high-speed, scalable, and easily updateable data structure are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

A “Variable-Rate Perfect Hasher” maps sparse variable-rate data of one or more dimensions into a hash table using a perfect hash function. In various embodiments, perfect hash tables are populated by first computing offset table address for each data point of a domain of sparse variable-rate data elements. Offset vectors are then computed for each offset table address based in part on the size of each data element by evaluating offset vectors in order of a sum of the data point addresses mapping to each offset vector. These offset vectors are then stored in the offset table. For each data point, the corresponding offset vector is then used to compute a hash table address. Data elements are then perfectly hashed into the hash table using the computed hash table addresses. The resulting hash tables support efficient random access of the variable-sized data elements stored therein.

Techniques for managing data objects of a data storage system are described herein. According to one embodiment, a perfect hash function is generated for data objects stored in a data storage system. For each of the data objects, a hash operation is performed using the perfect hash function to indicate whether the respective data object is alive. Resources associated with the respective data object is reclaimed if it is determined that the respective data object is not alive based on a result of the hash operation using the perfect hash function, where the reclaimed resources are released back to the data storage system as free resources. Other methods and apparatuses are also described.

Embodiments are directed to perfect physical garbage collection (PPGC) process that uses a perfect hash vector instead of large Bloom filters of the regular physical garbage collection process for the live and live instance vectors and consolidates both into a single live vector using the perfect hash vector. A method of PPGC includes an analysis phase walking an index containing a mapping of fingerprints to a container ID for a plurality of containers and building a perfect hash function for a walk vector and a live vector, wherein the live vector uses a perfect hash vector, an enumeration phase inserting live segments in memory into the perfect hash vector, a select phase traversing the plurality of containers and selecting containers that meet a defined liveness threshold and a copy phase copying live segments out of the selected containers.

Generating and using a high-speed, scalable and easily updateable data structures are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

Embodiments for a memory efficient perfect hashing for large records. A container ID set is divided into multiple fixed range sizes. These ranges are then mapped into perfect hash buckets until each bucket is filled to uniformly distribute the container IDs across different perfect hash buckets so that the number of CIDs in every perfect hash bucket is the same or nearly the same. Individual perfect hash functions are created for each perfect hash bucket. With container IDs as keys, the process maps n keys to n positions to reduce any extra memory overhead. The perfect hash function is implemented using a compress, hash, displace (CHD) algorithm using two levels of hash functions. The level 1 hash functions divides the keys into multiple internal buckets with a defined average number of keys per bucket. The CHD algorithm iteratively tries different level 2 hash variables to achieve collision-free mapping.

A representation used in a computer system to represent a set of data items that correspond to and are accessible by means of a set of keys. The representation includes an array of the data items and a bit string associated with the array. Each key is mapped onto a bit of the bit string by means of a hash function that is perfect for the set of keys. The mapped bit is set. The data item corresponding to the key has a position in the array that corresponds to the position of the bit for the key in the bit string. Methods for reading and writing the representation are disclosed as well as a technique based on the mod operation and a set of co-prime numbers for finding a perfect hash function for a given set of keys.

Generating and using a high-speed, scalable, and easily updateable data structure are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

Techniques for managing data objects of a data storage system are described herein. According to one embodiment, a perfect hash function is generated for data objects stored in a data storage system. For each of the data objects, a hash operation is performed using the perfect hash function to indicate whether the respective data object is alive. Resources associated with the respective data object is reclaimed if it is determined that the respective data object is not alive based on a result of the hash operation using the perfect hash function, where the reclaimed resources are released back to the data storage system as free resources. Other methods and apparatuses are also described.

Generating and using a high-speed, scalable, and easily updateable data structure are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

A high-speed, space-efficient, scalable and easily updateable data boundary hash-based structure is generated and used. The proposed boundary hash-based data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

A high-speed, space-efficient, scalable and easily updateable data boundary hash-based structure is generated and used. The proposed boundary hash-based data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

A secure perfect hash function that has properties similar to those of cryptographic hash functions without compromising features of a perfect hash function such as speed and collision-free outputs is provided. A cryptographic hash function is utilized to process the set S and the. output is divided into three sub-outputs of required length. Each output can now be treated as a separate hash function (g(x), f1(x), f2(x)), S is split into r buckets Bi, 0≦i<r, using the hash function g. Buckets Bi are permuted in a pseudorandom fashion. For each bucket Bi, a displacement pair (d0, d1) is chosen randomly from the sequence {(0,0), (0,1), . . . (0, m−1), (1,,0),(1,1), . . . , (1, m−1), . . . ,(m−1, m−1)} such that each element of Bi is placed in an empty bin given by (f1(x)+d0f2(x)+d1) mod m. The index of this displacement is stored in the sequence.

A recursively partitioned static router-table, the data structure incorporating a first-level partition including subtries and an auxiliary trie. A node of the subtrie includes a path Q(N) from the root R of a trie T to a root N of the subtrie, a stride s for a next-level partition, a mask that characterizes a next-level perfect hash function, and a pointer to the hash table for the next-level partition. At least one of the trie T, the first-level partition, the auxiliary trie, and the next-level partition is represented by a base structure selected from the group consisting of MBT and HSST.

In general, this disclosure describes techniques of storing data in and retrieving data from a cache of a computing device. More specifically, techniques are described for utilizing a “perfect hash” function to implement an associative cache within a computing device. That is, the associative cache implements a fully associative map between a predetermined set of addresses and data values, employing only a single tag fetch comparison.

Embodiments for a memory efficient perfect hashing for large records. A container ID set is divided into multiple fixed range sizes. These ranges are then mapped into perfect hash buckets until each bucket is filled to uniformly distribute the container IDs across different perfect hash buckets so that the number of CIDs in every perfect hash bucket is the same or nearly the same. Individual perfect hash functions are created for each perfect hash bucket. With container IDs as keys, the process maps n keys to n positions to reduce any extra memory overhead. The perfect hash function is implemented using a compress, hash, displace (CHD) algorithm using two levels of hash functions. The level 1 hash functions divides the keys into multiple internal buckets with a defined average number of keys per bucket. The CHD algorithm iteratively tries different level 2 hash variables to achieve collision-free mapping.

The embodiment of the invention provides a log parsing method and device based on perfect hashing. The method includes the steps of obtaining a to-be-parsed log, conducting information extraction on the to-be-parsed log according to a preset rule, and obtaining to-be-parsed keywords and corresponding values of the to-be-parsed log; according to a perfect hash function created in advance, calculating a first hash value corresponding to each to-be-parsed keyword; according to each first hash value, obtaining corresponding to-be-parsed keyword types, saving the corresponding value of each to-be-parsed keyword into a corresponding structure of the corresponding to-be-parsed keyword type for parsing to obtain structural data. The device is used for executing the method. According to the embodiment, by adopting a hash locating method for identifying the to-be-parsed keywords, there is no need to match characters of the to-be-parsed keywords one by one, the keywords can be obtained fast through hash locating, and therefore the efficiency of parsing the log is improved.

Generating and using a high-speed, scalable, and easily updateable data structure are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

Generating and using a high-speed, scalable and easily updateable data structures are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

Embodiments are directed to perfect physical garbage collection (PPGC) process that uses a NUMA-aware perfect hash vector. The process splits a perfect hash vector (PHVEC) into a number of perfect hash vectors, wherein the number corresponds to a number of nodes having a processing core and associated local memory, directs each perfect hash to a respective local memory of a node so that each perfect hash vector accesses only a local memory, and assigns fingerprints in the perfect hash vector to a respective node using a mask function. The process also performs a simultaneous creation of perfect hash vectors in a multi-threaded manner by scanning the Index once.

A representation of a new rule, defined as a set of a new transition(s), is inserted into a perfect hash table which includes previously placed transitions to generate an updated perfect hash table. This may be done by, for each new transition: (a) hashing the new transition; and (b) if there is no conflict, inserting the hashed new transition into the table. If, however, the hashed new transition conflicts with any of the previously placed transitions, either (A) any transitions of the state associated with the conflicting transition are removed from the table, the hashed new transition is placed into the table, and the removed transitions are re-placed into the table, or (B) any previously placed transitions of the state associated with the new transition are removed, and the transitions of the state associated with the new transition are re-placed into the table.

The invention relates to a perfect function for processing large scale dictionary, comprising smoothing the word in the dictionary and dividing the smoothed word dictionary into n sub dictionaries with relevant structural charts, with n being the natural number, sequencing the points of these relevant charts and mapping each word to various address to get the perfect function of the dictionary. It can include millions of words' dictionary to form perfect Hash function to process large character collection like Chinese, with reduced time and working space.

A system and method that allows two parties to find common records in their data sets without having to actually share the data sets with each other or a third party. Two primitives, perfect hash functions and public key cryptography, are combined in a unique way to obtain a secure and efficient private matching solution. Since the data that is exchanged is always encrypted during the match process, neither party reveals its data to the other party. This solution enables two parties to match sensitive data such as PII (Personally Identifiable Information) or PHI (Protected Health Information) without having to disclose the data to other or to any third party. One or both of the parties only learn of matching records without learning any information about the records that do not match.

The embodiment of the invention discloses a gateway message forwarding method and device, a storage medium and a gateway, and belongs to the technical field of communication. The method comprises thefollowing steps: receiving a CAN message sent by a first node; acquiring identification information from the CAN message; performing hash operation on the identification information by utilizing a minimum perfect hash function to obtain a minimum perfect hash value; searching a second node in a routing table according to the minimum perfect hash value; and forwarding the CAN message to the secondnode. According to the embodiment of the invention, the minimum perfect hash value is calculated for the identification information through the minimum perfect hash function, and the minimum perfect hash values calculated according to different identification information are different, so that hash collision is avoided, CAN messages of all nodes can be forwarded in real time, and the time delay ofmessage forwarding is reduced.

A system and method for compressing regular data streams using instruction cache-resident perfect hash functions. The system employs machine learning to analyze regular data streams and generate optimized perfect hash functions that remain resident in CPU instruction cache during compression operations. Unlike traditional compression methods that store lookup tables in data cache, this approach transforms compression operations into executable instructions, enabling efficient processing while leaving data cache available for application data. The system comprises continuous monitoring of compression performance and data stream characteristics, automatically regenerating hash functions when significant changes are detected. The method encompasses analyzing data streams, generating hash function characteristics through machine learning, transforming these characteristics into optimized instructions, and deploying them in instruction cache for runtime compression operations. This approach improves compression performance for regular data streams by leveraging CPU cache hierarchy more efficiently than traditional lookup table-based methods.

A recursively partitioned static router-table, the data structure incorporating a first-level partition including subtries and an auxiliary trie. A node of the subtrie includes a path Q(N) from the root R of a trie T to a root N of the subtrie, a stride s for a next-level partition, a mask that characterizes a next-level perfect hash function, and a pointer to the hash table for the next-level partition. At least one of the trie T, the first-level partition, the auxiliary trie, and the next-level partition is represented by a base structure selected from the group consisting of MBT and HSST.

The embodiment of the application discloses a gateway message forwarding method, device, storage medium and gateway, which belong to the technical field of communication. The method includes: receiving a CAN message sent by a first node; obtaining identification information from the CAN message; performing a hash operation on the identification information using a minimum perfect hash function to obtain a minimum perfect hash value; according to The minimum perfect hash value is searched for a second node in a routing table; and the CAN message is forwarded to the second node. In the embodiment of the present application, the minimum perfect hash value is calculated for the identification information through the minimum perfect hash function. Since the minimum perfect hash values ​​calculated according to different identification information are different, there will be no hash collision. It can also ensure that the CAN messages of all nodes can be forwarded in real time, reducing the delay of message forwarding.