35 results about "Perfect hash function" patented technology

Generating and using a high-speed, scalable, and easily updateable data structure are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

A “Variable-Rate Perfect Hasher” maps sparse variable-rate data of one or more dimensions into a hash table using a perfect hash function. In various embodiments, perfect hash tables are populated by first computing offset table address for each data point of a domain of sparse variable-rate data elements. Offset vectors are then computed for each offset table address based in part on the size of each data element by evaluating offset vectors in order of a sum of the data point addresses mapping to each offset vector. These offset vectors are then stored in the offset table. For each data point, the corresponding offset vector is then used to compute a hash table address. Data elements are then perfectly hashed into the hash table using the computed hash table addresses. The resulting hash tables support efficient random access of the variable-sized data elements stored therein.

Techniques for managing data objects of a data storage system are described herein. According to one embodiment, a perfect hash function is generated for data objects stored in a data storage system. For each of the data objects, a hash operation is performed using the perfect hash function to indicate whether the respective data object is alive. Resources associated with the respective data object is reclaimed if it is determined that the respective data object is not alive based on a result of the hash operation using the perfect hash function, where the reclaimed resources are released back to the data storage system as free resources. Other methods and apparatuses are also described.

Embodiments are directed to perfect physical garbage collection (PPGC) process that uses a perfect hash vector instead of large Bloom filters of the regular physical garbage collection process for the live and live instance vectors and consolidates both into a single live vector using the perfect hash vector. A method of PPGC includes an analysis phase walking an index containing a mapping of fingerprints to a container ID for a plurality of containers and building a perfect hash function for a walk vector and a live vector, wherein the live vector uses a perfect hash vector, an enumeration phase inserting live segments in memory into the perfect hash vector, a select phase traversing the plurality of containers and selecting containers that meet a defined liveness threshold and a copy phase copying live segments out of the selected containers.

Embodiments for a memory efficient perfect hashing for large records. A container ID set is divided into multiple fixed range sizes. These ranges are then mapped into perfect hash buckets until each bucket is filled to uniformly distribute the container IDs across different perfect hash buckets so that the number of CIDs in every perfect hash bucket is the same or nearly the same. Individual perfect hash functions are created for each perfect hash bucket. With container IDs as keys, the process maps n keys to n positions to reduce any extra memory overhead. The perfect hash function is implemented using a compress, hash, displace (CHD) algorithm using two levels of hash functions. The level 1 hash functions divides the keys into multiple internal buckets with a defined average number of keys per bucket. The CHD algorithm iteratively tries different level 2 hash variables to achieve collision-free mapping.

A representation used in a computer system to represent a set of data items that correspond to and are accessible by means of a set of keys. The representation includes an array of the data items and a bit string associated with the array. Each key is mapped onto a bit of the bit string by means of a hash function that is perfect for the set of keys. The mapped bit is set. The data item corresponding to the key has a position in the array that corresponds to the position of the bit for the key in the bit string. Methods for reading and writing the representation are disclosed as well as a technique based on the mod operation and a set of co-prime numbers for finding a perfect hash function for a given set of keys.

Generating and using a high-speed, scalable, and easily updateable data structure are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

A secure perfect hash function that has properties similar to those of cryptographic hash functions without compromising features of a perfect hash function such as speed and collision-free outputs is provided. A cryptographic hash function is utilized to process the set S and the. output is divided into three sub-outputs of required length. Each output can now be treated as a separate hash function (g(x), f1(x), f2(x)), S is split into r buckets Bi, 0≦i<r, using the hash function g. Buckets Bi are permuted in a pseudorandom fashion. For each bucket Bi, a displacement pair (d0, d1) is chosen randomly from the sequence {(0,0), (0,1), . . . (0, m−1), (1,,0),(1,1), . . . , (1, m−1), . . . ,(m−1, m−1)} such that each element of Bi is placed in an empty bin given by (f1(x)+d0f2(x)+d1) mod m. The index of this displacement is stored in the sequence.

A recursively partitioned static router-table, the data structure incorporating a first-level partition including subtries and an auxiliary trie. A node of the subtrie includes a path Q(N) from the root R of a trie T to a root N of the subtrie, a stride s for a next-level partition, a mask that characterizes a next-level perfect hash function, and a pointer to the hash table for the next-level partition. At least one of the trie T, the first-level partition, the auxiliary trie, and the next-level partition is represented by a base structure selected from the group consisting of MBT and HSST.

In general, this disclosure describes techniques of storing data in and retrieving data from a cache of a computing device. More specifically, techniques are described for utilizing a “perfect hash” function to implement an associative cache within a computing device. That is, the associative cache implements a fully associative map between a predetermined set of addresses and data values, employing only a single tag fetch comparison.

The embodiment of the invention provides a log parsing method and device based on perfect hashing. The method includes the steps of obtaining a to-be-parsed log, conducting information extraction on the to-be-parsed log according to a preset rule, and obtaining to-be-parsed keywords and corresponding values of the to-be-parsed log; according to a perfect hash function created in advance, calculating a first hash value corresponding to each to-be-parsed keyword; according to each first hash value, obtaining corresponding to-be-parsed keyword types, saving the corresponding value of each to-be-parsed keyword into a corresponding structure of the corresponding to-be-parsed keyword type for parsing to obtain structural data. The device is used for executing the method. According to the embodiment, by adopting a hash locating method for identifying the to-be-parsed keywords, there is no need to match characters of the to-be-parsed keywords one by one, the keywords can be obtained fast through hash locating, and therefore the efficiency of parsing the log is improved.

The invention relates to a perfect function for processing large scale dictionary, comprising smoothing the word in the dictionary and dividing the smoothed word dictionary into n sub dictionaries with relevant structural charts, with n being the natural number, sequencing the points of these relevant charts and mapping each word to various address to get the perfect function of the dictionary. It can include millions of words' dictionary to form perfect Hash function to process large character collection like Chinese, with reduced time and working space.

Method for searching an item using a parametric hash filter includes forming an input vector from input data stream; forming a hash matrix having a first portion and a second portion; multiplying the hash matrix with the input vector to generate a second input vector including a hash values of the first input vector; generating a perfect hash vector and a universal hash vector, by applying a smooth periodic function to the second input vector; mapping onto a Markov random field the coordinates of locations of hash values in a search domain for which there is no possibility of collisions in the perfect hash vector to form an energy function; minimizing the energy function to generate a compressed hash table; fitting a band of acceptable locations in the compressed hash table, based on a predetermined false positive rate; and searching for a new item in the band of acceptable locations.