821 results about "Bloom filter" patented technology

According to an embodiment of the invention, cache management comprises maintaining a cache comprising a hash table including rows of data items in the cache, wherein each row in the hash table is associated with a hash value representing a logical block address (LBA) of each data item in that row. Searching for a target data item in the cache includes calculating a hash value representing a LBA of the target data item, and using the hash value to index into a counting Bloom filter that indicates that the target data item is either not in the cache, indicating a cache miss, or that the target data item may be in the cache. If a cache miss is not indicated, using the hash value to select a row in the hash table, and indicating a cache miss if the target data item is not found in the selected row.

A method and system for generating a bloom filter by mapping into respective groups each of a plurality of initial keys according to a first hash function and mapping each group hashed key into a bloom filter using k respective hash functions.

The present invention relates to a method and apparatus based on Bloom filters for detecting predefined signatures (a string of bytes) in a network packet payload. A Bloom filter is a data structure for representing a set of strings in order to support membership queries. Hardware Bloom filters isolate all packets that potentially contain predefined signatures. Another independent process eliminates false positives produced by the Bloom filters. The system is implemented on a FPGA platform, resulting in a set of 10,000 strings being scanned in the network data at the line speed of 2.4 Gbps.

A system and method for detecting duplicate checks during processing. The duplicate detection may be performed by a financial institution, such as a bank. The method may be implemented on a computer based system. The duplicate detection method may be automated. The method may be applied to incoming check files prior to processing of the check data to prevent processing of duplicate checks. The system and method may use a function, such as a hash function, to perform the duplicate detection. Other functions, such as a Bloom filter which may use multiple hash functions, may be used to perform the duplicate detection.

The Computationally Efficient Transfer Processing and Auditing Apparatuses, Methods and Systems (“CETPA”) transforms transaction record inputs via CETPA components into matrix and list tuple outputs for computationally efficient auditing. A blockchain transaction data auditing apparatus comprises a blockchain recordation component, a matrix Conversion component, and a bloom filter component. The blockchain recordation component receives a plurality of transaction records for each of a plurality of transactions, each transaction record comprising a source address, a destination address, a transaction amount and a timestamp of a transaction; the source address comprising a source wallet address corresponding to a source digital wallet, and the destination address comprising a destination wallet address corresponding to a destination virtual currency wallet; verifies that the transaction amount is available in the source virtual currency wallet; and when the transaction amount is available, cryptographically records the transaction in a blockchain comprising a plurality of hashes of transaction records. The Bloom Filter component receives the source address and the destination address, hashes the source address using a Bloom Filter to generate a source wallet address, and hashes the destination address using the Bloom Filter to generate a destination wallet address. The Matrix Conversion component adds the source wallet address as a first row and a column entry to a stored distance matrix representing the plurality of transactions, adds the destination wallet address as a second row and column entry to the stored distance matrix representing the plurality of transactions, adds the transaction amount and the timestamp as an entry to the row corresponding to the source wallet address and the column corresponding to the destination wallet address; and generate a list representation of the matrix, where each entry in the list comprises a tuple having the source wallet address, the destination wallet address, the transaction amount and the timestamp.

Systems and methods are disclosed for detecting covert DNS tunnels using n-grams. The majority of legitimate DNS requests originate from network content itself, for example, through hyperlinks in websites. So, comparing data from incoming network communications to a hostname included in a DNS request can give an indication on whether the DNS request is a legitimate request or associated with a covert DNS tunnel. This process can be made computationally efficient by extracting n-grams from incoming network content and storing the n-grams in an efficient data structure, such as a Bloom filter. The stored n-grams are compared with n-grams extracted from outgoing DNS requests. If n-grams from an outgoing DNS request are not found in the data structure, the domain associated with the DNS request is determined to be associated with a suspected covert DNS tunnel.

Methods for performing exact search string matches in the data plane of a network processor. The methods employ a two-stage string search mechanism to identify the existence of a search string from a set S in a packet payload. A first pre-processing stage identifies a potential search string match and a second analysis stage determines whether the first stage match corresponds to an exact string match. The first stage is implemented using hash values derived from at least one of search strings in set S or sub-strings of those search strings. In one embodiment, a plurality of Bloom filters are used to perform the first pre-processing stage, while in other embodiments various CAM-based technique are used. Various TCAM-based schemes are disclosed for performing the second analysis stage. The methods enable packet payloads to be searched for search strings at line-rate speeds.

The invention discloses a distributed file system metadata management method facing to high-performance calculation. The method comprises the following steps of: 1) establishing a catalogue metadata server cluster, a file metadata server cluster and an object storage server cluster; 2) establishing a global counting-based bloom filter in the catalogue metadata server cluster; 3) when the operation request of a client side arrives, skipping to execute step 4) or 5); 4) enabling the catalogue metadata server cluster to respond to the catalogue operation request of the client side to manage the catalogue metadata; and 5) enabling the file metadata server cluster to respond to the file operation request of the client side to manage the file metadata data. According to the distributed file system metadata management method disclosed by the invention, the metadata transferring problem brought by catalogue renaming can be effectively solved, and the distributed file system metadata management method has the advantages of high storage performance, small maintenance expenditure, high load, no bottleneck, good expansibility and balanced load.

An integrated circuit, and method of reviewing values of one or more signals occurring within that integrated circuit, are provided. The integrated circuit comprises processing logic for executing a program, and monitoring logic for reviewing values of one or more signals occurring within the integrated circuit as a result of execution of the program. The monitoring logic stores configuration data, which can be software programmed in relation to the signals to be monitored. Further, the monitoring logic makes use of a Bloom filter which, for a value to be reviewed, performs a hash operation on that value in order to reference the configuration data to determine whether that value is either definitely not a value within the range or is potentially a value within the range of values. If the value is determined to be within the set of values, then a trigger signal is generated which can be used to trigger a further monitoring process.

Implementation of a Bloom filter using multiple single-ported memory slices. A control value is combined with a hashed address value such that the resultant address value has the property that one, and only one, of the k memories or slices is selected for a given input value, a, for each bank. Collisions are thereby avoided and the multiple hash accesses for a given input value, a, may be performed concurrently. Other embodiments are also described and claimed.

Methods, systems, and machine-readable media are disclosed for searching a corpus of information by utilizing a Bloom filter for caching query results. According to one aspect of the present invention, a method of caching information from a corpus of information can include populating one or more Bloom filters with a plurality of bits representative of information in the corpus of information. A search request can be received identifying requested information from the corpus of information. One or more bits in the filter(s) associated with the requested information can be checked and the requested information can be retrieved from the corpus of information based on results of said checking. Furthermore, the filter(s) can be used to determine which information to make available to a particular user in a system where certain information is associated with or access is limited to certain users or groups of users.

Symmetric Connection Detection (SCD) is a method of detecting when a connection has been fully established in a resource-constrained environment, and works in high-speed routers, at line speed. Many network monitoring applications are only interested in connections that become fully established, so other connection attempts, such as port scanning attempts, simply waste resources if not filtered. SCD filters out unsuccessful connection attempts using a simple combination of Bloom filters to track the state of connection establishment for every flow in the network. Unsuccessful flows can be filtered out to a very high degree of accuracy, depending on the size of the bloom filter and traffic rate. The SCD methodology can also easily be adapted to accomplish port scan detection, and to detect or filter other types of invalid TCP traffic.

Systems and methodologies are described that facilitate identifying peers based upon encoded signals during peer discovery in a peer to peer network. For example, direct signaling that partitions a time-frequency resource into a number of segments can be utilized to communicate an identifier within a peer discovery interval; thus, a particular segment selected for transmission can signal a portion of the identifier, while a remainder can be signaled based upon tones communicated within the selected segment. Moreover, a subset of symbols within the resource can be reserved (e.g., unused) to enable identifying and/or correcting timing offset. Further, signaling can be effectuated over a plurality of peer discovery intervals such that partial identifiers communicated during each of the peer discovery intervals can be linked (e.g., based upon overlapping bits and/or bloom filter information).

In one embodiment of the invention, service providers generate bloom filters with the user ID codes of registered users and exchange the bloom filters with one another. In response to a request to locate a first user, a first service provider will query its own registration database to determine if the first user is registered with the first service provider. If the first user is not registered with the first service provider, then the first service provider will query its bloom filters to identify other service providers with which the first user may be registered. A positive response from a bloom filter indicates that the first user may or may not be registered with the service provider associated with that bloom filter, and a negative response indicates with certainty that the first user is not registered with the service provider associated with that bloom filter.

A system that determines whether a pattern of data elements was observed in a stream of data. During operation, the system receives a query which seeks to determine whether the pattern of data elements was observed in the stream of data. In response to the query, the system performs a number of lookups in an enhanced Bloom filter to determine whether the pattern of data elements was observed in the stream of data, wherein the enhanced Bloom filter includes multiple instances of a Bloom filter, and wherein each instance of the Bloom filter is associated with a different time interval. If so, the system generates a notification that the pattern of data elements was observed.

Compressed Bloom filters that act as a message as well as a data structure provide smaller false positive rates, reduced bits broadcast and/or reduced computational overhead in distributed Web proxy servers and other distributed networks.

Some embodiments provide an optimized multi-hit caching technique that minimizes the performance impact associated with caching of long-tail content while retaining much of the efficiency and minimal overhead associated with first hit caching in determining when to cache content. The optimized multi-hit caching utilizes a modified bloom filter implementation that performs flushing and state rolling to delete indices representing stale content from a bit array used to track hit counts without affecting identification of other content that may be represented with indices overlapping with those representing the stale content. Specifically, a copy of the bit array is stored prior to flushing the bit array so as to avoid losing track of previously requested and cached content when flushing the bit arrays and the flushing is performed to remove the bit indices representing stale content from the bit array and to minimize the possibility of a false positive.

A network node (NB1) located within a domain is adapted to receive, from another node, a packet having an in-packet Bloom filter or Bloom filter equivalent encoding information about a route within the domain. The node reversibly modifies the in- packet Bloom filter or Bloom filter equivalent in a manner which is linear with respect to the operation used to add links to the Bloom filter or Bloom filter equivalent. The node then forwards the packet with its header containing the modified Bloom filter or Bloom filter to another node (NA1). The invention allows secure Bloom filter-based routing in a domain (Domain B), while requiring that only routers (NB1) at the domain boundary are secure routers. Other routers (NB2, NB3, NB4) in the domain may operate conventionally, and may be secure routers or insecure routers. The modification may be a bit permutation.

A system for index searching comprises a first index, a second index, a processor, and a memory. The processor is configured to determine, using a Bloom filter for the second index of fingerprints, whether the Bloom filter indicates that an entry corresponding to a segment has been previously stored in the second index. The processor is further configured to, in the event that the Bloom filter indicates that an entry corresponding to the segment has been previously stored in the second index, retrieve a second index entry associated with the segment that has been previously stored. The memory is coupled to the processor and configured to provide the processor with instructions.

In one embodiment, IP lookup into a routing table having prefixes of different prefix lengths is performed by hashing a candidate prefix value to generate a plurality of hash values, where m seed hash values are generated by applying m seed hash functions and one or more additional hash values are generated by combining two or more of the seed hash values in different ways, e.g., using a bit-wise XOR function. The hash values are used to perform membership probes into a Bloom filter programmed with the prefixes corresponding to all of the different prefix lengths in the routing table without having to expand any of the prefixes programmed into the Bloom filter.

A hierarchical data structure of digested payload information (e.g., information within a payload, or information spanning two or more payloads) allows a payload excerpt to be attributed to earlier network flow information. These compact data structures permit data storage reduction, while permitting efficient query processing with a low level of false positives. One example of such a compact data structure is a hierarchical Bloom filter. Different layers of the hierarchy may correspond to different block sizes.

The invention provides a method for implementing transaction query in a distributed database on the basis of a block chain. The method comprises the following steps: step S101, designing a Bloom filter, and finding out a candidate block which comprises an assigned transaction ID according to the designed Bloom filter; step S102, traversing and screening the candidate block, finding out a specific block, which comprises a transaction ID, of target transaction, and then constructing a Verify Message; step S103, verifying the completeness of a transaction path of the block by analysis of the Verify Message; and step S104, returning corresponding information according to a verify result, and implementing query of optional transaction in the distributed database on the basis of the block chain. By the method, query of the optional transaction in the distributed database on the basis of the block chain is realized, and the utilization rate of data of the block chain is increased.