Method and system for meta-estimation of bloom filters with counting vectors

Abstract

Systems and methods are disclosed for generating meta-estimates of data structures representing identifiers. A system maintains one or more data records, the one or more data records including one or more identifiers and one or more attributes. Using the data records, the system can generate a first data structure representing the plurality of data records, such as a probabilistic data structure. The first data structure can have a plurality of registers. The system can identify a subset of the plurality of registers that are equal to a predetermined value and generate a second data structure, the second data structure representing the subset of the plurality of registers. The system can then store the second data structure as a meta-estimate of the first and can use the second data structure for additional processing operations.

Description

[0001] Related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application Serial No. 63 / 087,791, filed October 5, 2020, which is hereby incorporated herein by reference in its entirety. Background Technology

[0003] In many cases, as part of data analysis or processing, computational and data analysis systems can determine the intersection or union of large datasets. Computing the union, intersection, or frequency of large datasets distributed across multiple sources typically involves sharing information about the large datasets among multiple sources. Information from each source may include private or protected information, and sharing such information can have negative impacts on privacy and security. Summary of the Invention

[0004] At least one aspect of this disclosure relates to a method for meta-estimating a data structure representing identifiers. The method may include maintaining a plurality of data records, the plurality of data records including one or more identifiers and one or more attributes. The method may include generating a first data structure representing the plurality of data records. The first data structure may have a plurality of first registers. At least one of the plurality of first registers may be populated based on a hash of each of the plurality of identifiers. The method may include identifying a subset of the plurality of first registers equal to a predetermined value. Each subset of the plurality of first registers may be identified by an index value. The method may include generating a second data structure representing a subset of the plurality of first registers. The second data structure may have a plurality of second registers. At least one of the plurality of second registers may be populated based on a hash of the index values ​​of each subset of the plurality of first registers. The method may include storing the second data structure in memory, the second data structure representing a subset of the plurality of first registers.

[0005] In some embodiments of the method, the method may include retrieving a third data structure having a plurality of third registers. In some embodiments of the method, each of the plurality of third registers may correspond to a corresponding one of a plurality of second registers of a second data structure. In some embodiments of the method, the method may include using the second data structure and the third data structure to determine the intersection value of information represented by the second data structure and information represented by the third data structure.

[0006] In some embodiments of the method, the method may further include using intersection values ​​to determine the union value of information represented by a second data structure and information represented by a third data structure.

[0007] These and other aspects and implementations are discussed in detail below. The foregoing information and the following detailed description include illustrative examples of various aspects and implementations, and provide an overview or framework for understanding the nature and character of the claimed aspects and implementations. The drawings provide illustration and further understanding of the various aspects and implementations, and are incorporated in and constitute a part of this specification. Aspects can be combined, and it will be readily understood that features set forth in the context of one aspect can be combined with features set forth in the context of other aspects. The aspects can be implemented in any convenient form. For example, by way of a suitable computer program, which can be carried on a suitable carrier medium (computer readable medium), which can be a tangible carrier medium (e.g., a magnetic disk) or an intangible carrier medium (e.g., a communications signal). The aspects can also be implemented using suitable apparatus, which can take the form of a programmable computer running a computer program arranged to implement the aspect. As used in the specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. BRIEF DESCRIPTION OF DRAWINGS

[0008] The drawings are not intended to be to scale. Like references signs and names in the various drawings indicate similar elements. For clarity not every component can be marked in every drawing. In the drawings:

[0009] Figure 1 a block diagram illustrating an example system for meta-estimating data structures representing large data sets;

[0010] Figure 2 a block diagram illustrating an example data flow for meta-estimating data structures representing large data sets;

[0011] Figure 3 a block diagram illustrating an example method for meta-estimating data structures representing large data sets;

[0012] Figure 4 a general architecture of an illustrative computer system that can be used to implement any of the computers discussed herein;

[0013] Figure 5 a plot illustrating example experimental data including error and bias as the bloom filter size is increased;

[0014] Figure 6 a plot illustrating example experimental data including error and bias as the VoC sketch size is increased; and

[0015] Figure 7A graph depicting example experimental data including error and bias for an exponential Bloom filter implementation as the Bloom filter size is increased. DETAILED DESCRIPTION

[0016] The following is a detailed description of various concepts related to techniques, approaches, methods, apparatuses, and systems for meta-estimating data structures representing identifiers, and implementations thereof. The various concepts introduced above and discussed in greater detail below can be implemented in any of numerous ways, as the described concepts are not limited to any particular manner of implementation. Examples of specific implementations and applications are provided for illustrative purposes only.

[0017] Large data sets, such as data sets representing groups of client device identifiers, can be difficult to represent efficiently. In particular, analysis of large data sets, such as data sets maintained by large content providers, can be challenging in terms of computational efficiency and privacy protection. For example, certain content providers can store lists of client device identifiers associated with various attributes, such as demographic information (e.g., age, gender, location, interests, etc.), client device attributes (e.g., device type), and other information received from client devices. Often, this information can include private or protected information. Content providers can generate data structures or sketches that represent large data sets in a manner that protects privacy. These sketches can be shared among other content providers for further analysis without revealing the information represented by each sketch.

[0018] However, sketch representations of large data sets, such as exponential Bloom filter (EBF) sketches (sometimes referred to as LiquidLegions (LL) sketches or any distributed Bloom filter (ADBF) sketches), or Bloom filter (BF) sketches, etc., can be vulnerable to certain attacks in nature. In short, in certain cases, if certain preventative measures (e.g., distributed noise) are not included at the time of sketch generation, certain data represented by such sketches can be retrieved from the sketches themselves. This poses a security risk to systems that generate and rely on such sketches. Other types of sketches, such as count vector (VoC) sketches, do not have this issue and can improve the overall privacy and security of the information used to generate such sketches.

[0019] However, generating a VoC sketch using a large dataset can be more computationally intensive than generating an exponential Bloom filter sketch or a Bloom filter sketch. For example, unlike an EBF or BF sketch, the data items used to generate a VoC sketch must first be de-duplicated prior to sketch generation. The de-duplication process can introduce a significant amount of computational overhead to the sketch generation process. Unlike this, an EBF or BF sketch does not require data de-duplication prior to sketch generation, which makes them more efficient to generate, but also more susceptible to privacy and security issues. Thus, it would be advantageous for a system to convert an EBF or BF sketch to a VoC sketch to obtain the generation advantages of an EBF or BF sketch, while also having the privacy and security advantages of a VoC sketch.

[0020] The systems and methods of this technical solution provide such improvements. After generating an EBF or BF sketch, the systems and methods described herein can generate a VoC sketch that represents the information included in the EBF or BF sketch. After the VoC sketch has been generated, the VoC sketch can be securely transmitted to other computing systems for further processing, such as for computing the intersection, union, or other set properties of datasets represented in a similar VoC format. The de-duplication process required to generate a VoC sketch from the original data items is significantly more computationally intensive than first generating an EBF or BF sketch and converting it to a VoC representation. Thus, the systems and methods of this technical solution provide significant improvements to data processing systems that generate sketches of extremely large datasets.

[0021] A simplified example of one algorithm for performing the meta-estimation of a first EBF or BF sketch is included below in the pseudo-code of Algorithm 1.

[0022] Algorithm 1, in pseudo-code form

[0023] Input: stream of objects O, class S of sketch (e.g., S can be a VoC, or any other type of probabilistic data structure (e.g., HyperLogLog (HLL) sketch, BF sketch, EBF sketch, ADBF sketch, LL sketch, etc.).

[0024] Output: sketch of type ‘S’ (e.g., including a data structure selected to probabilistically represent elements of the input set, etc.).

[0025] Algorithm 1 pseudo-code steps:

[0026] b = ADBF()

[0027] for o in O:

[0028] b.add(o);

[0029] s = S();

[0030] for register_index in b:

[0031] if b[register_index]== 1:

[0032] s.add(register index)

[0033] return s

[0034] The output sketch S can be used for further processing operations, such as for determining intersections (or unions) between other sketches that have similar generation parameters (e.g., generated in the same way within the same identifier universe). One way that sketches can be used is in Algorithm 2, which can be used to estimate the cardinality of the union (e.g., or the intersection, etc.) of the set of identifiers (e.g., or the set of inputs, etc.) estimated from all of the original data streams that generated the input sketches to Algorithm 1 described above. One implementation of Algorithm 2 is described below in the following pseudo code. In the pseudo code of Algorithm 2, it should be understood that the "→" operation indicates a transformation from one data type to another data type. For example, "Set1 → ABDF1 → VoC1" can be understood to mean that the data set "Set 1" is used as input to the above Algorithm 1 to transform into the ABDF sketch "ABDF1", which is then transformed into the VoC sketch "VoC1".

[0035] Algorithm 2, in pseudo code

[0036] Input: A set of sketches Z created by Algorithm 1 with the same settings (e.g., the same input sketch type as the ABDF described above, but it should be understood that any other sketch type can be used).

[0037] Output: An estimate of the cardinality of the union of all of the original streams (e.g., O for each ABDF, etc.) that generated the sketches Z.

[0038] Algorithm 1 pseudo code steps:

[0039] Set1 → ADBF1 → VoC1

[0040] Set2 → ADBF2 → VoC2

[0041] VoC.estimate_union([VoC1, VoC2]) = number of 1s in ABDF1 | ABDF2.

[0042] Cardinality(Set1 | Set2) = ADBF.estimate_from_ones(number of 1s in ADBF1 | ADBF2).

[0043] Thus,

[0044] Cardinality (Set1 | Set2)= ADBF.estimate_from_ones(VoC.estimate_union([VoC1, VoC2]))

[0045] In the above pseudo code, it should be understood that the “|” operator can correspond to a union operation. For example, in the context of sets, the “|” operator can correspond to a set union operation and returns a set containing the elements in Set1, the elements in Set2, and the elements in both Set1 and Set2. In the context of sketches, such as ADBF sketches, the “|” operator can correspond to an “or” operator, or more simply, a binary “or” operation across all corresponding register positions in ADBF1 and ADBF2.

[0046] Before describing the systems and methods in detail, it can be helpful to consider the following pseudo code, which can be similar to Python code. The following pseudo code presents various algorithms for generating and controlling the data structures described herein (e.g., ADBF sketches, VoC sketches, etc.).

[0047] The following pseudo code provides algorithms and functions for generating and controlling VoC sketches. Operations related to VoC sketches can be contained in a class, such as the VoCOps class shown below.

[0048] class VoCOps():

[0049] def __init__(self, hash_func=default_hash_func, seeds=1):

[0050] '''

[0051] Create VoC operations using the specified hash function.

[0052] '''

[0053] self.hash_func=hash_func

[0054] self.seeds = [i for i in range(seeds)]

[0055] def add(self, bit_vector, element):

[0056] '''

[0057] Add an element to the bit_vector.

[0058] bit_vector: The bit vector to be modified

[0059] element: The object to be consumed by the hash function.

[0060] '''

[0061] vector_size = len(bit_vector)

[0062] For seed in self.seeds:

[0063] bit_vector[self.hash_func(element,seed)% vector_size]+= 1 / len(self.seeds)

[0064] def estimated_length(self, bit_vector):

[0065] return np.sum(bit_vector)

[0066] def intersection(self, bit_vector1, bit_vector2):

[0067] The statement `assert len(bit_vector1) == len(bit_vector2)` indicates that "intersecting VoC vectors may require the same size."

[0068] k = len(bit_vector1)

[0069] n1 = self.estimated_length(bit_vector1)

[0070] n2 = self.estimated_length(bit_vector2)

[0071] return (np.sum((bit_vector1 - n1 / k) (bit_vector2 - n2 / k))

[0072] len(self.seeds) k / (k-1))

[0073] def union(self, bit_vector1, bit_vector2):

[0074] assert len(bit_vector1)== len(bit_vector2), "For union VoC vectors can need same size"

[0075] a = self.estimated_length(bit_vector1)

[0076] b = self.estimated_length(bit_vector2)

[0077] return a + b - self.intersection(bit_vector1, bit_vector2)

[0078] As shown in the above pseudo code, VoCOps classes can include functions that perform operations on VoC sketches (e.g., referred to as bit_vector, bit_vector1, bit_vector2, etc.). For example, at least one function that initializes VoC operations, at least one function that adds elements to a VoC sketch, at least one function that provides an estimated length of a VoC sketch, at least one function that returns a number of intersecting elements of information sets represented by two VoC sketches, and at least one function that returns an estimated number of elements in a union of data sets represented by two VoC sketches.

[0079] To further illustrate examples of various algorithms that perform the functions described herein, included below are algorithms and functions for generating and manipulating Bloom filter data structures. Operations related to BF sketches can be contained in a class, such as the BloomFilterOps class shown below.

[0080] class BloomFilterOps:

[0081] '''

[0082] Defines BloomFilter-type actions.

[0083] '''

[0084] def __init__(self, num_hash_funcs, hash_func=default_hash_func):

[0085] '''

[0086] Creates a Bloom filter using the specified hash functions and the specified bit vector size.

[0087] num_hash_funcs: Number of hash functions to use.

[0088] hash_func (function: (element, seed)-> integer): Hash function to use to set the bit vector.

[0089] '''

[0090] self.hash_func=hash_func

[0091] self.seeds = [i for i in range(num_hash_funcs)]

[0092] def add(self, bit_vector, element):

[0093] '''

[0094] Adds an element to the Bloom filter.

[0095] bit_vector: Bit vector to modify

[0096] element: Object to consume by the hash function.

[0097] '''

[0098] vector_size = len(bit_vector)

[0099] for seed in self.seeds:

[0100] bit_vector[self.hash_func(element, seed)% vector_size]= 1

[0101] def contains(self, bit_vector, element):

[0102] '''

[0103] Checks membership of an element. False positive rate can be queried by calling false_positive_rate().

[0104] '''

[0105] vector_size = len(bit_vector)

[0106] for seed in self.seeds:

[0107] if not bit_vector[self.hash_func(element, seed)% vector_size]:

[0108] return False

[0109] return True

[0110] def expected_false_positive_rate(self, bit_vector, expected_n):

[0111] '''

[0112] Assuming a uniform hash function, query the false positive rate of this Bloom filter for the given number of elements.

[0113] If no arguments are given, check the estimated false positive rate of this Bloom filter for the current capacity.

[0114] '''

[0115] vector_size = len(bit_vector)

[0116] # Probability that a particular bit is not set to 1 by the hash function

[0117] prob_bit_not_one = 1 - 1 / vector_size

[0118] # Probability that a particular bit is not set to 1 by all hash functions

[0119] prob_bit_not_one_all = prob_bit_not_one len(self.seeds)

[0120] # Probability that a particular bit is not 1 after N elements

[0121] prob_bit_not_one_n = prob_bit_not_one_all expected_n

[0122] # Probability that a particular bit is 1

[0123] prob_bit_one = 1 - prob_bit_not_one_n

[0124] # Probability of a negative element colliding with all 1 bits

[0125] return prob_bit_one len(self.seeds)

[0126] def estimated_length(self, bit_vector):

[0127] return self.estimated_length_manual(np.sum(bit_vector), len(bit_vector))

[0128] def estimated_length_manual(self, num_ones, length):

[0129] m = length

[0130] if num_ones == m:

[0131] return math.inf

[0132] k = len(self.seeds)

[0133] return -(m / k) (math.log(1-num_ones / m))

[0134] def union(self, bit_vector1, bit_vector2):

[0135] '''Returns the size of the Bloom filter union of two bit vectors'''

[0136] bit_vector3 = np.logical_or(bit_vector1, bit_vector2)

[0137] return self.estimated_length(bit_vector3)

[0138] def intersection(self, bit_vector1, bit_vector2):

[0139] '''Returns the size of the intersection of two bit vectors' Bloom filters'''

[0140] return (self.estimated_length(bit_vector1)+\

[0141] self.estimated_length(bit_vector2)-\

[0142] self.union(bit_vector1, bit_vector2))

[0143] As shown in the above pseudo code, the BloomFilterOps class can include functions that perform operations on BF sketches (e.g., referred to as bit_vector, bit_vector1, bit_vector2, etc.). For example, at least one function that initializes a BF sketch operation, at least one function that adds an element to a BF sketch, at least one function that checks whether an element is in a BF sketch, at least one function that returns the false positive rate of a BF sketch member, at least one function that returns the estimated length of a BF sketch, at least one function that returns the number of intersection elements of information sets represented by two BF sketches, and at least one function that returns the estimated number of elements in the union of data sets represented by two BF sketches.

[0144] To further illustrate examples of various algorithms that perform the functions described herein, the following includes algorithms and functions for generating and manipulating exponential Bloom filter data structures. Operations related to EBF sketches can be contained in a class, such as the ExpBloomFilterOps class shown below.

[0145] class ExpBloomFilterOps:

[0146] def __init__(self, decay_rate, hash_func=default_hash_func):

[0147] self.hash_func = hash_func

[0148] self.decay_rate = decay_rate

[0149] self.seed = 0

[0150] self.seeds =[0]

[0151] def estimated_length(self, vector):

[0152] return self.estimated_length_manual(sum(vector), len(vector))

[0153] @lru_cache()

[0154] def estimated_length_manual(self, num_ones, buckets):

[0155] a = self.decay_rate

[0156] def _expected_num_bits(reach):

[0157] """Expected number of bits for base activation."""

[0158] if reach <= 0:

[0159] return 0

[0160] return (1 - (- special.expi(-a reach / (np.exp(a)- 1))+

[0161] special.expi(-a np.exp(a) reach / (np.exp(a)- 1))) / a)

[0162] def _clip(x, lower_bound, upper_bound):

[0163] return max(min(x, upper_bound), lower_bound)

[0164] x = num_ones

[0165] m = buckets

[0166] p = _clip(x / m, 0, 1)

[0167] result = invert_monotonic(_expected_num_bits, epsilon=1e-7)(p) m

[0168] assert result >= 0, "Negative estimates should never occur."

[0169] return result

[0170] @lru_cache()

[0171] def compute_register_probs(self, num_values):

[0172] """Calculate the probability per register."""

[0173] probs = np.exp(- self.decay_rate (np.arange(num_values)+ 1) /

[0174] (num_values ​​+ 1))

[0175] return np.cumsum(probs / np.sum(probs))

[0176] def union(self, bit_vector1, bit_vector2):

[0177] '''Returns the size of the union of two bit vectors from the Bloom filter'''

[0178] bit_vector3 = np.logical_or(bit_vector1, bit_vector2)

[0179] return self.estimated_length(bit_vector3)

[0180] def intersection(self, bit_vector1, bit_vector2):

[0181] '''Returns the size of the intersection of two bit vectors obtained through Bloom filters'''

[0182] return (self.estimated_length(bit_vector1)+\

[0183] self.estimated_length(bit_vector2)-

[0184] self.union(bit_vector1, bit_vector2))

[0185] def add(self, vector, element):

[0186] float_hash = abs(int(self.hash_func(element, self.seed))) / 2147483647

[0187] probs = self.compute_register_probs(len(vector))

[0188] index = bisect.bisect_left(probs, float_hash)- 1

[0189] vector[index]= 1

[0190] As shown in the above pseudo code, the ExpBloomFilterOps class can include functions that perform operations on EBF sketches (e.g., referred to as bit_vector, bit_vector1, bit_vector2, etc.). For example, at least one function that initializes an EBF sketch operation, at least one function that adds an element to an EBF sketch, at least one function that returns the probability of filling for each register from a plurality of elements, at least one function that returns an estimated length of an EBF sketch, at least one function that returns the number of elements in the intersection of the information sets represented by two EBF sketches, and at least one function that returns an estimated number of elements in the union of the data sets represented by two EBF sketches.

[0191] While the above algorithms and pseudo code, which can be or be similar to python code, are shown at a high level of generality, it should be understood that any of the computing devices described herein can perform any or any portion of the above described functions or operations. However, it should also be understood that the above pseudo code should not be considered limiting as to the range of functions that the computing devices described herein can perform.

[0192] Reference is now made to Figure 1, illustrating a block diagram of an example system 100 for meta-estimating data structures representing identifiers, in accordance with one or more embodiments. The system 100 can include at least one data processing system 105, at least one network 110, one or more client devices 120A-N (sometimes referred to as client devices 120), and at least one content provider 160. The data processing system 105 can include at least one data record maintainer 130, at least one data structure generator 135, at least one register identifier 140, at least one meta-estimator 145, at least one meta-estimation data manager 150, at least one union determiner 155, and at least one database 115. The database 115 can include one or more identifiers 170A-N (sometimes generally referred to as identifiers 170) and one or more attributes 175A-N (sometimes generally referred to as attributes 175). In some embodiments, the database 115 is external to the data processing system 105 and can be accessed by one or more of the computing devices described herein (e.g., the data processing system 105, the content provider 160, the client devices 120, the computing system 400, etc.) via the network 110.

[0193] Each of the components of the system 100 (e.g., the data processing system 105, the network 110, the client devices 120A-N, the content provider 160, the data record maintainer 130, the data structure generator 135, the register identifier 140, the meta-estimator 145, the meta-estimation data manager 150, the union determiner 155, the database 115, etc.) can be implemented using the hardware components or a combination of software and hardware components of the computing system (e.g., the computing system 400, the data processing system 105, any other computing system described herein, etc.) detailed herein. Each of the components of the data processing system 105 can perform the functions detailed herein. Figure 4

[0194] ​The data processing system 105 can include at least one processor and memory, e.g., processing circuitry. The memory can store processor-executable instructions that, when executed by the processor, cause the processor to perform one or more of the operations described herein. The processor can include a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc., or a combination thereof. The memory can include, but is not limited to, electronic, optical, magnetic, or any other storage or transmission device capable of providing the processor with program instructions. The memory can further include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ASIC, field-programmable gate array (FPGA), read-only memory (ROM), random-access memory (RAM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory from which the processor can read instructions. The instructions can include code from any suitable computer programming language. The data processing system 105 can include one or more computing devices or servers that can perform various functions as described herein. The data processing system 105 can include any or all of the components and perform any or all of the functions of the computer system 400 described herein. Figure 4 The data processing system 105 can include at least one processor and memory, e.g., processing circuitry. The memory can store processor-executable instructions that, when executed by the processor, cause the processor to perform one or more of the operations described herein. The processor can include a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc., or a combination thereof. The memory can include, but is not limited to, electronic, optical, magnetic, or any other storage or transmission device capable of providing the processor with program instructions. The memory can further include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ASIC, field-programmable gate array (FPGA), read-only memory (ROM), random-access memory (RAM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory from which the processor can read instructions. The instructions can include code from any suitable computer programming language. The data processing system 105 can include one or more computing devices or servers that can perform various functions as described herein. The data processing system 105 can include any or all of the components and perform any or all of the functions of the computer system 400 described herein.

[0195] The network 110 can include a computer network, such as the Internet, a local area network, a wide area network, a metropolitan area network, or other regional network, an intranet, a satellite network, other computer networks such as voice or data mobile telephone communication networks, and combinations thereof. The data processing system 105 of the system 1 can communicate through the network 110, for example, with one or more client devices 120 or content providers 160. The network 110 can be any form of computer network that can relay information between the client devices 120, the content providers 160, the data processing system 105, and one or more content sources (e.g., web servers, etc.). In some embodiments, the network 110 can include the Internet and / or other types of data networks, such as a local area network (LAN), a wide area network (WAN), a cellular network, a satellite network, or other types of data networks. The network 110 can also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) configured to receive and / or transmit data within the network 110. The network 110 can further include any number of hardwired and / or wireless connections. Any or all of the computing devices described herein (e.g., the data processing system 105, the computer system 400, etc.) can communicate wirelessly (e.g., via WiFi, cellular, radio, etc.) with a transceiver that is hardwired (e.g., via a fiber optic cable, a CAT5 cable, etc.) to other computing devices in the network 110. Any or all of the computing devices described herein (e.g., the data processing system 105, the computer system 400, etc.) can also communicate wirelessly with the computing devices of the network 110 via a proxy device (e.g., a router, a network switch, or a gateway).

[0196] Each of the one or more client devices 120 can each include at least one processor and memory, e.g., processing circuitry. The memory can store processor-executable instructions that, when executed by the processor, cause the processor to perform one or more of the operations described herein. The processor can include a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc., or a combination thereof. The memory can include, but is not limited to, electronic, optical, magnetic, or any other storage or transmission device capable of providing the processor with program instructions. The memory can further include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ASIC, read-only memory (ROM), random-access memory (RAM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory from which the processor can read instructions. The instructions can include code from any suitable computer programming language. The client devices 120 can each include one or more computing devices or servers that can perform various functions as described herein. The client devices 120 can each include any or all of the components and perform any or all of the functions as described herein in connection with Figure 4 any or all of the computer systems 400 described.

[0197] Each of the client devices 120 can be a computing device (e.g., computing device 400, etc.) configured to communicate via the network 110 to transmit messages to or receive messages from the data processing system 105 or the content provider 160. The client devices 120 can be desktop computers, laptop computers, tablet computers, smartphones, personal digital assistants, mobile devices, consumer computing devices, servers, clients, digital video recorders, television set-top boxes, video game consoles, or any other computing devices configured to communicate via the network 110, etc. The client devices 120 can transmit one or more content requests to the content publisher, the content provider 160, or the data processing system 105. The client devices 120 can receive online content from the device to which the request was transmitted, where the content is selected based on device information (e.g., device model, device type, device operating system, device operating system version, device time, device date, any other contextual information described herein) about the client device 120 responsible for the request. The client devices 120 can be associated with various attributes and identifiers that can be transmitted in requests for content or other messages to the content provider 160 or the data processing system 105. For example, the client devices 120 can request an information resource from the content provider 160 or from the data processing system 105, can include a client device identifier and client device attributes in the request. The client devices 120 can receive a response to the transmitted request (e.g., from the computing device to which the request was transmitted), which can include content requested by the client devices 120.

[0198] The content provider 160 can include at least one processor and memory, e.g., processing circuitry. The memory can store processor-executable instructions that, when executed by the processor, cause the processor to perform one or more of the operations described herein. The processor can include a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc., or a combination thereof. The memory can include, but is not limited to, electronic, optical, magnetic, or any other storage or transmission device capable of providing the processor with program instructions. The memory can further include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ASIC, read-only memory (ROM), random-access memory (RAM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory from which the processor can read instructions. The instructions can include code from any suitable computer programming language. The content provider 160 can include one or more computing devices or servers that can perform various functions as described herein. The content provider 160 can include any or all components and perform any of the functions as described herein in connection withFigure 4 Any or all of the functionality of the computer system 400 described.

[0199] The content provider 160 can provide one or more information resources, which can include content (e.g., text, images, audio, video, any combination thereof, etc.) or scripts for retrieving content. The content provider 160 can record which client devices access online content items, including the time, identifiers, and attributes of the client devices that access the online content. The content provider 160 can provide content in response to one or more content requests from a client device. For example, the client device 120 can receive an information resource from the content provider 160, and the content provider can provide the information resource to the client device for display. The information resource can be a web page, a video, an image, or any other type of content described herein. The information resource provided by the content provider 160 can include one or more scripts that, when executed by the client device 120, cause the client device 120 to request additional content for insertion into the information resource. Thus, the content provider 160 can provide and record the attributes and identifiers of the client device 120 on a one-to-one basis with the client device 120 that makes the request. In some embodiments, the data processing system 105 can be the content provider 160 or perform the same functions as the content provider 160. In some embodiments, the content provider 160 can transmit client device identifiers and attributes extracted from content requests or from other messages to the data processing system 105.

[0200] Database 115 can be a database configured to store and / or maintain any information described herein. Database 115 can maintain one or more data structures that can contain, index, or otherwise store each of the values, pluralities, sets, variables, vectors, or thresholds described herein. Database 115 can be accessed using one or more memory addresses, index values, or identifiers of any item, structure, or region maintained in database 115. Database 115 can be accessed by components of data processing system 105 or any other computing device described herein via network 110. In some embodiments, database 115 can be internal to data processing system 105. In some embodiments, database 115 can exist external to data processing system 105 and can be accessed via network 110. Database 115 can be distributed across many different computer systems or storage elements and can be accessed via network 110 or a suitable computer bus interface. Data processing system 105 can store any or all of the results of computing, determining, selecting, identifying, generating, constructing, or calculating in one or more data structures indexed or identified with appropriate values in a memory of data processing system 105, or in database 115. Any or all of the values stored in database 115 can be accessed by any computing device described herein (e.g., data processing system 105) to perform any of the functionality or functions described herein.

[0201] Database 115 can include one or more data records, which can be data structures containing indexed information. One example of an indexed data record is client device identifier 170 (sometimes referred to as identifier 170), which can be received from client device 120 or from content provider 160. Identifier 170 can be stored in association with client device attributes 175 (sometimes generally referred to as attributes 175). In some embodiments, attributes can be indexed in one or more data structures in database 115 by corresponding identifier 170. For example, identifier 170 can point to or otherwise identify attributes 175 received from client device 120 corresponding to identifier 170.

[0202] The identifiers 170 maintained in the database 115 can be stored in one or more data structures, such as lists or index lookup tables. Each of the identifiers 170 can be associated with one or more attributes 175 that describe aspects of the corresponding client devices 120. Thus, each client device 120 can be associated with a respective identifier 170 and a respective one or more attributes 175. The attributes 175 can include attributes associated with the respective client devices 120 and can be stored in one or more lists or index data structures in the database 115. The attributes 175 can include demographic information, such as age, gender, interests, and other demographic information, and can include location information of various granularities, including regional information, coordinate information, or other types of location information, and can include device information, such as device type, device operating system version, device network speed, and any other device information described herein. Each of the attributes 175 can be associated with at least one respective client device 120 and at least one respective identifier 170.

[0203] The data record maintainer 130 can maintain one or more data records that include one or more identifiers 170 or one or more attributes 175. A data record can be a data structure that stores information in computer memory and can include an identifier 170 or an attribute 175 (of a client device 120, etc.). The data record maintainer 130 can receive a message, such as a request for content, from a client device 120 and can extract an identifier 170 of the client device and attributes 175 of the client device from the message. The data record maintainer 130 can generate a data record, such as a data structure (e.g., in memory of the data processing system 105, etc.), to store the identifier 170 and the attributes 175 received from the client device 120. The data record can be stored in the database 115. Each of the data records stored by the data record maintainer 130 can be associated with a location, address, or pointer to a location in the database 115 where the respective data record is located. Using the location of the data record, the data record maintainer 130 (e.g., or any other component of the data processing system 105) can access the data record and the identifiers 170 and attributes 175 included therein. In some embodiments, the data record maintainer 130 stores the data record (e.g., and the identifiers 170 and attributes 175, or any other data, etc.) in memory of the data processing system 105.

[0204] The data structure generator 135 can generate a sketch data structure (e.g., a BF sketch, an EBF sketch, etc.) that represents one or more of the data records maintained by the data record maintainer 130. The sketch data structure can be a Bloom filter data structure having a first number of registers initialized to zero. A Bloom filter can be a vector of registers (sometimes referred to as “buckets”), where each bucket is associated with a bucket identifier that corresponds to its position or index value in the Bloom filter. In some embodiments, a Bloom filter can be a bit string, where each position in the bit string functions as a register that can be set to zero or one. A bucket can have a value equal to zero or one. When initializing or generating a Bloom filter, the data structure generator 135 can initialize the Bloom filter buckets to an initialization value (e.g., zero, etc.). To add a data record to the Bloom filter, the data structure generator can apply one or more hash functions (e.g., MD5, SHA-1, SHA-2, etc.) to the data record (e.g., a data structure including an identifier 170 and attributes 175, etc.) to generate one or more hashed data records. The data structure generator 135 can extract a pointer to one or more buckets of the Bloom filter from the hashed data record and set the pointed-to bucket to a set value (e.g., one, etc.). The data structure generator 135 can repeat this process until all of the desired data records are included in the Bloom filter. The data structure generator 135 can retrieve or otherwise access one or more parameters of the Bloom filter when generating the sketch data structure. For example, the data structure generator 135 can receive parameters for the sketch to be generated, e.g., from an external computing device. The parameters can include the type of hash function used, the number of hash functions used, the number of buckets or positions in the Bloom filter, or any other information needed to generate the Bloom filter. The cardinality of a dataset (e.g., data records, etc.) used to generate a Bloom filter can be estimated by evaluating the following formula:

[0205] ,

[0206] where k is the number of hashes used to generate the Bloom filter, m is the number of bits (e.g., buckets) in the Bloom filter, x is the number of filled (e.g., set) buckets in the Bloom filter, and n is the estimated number of the cardinality of the Bloom filter.

[0207] In some embodiments, the data structure generator 135 can generate an exponential Bloom filter (EBF) as a sketch data structure. The exponential Bloom filter (EBF) can be similar to a Bloom filter data structure, except that a different range of probability values can be assigned to each bucket or bit of the exponential data structure. In some embodiments, the EBF can be a bit string, where each position in the bit string serves as a register (or bucket) that can be set to zero or one. When initializing or generating the EBF, the data structure generator 135 can initialize the Bloom filter buckets to an initialization value (e.g., zero, etc.). For example, the probability assigned to each bucket of the EBF sketch can correspond to a value of an exponential distribution, as shown in the “compute_register_probs” pseudo-code function described above. By adjusting the decay rate of the exponential distribution of the EBF sketch, different probabilities (e.g., bit population, etc.) can be determined. To add a data record to the EBF, the data structure generator 135 can compute a hash value of the data record (e.g., using a hash function on the data structure and its contents, etc.). The hash value can be converted to a number in the range [0, 1] (e.g., by dividing the range of the output of the hash function, etc.). Because each bucket in the EBF corresponds to a range of probabilities in the domain [0, 1], the data structure generator 135 can identify which range of probabilities the converted hash value falls into. The bucket associated with the corresponding range can then be set to a set value (e.g., 1). This process can be repeated until all desired data records are added to the EBF sketch. Each position (e.g., bucket, register, etc.) in the Bloom filter sketch or EBF sketch can be identified by a corresponding index value.

[0208] The register identifier 140 can identify a subset of registers in the sketch generated by the data structure generator 135 that are equal to a predetermined value, such as a set value. To do so, the register identifier 140 can iterate through each register in the populated sketch (e.g., by index value, etc.) and identify the subset of registers in the sketch that are set to the set value by comparing each register to the predetermined value (e.g., set value, unset value). The register identifier 140 can provide the index values of the registers in the subset to the meta-estimator 145 for further processing. For example, if positions 1, 9, and 83 in the sketch are identified as having a set value, and the other positions of the sketch are identified as having a sketch value, the register identifier 140 can provide the indices 1, 9, and 83 to the meta-estimator 145.

[0209] The meta-estimator 145 can generate a second data structure, e.g., a VoC sketch, that represents the index values of a subset of registers in the sketch generated by the data structure generator 135. The VoC sketch can include a counter register that can be set to an unset value (e.g., zero) and can be populated by the meta-estimator 145 when populating the VoC sketch. To initialize the VoC sketch, the meta-estimator can generate a VoC sketch with a predetermined (e.g., retrieved from a setting or from another computing device, etc.) length (e.g., number of registers or buckets) where each register is initially set to an unset value (e.g., zero). To populate the VoC data structure, the meta-estimator can take an input value, e.g., an index value received from the register identifier 140, perform a hash function on the value to generate a hashed value. The meta-estimator 145 can then perform a modulo operation on the hashed value and the number of registers in the VoC sketch. The result of the modulo operation can have a value equal to an index value of one of the registers in the VoC sketch. The meta-estimator can increment the register associated with the index value equal to the modulo operation, thereby populating the VoC sketch with the input value. The meta-estimator can repeat this process for each index value received from the register identifier 140, thereby generating a populated second data structure that represents the registers in the Bloom filter or the Exponential Bloom filter sketch that are set to a set value.

[0210] The meta-estimated data manager 150 can store the second data structure, e.g., the VoC sketch, that represents a subset of registers of a Bloom filter or an Exponential Bloom filter in a memory of the data processing system 105. In some embodiments, the meta-estimated data manager 150 can store values in association with an identifier of the VoC sketch in the database 115. In some embodiments, the identifier of the VoC sketch can correspond to a set of data records (e.g., the identifier 170 and the attributes 175) that were added to an initial Bloom filter (or Exponential Bloom filter) sketch. In some embodiments, the meta-estimated data manager 150 can transmit the VoC sketch to another computing device, e.g., the content provider 160 that requested the sketch, or to an external computing device that can aggregate the sketch in a multi-party computation (MPC) protocol. In some embodiments, the meta-estimated data manager 150 can receive a request for a VoC sketch that represents a meta-estimate of a sketch generated from an identifier and attributes.

[0211] The request can identify events or attributes, or other parameters, of the identifiers 170 or attributes 175 that should be included in the initial sketch. The meta-estimate data manager 150 can identify data records (e.g., identifiers 170 or attributes) that should be included in the initial sketch by scanning the database 115 for matching attributes 175. If a matching attribute is identified, the meta-estimate data manager 150 can provide the identifier 170 associated with the matching attribute to the data structure generator 135 for inclusion in the initial (e.g., Bloom filter, exponential Bloom filter, etc.) sketch. In some embodiments, the data records used to generate the initial sketch do not include any attributes 175, but rather identifiers 170 that correspond to the attributes required in the matching meta-estimate sketch request. After generating the VoC sketch, the meta-estimate data manager 150 can transmit the sketch to the computing device responsible for the request.

[0212] To compute a union value of two VoC sketches (e.g., a cardinality of a union of the sets of index values used to generate the two VoC sketches, etc.), the union determiner 155 can retrieve two VoC sketches generated from the same universe of data records (e.g., but can contain different data records) using the same parameters. To compute the union value, the union determiner 155 can first determine an intersection value of the two VoC sketches (e.g., a cardinality of an intersection of the sets of index values used to generate the two VoC sketches, etc.). To determine the intersection value, the union determiner 155 can determine an average register value (e.g., an average of the register values) in the first VoC sketch and an average register value of the second VoC sketch. The average register value of the first VoC sketch can be subtracted from each register value in the first VoC sketch to generate a first normalized VoC sketch, and the average register value of the second VoC sketch can be subtracted from each register value in the second VoC sketch to generate a second normalized VoC sketch. Next, a dot product can be computed between the first normalized VoC sketch and the second normalized VoC sketch by multiplying each corresponding register in the two normalized sketches and summing to compute a sum of the resulting products. In some embodiments, the union determiner 155 can multiply the resulting sum by an adjustment value equal to a number of positions in the first vector divided by a difference of the number of positions in the first vector minus one. In some embodiments, no adjustment value is used. The summed value, or the summed value multiplied by the adjustment value if used, can be the intersection value of the two VoC sketches.

[0213] The union determiner 155 can then use the intersection value to determine the cardinality of the union of the index values that have been used to generate both sketches (e.g., the number of unique index values that contributed to both VoC sketches, etc.). To do so, the union determiner 155 can estimate the number of index values used to generate the first sketch by summing the register values of the VoC sketch. The union determiner 155 can estimate the number of index values used to generate the second sketch in the same manner. To compute the union value, the union determiner 155 can simply add the estimated number of index values of the first VoC sketch to the estimated number of index values of the second VoC sketch and subtract the intersection value. The resulting value is the union value, which represents the number of unique index values in both VoC sketches (or set registers in the initial sketch).

[0214] A Bloom filter and an exponential Bloom filter can be combined using an OR operation (e.g., an OR operation on all corresponding registers, etc.). Thus, the resulting union value of the two VoC sketches is an estimate of the total number of set values of the combination of initial sketches used to generate the two VoC sketches. Since the estimate of the cardinality of the data records used to generate the initial sketches depends only on the number of set values, the cardinality of the data records used to generate the initial sketches can be computed by evaluating the following formula:

[0215]

[0216] where k is the number of hashes used to generate the Bloom filter, m is the number of bits (e.g., buckets) in the Bloom filter, x is the union value of the two VoC sketches, and n is the estimated number of the cardinality of the Bloom filter.

[0217] Reference is now briefly made to Figure 2 a block diagram 200 of an example data flow for meta-estimating a data structure representing a large data set. In the figure, an input data set 205 can be converted into a first data structure 210, which can set registers to set values that are not set. In this figure, "1" is a set value and "0" is a not set value. The index values of the set registers can be used in the meta-estimation 215 process described above (e.g., by the meta-estimator 145, etc.). The output of the meta-estimation 215 process can be a second data structure (e.g., a VoC sketch, etc.). The second data structure can be composed of counter registers that can be incremented as described above.

[0218] Reference is now made to Figure 3 an illustrative flow diagram of a method 300 for meta-estimating a data structure representing an identifier. The method 300 can be performed by the data processing system 105, described herein in connection with Figure 4 ​The computer system 400 described herein or any other computing device described herein performs, carries out, or otherwise implements the process. In short, the data processing system (e.g., data processing system 105, any other computing device described herein, etc.) can maintain data records (step 302); generate a first data structure (step 304); select the first data structure... k Step 306: 1. Register; 2. Determine if the register is equal to a predetermined value (step 308); 3. Generate the second data structure (step 310); 4. Determine the counter register. k Is it equal to the number of registers in the first data structure? n (Step 312); Increment counter register k (Step 314); Store the second data structure (Step 316); and determine the union value (Step 318).

[0219] More specifically, the data processing system can maintain data records (step 302). The data processing system can maintain one or more data records, which include one or more identifiers (e.g., identifier 170, etc.) or one or more attributes (e.g., attribute 175, etc.). Data records can be data structures that store information in computer memory and can include identifiers or attributes of client devices (e.g., client device 120, etc.). The data processing system can receive messages from client devices, such as content requests, and can extract the identifiers or attributes of client devices from the messages. The data processing system can generate data records, such as data structures (e.g., in the data processing system's memory, etc.), to store the identifiers or attributes received from the client devices. Data records can be stored in a database (e.g., database 115). Each data record stored by the data processing system can be associated with a location, address, or pointer to a location where the corresponding data record resides in the database or memory. Using the location of the data record, the data processing system can access the data record and the identifiers or attributes included therein. In some embodiments, the data processing system stores data records in the memory of data processing system 105.

[0220] The data processing system can generate a first data structure (step 304). The data processing system can generate a sketch data structure (e.g., a BF sketch, an EBF sketch, etc.) that represents one or more of the data records maintained by the data processing system as the first data structure. The sketch data structure can be a Bloom filter data structure having a first number of registers initialized to zero. A Bloom filter can be a vector of registers (sometimes referred to as “buckets”), where each bucket is associated with a bucket identifier that corresponds to its position or index value in the Bloom filter. In some embodiments, a Bloom filter can be a bit string, where each position in the bit string functions as a register that can be set to zero or one. A bucket can have a value equal to zero or one. When initializing or generating a Bloom filter, the data processing system can initialize the Bloom filter buckets to an initialization value (e.g., zero, etc.). To add a data record to the Bloom filter, the data processing system can apply one or more hash functions (e.g., MD5, SHA-1, SHA-2, etc.) to the data record to generate one or more hashed data records. The data processing system can extract a pointer to one or more buckets of the Bloom filter from each hashed data record and set the pointed-to buckets to a set value (e.g., one, etc.). The data processing system can repeat this process until all desired data records have been hashed and represented in the Bloom filter. The data processing system can retrieve or otherwise access one or more parameters of the Bloom filter when generating the sketch data structure. For example, the data processing system can receive parameters for the sketch to generate, e.g., from an external computing device. The parameters can include the type of hash function used, the number of hash functions used, the number of buckets or positions in the Bloom filter, or any other information needed to generate the Bloom filter. The cardinality of a dataset (e.g., data records, etc.) used to generate a Bloom filter can be estimated by evaluating the following formula (e.g., by the data processing system or any component thereof):

[0221] ,

[0222] wherein k is the number of hashes used to generate the Bloom filter, m is the number of bits (e.g., buckets) in the Bloom filter, x is the number of filled (e.g., set) buckets in the Bloom filter, and n is the estimated number of the cardinality of the Bloom filter.

[0223] In some embodiments, the data processing system can generate an exponential Bloom filter (EBF) as the sketch data structure. The exponential Bloom filter (EBF) can be similar to the Bloom filter data structure, except that a different range of probability values can be assigned to each bucket or bit of the exponential data structure. In some embodiments, the EBF can be a bit string, where each position in the bit string serves as a register (or bucket) that can be set to zero or one. When initializing or generating the EBF, the data processing system can initialize the Bloom filter buckets to an initialization value (e.g., zero, etc.). For example, the probability assigned to each bucket of the EBF sketch can correspond to a value of an exponential distribution, as shown in the “compute_register_probs” pseudocode function described above. By adjusting the decay rate of the exponential distribution of the EBF sketch, different probabilities (e.g., of bit filling, etc.) can be determined. To add a data record to the EBF, the data processing system can compute a hash value of the data record (e.g., using a hash function on the data structure and its contents, etc.). The hash value can be converted to a number in the range [0, 1] (e.g., by dividing by the range of the output of the hash function, etc.). Because each bucket in the EBF corresponds to a range of probabilities in the domain [0, 1], the data processing system can identify which range of probabilities the converted hash value falls into. The bucket associated with the corresponding range can then be set to a set value (e.g., 1). This process can be repeated until all desired data records are added to the EBF sketch. Each position (e.g., bucket, register, etc.) in the Bloom filter sketch or EBF sketch can be identified by a corresponding index value.

[0224] The data processing system can select the first data structure’s first k register (step 306). To populate the second data structure with index values of the first data structure’s registers that are set to a set value, the data processing system can iterate through each register of the first data structure based on the counter register k The data processing system can select the first data structure’s first k register (step 306). To populate the second data structure with index values of the first data structure’s registers that are set to a set value, the data processing system can iterate through each register of the first data structure based on the counter register k The data processing system can select the first data structure’s first k register (step 306). To populate the second data structure with index values of the first data structure’s registers that are set to a set value, the data processing system can iterate through each register of the first data structure based on the counter register k = The data processing system can select the first data structure’s first The data processing system can select the first data structure’s first

[0225] The data processing system can determine whether the register is equal to a predetermined value (step 308). To do so, the data processing system can compare the value of the selected register to a set value (e.g., 1, etc.) to determine whether the index value of the register should be added to the second data structure. If the value of the register is equal to the set value, the data processing system can perform (step 310). If the value of the register is not equal to the set value, the data processing system can perform (step 312).

[0226] The data processing system can generate a second data structure (step 310). The data processing system can generate a second data structure, such as a VoC sketch, that represents the index values of the subset of set registers in the first data structure. The VoC sketch can include a counter register that can be initialized to an unset value (e.g., zero) and can be incremented by the data processing system as the VoC sketch is populated. To initialize the VoC sketch, the data processing system can generate a VoC sketch having a predetermined (e.g., retrieved from a set or from another computing device, etc.) length (e.g., number of registers or buckets) with each register initially set to an unset value (e.g., zero). To populate the VoC data structure, the data processing system can take an input value, such as the index value of the selected register, and perform a hash function on the selected index value to generate a hash value. The data processing system can then perform a modulo operation on the hash value and the number of registers in the VoC sketch. The result of the modulo operation can have a value equal to the index value of one of the registers in the VoC sketch. The data processing system can increment the register associated with the index value equal to the modulo operation, thereby populating the VoC sketch with the selected index value.

[0227] The data processing system can determine whether the counter register k is equal to the number of registers in the first data structure n (step 312). To determine whether all of the index values in the first data structure have been considered by the data processing system, the data processing system can compare the counter register used to select each register in the first data structure to the total number of registers in the first data structure n If the counter register k is not equal to (e.g., less than) the total number of registers in the first data structure n , the data processing system can perform (step 314). If the counter register k is equal to (e.g., equal to or greater than) the total number of registers in the first data structure n , the data processing system can perform (step 316).

[0228] The data processing system can increment the counter register k(Step 314). To account for each register in the first data structure for addition to the second data structure, the data processing system can increment a counter register k to indicate the number of registers in the first data structure that have been accounted for addition to the second data structure. In some embodiments, the data processing system can set the counter register k to a memory address value (e.g., a location in computer memory) of a next location in memory of a next register in the first data structure. If this is the first iteration of the loop, the data processing system can initialize the counter register k to an initial value of, for example, zero, before incrementing the counter register k . After increasing the value of the counter register

[0229] The data processing system can store the second data structure (Step 316). The data processing system can store the second data structure (e.g., VoC sketch) in memory of the data processing system, which represents the selected subset of registers of the first data structure (e.g., Bloom filter or exponential Bloom filter, etc.). In some embodiments, the data processing system can store the value in association with an identifier of the second data structure (e.g., VoC sketch, etc.) in a database (e.g., database 115, etc.). In some embodiments, the identifier of the second data structure can correspond to the set of data records added to the first data structure (e.g., identifier 170 or attribute 175). In some embodiments, the data processing system can transmit the second data structure to another computing device, such as a content provider (e.g., content provider 160, etc.) that requested the second data structure, or an external computing device that can aggregate sketches in a multi-party computation (MPC) protocol. In some embodiments, the data processing system can receive a request for a VoC sketch that represents a meta-estimate of sketches generated from identifiers associated with a particular attribute.

[0230] The data processing system can determine a union value (step 318). To compute a union value for two VoC sketches (e.g., a cardinality of a union of the sets of index values used to generate the two VoC sketches, etc.), the data processing system can retrieve two VoC sketches generated from the same data record universe (e.g., but can contain different data records) using the same parameters. To compute the union value, the data processing system can first determine an intersection value for the two VoC sketches (e.g., a cardinality of an intersection of the sets of index values used to generate the two VoC sketches, etc.). To determine the intersection value, the data processing system can determine an average register value (e.g., an average of the register values) in the first VoC sketch and an average register value for the second VoC sketch. The average register value for the first VoC sketch can be subtracted from each register value in the first VoC sketch to generate a first normalized VoC sketch, and the average register value for the second VoC sketch can be subtracted from each register value in the second VoC sketch to generate a second normalized VoC sketch. Next, a dot product can be computed between the first normalized VoC sketch and the second normalized VoC sketch by multiplying each corresponding register in the two normalized sketches and summing to compute a sum of the resulting products. In some embodiments, the data processing system can multiply the resulting sum by an adjustment value equal to a number of positions in the first vector divided by a difference of the number of positions in the first vector minus one. In some embodiments, no adjustment value is used. The summed value, or the summed value multiplied by the adjustment value (if used), can be the intersection value for the two VoC sketches.

[0231] The data processing system can then use the intersection value to determine a cardinality of a union of the index values used to generate the two sketches (e.g., a number of unique index values that contributed to the two VoC sketches, etc.). To do so, the data processing system can estimate a number of index values used to generate the first sketch by summing the register values of the VoC sketch. The data processing system can estimate a number of index values used to generate the second sketch in the same manner. To compute the union value, the data processing system can simply add the estimated number of index values for the first VoC sketch to the estimated number of index values for the second VoC sketch, and subtract the intersection value. The resulting value is the union value, which represents a number of unique index values (or set registers in the original sketches) in the two VoC sketches.

[0232] Figure 4 A general architecture of an illustrative computer system 400 that can be used to implement any of the computer systems discussed herein is shown in accordance with some embodiments. The computer system 400 can be used to provide information for display via the network 110. Figure 4The computer system 400 includes one or more processors 420, which are communicatively coupled to a memory 425, one or more communication interfaces 405, and one or more output devices 410 (e.g., one or more display units) and one or more input devices 415. The processor 420 may be included in any computing device described herein.

[0233] exist Figure 4 In the computer system 400, memory 425 may include any computer-readable storage medium and may store computer instructions, such as processor execution instructions for implementing the various functions described herein for the corresponding system, and any data associated with, generated therefrom, or received via a communication interface or input device (if present). See again Figure 4 The system 400, computer system 400 may include memory 425 to store any information, variables, vectors, data structures, or other computer-readable information as described herein. Figure 4 The processor 420 shown can be used to execute instructions stored in memory 425, and in doing so, can also read from memory various information generated according to the execution of instructions or written to memory.

[0234] Figure 4 The processor 420 of the computer system 400 shown can also be communicatively coupled to or control a communication interface 405 to transmit or receive various information according to instructions. For example, the communication interface 405 can be coupled to a wired or wireless network, a bus, or other communication device, and thus allow the computer system 400 to transmit information to or receive information from other devices (e.g., other computer systems). Although in Figure 4 While not explicitly shown in the system, one or more communication interfaces facilitate the flow of information between components of system 400. In some embodiments, the communication interface may be configured (e.g., via various hardware or software components) to provide a website as an access portal to at least some aspects of computer system 400. Examples of communication interface 405 include user interfaces (e.g., web pages) through which users can communicate with computer system 400.

[0235] For example, it can provide Figure 4 The output device 410 of the computer system 400 shown herein allows for viewing or otherwise perceiving various information in conjunction with the execution of instructions. For example, an input device 415 may be provided to allow a user to manually adjust, make selections, input data, or interact with the processor in any of a variety of ways during instruction execution. This document further provides additional information relating to general computer system architectures that can be used in the various systems discussed herein.

[0236] Various experimental results related to Bloom filter and meta-estimation with count vectors are described below. It should be understood that this is example data only and should be interpreted as limiting any aspect of the systems and methods described herein.

[0237] Referring now to Figure 5 , depicted is a graph showing example experimental data including error and bias as Bloom filter size increases. The parameters and functions used to generate the graph in Figure 5 are presented below in the following pseudocode, which can be similar to Python code.

[0238] num_hash = 1

[0239] num_trials = 500

[0240] size_bounds = (2048 2, 2048 100)

[0241] bf_size_range = list(range( size_bounds, 2048 2))

[0242] voc_size = 2048

[0243] bf = BloomFilterOps(num_hash)

[0244] methods = [('Sharp Scaling', norm(bf.bf_to_voc_sharp_scaling(muted=False))),

[0245] # ('Muted Scaling', norm(bf.bf_to_voc_sharp_scaling(muted=True))),

[0246] ('Linear Scaling', norm(bf.bf_to_voc_scaling)),

[0247] # ('Sharp Meta', folded_bloom_filter(bf.bf_to_voc_sharp_scaling(muted=False))),

[0248] ('Meta', folded_bloom_filter(bf.bf_to_voc_meta)),

[0249] # ('Modded Transform', folded_bloom_filter(bf.bf_to_voc_modded_transform)), ]

[0251] true_intersection_size = 1000

[0252] true_set_size = 4096

[0253] run_experiment("Bloom Filter Size", num_trials, bf_size_range,methods, [bf],

[0254] [true_intersection_size], [true_set_size], bf_size_range, [voc_size])

[0255] As shown in Figure 5 , as the size of the Bloom filter (number of registers) increases, the meta estimate generally performs better than other techniques for Bloom filter to VoC conversion in terms of error and bias.

[0256] Referring now to Figure 6 , a graph depicting example experimental data including error and bias as the size of the VoC sketch increases. The parameters and functions used to generate the graph in Figure 6 are presented below in the following pseudocode, which can be similar to Python code.

[0257] num_hash = 1

[0258] num_trials = 1000

[0259] size_bounds = (2048 4, 2048 50)

[0260] voc_size_range = list(range( size_bounds, 2048 2))

[0261] bf_size = 2048 100

[0262] bf = BloomFilterOps(num_hash)

[0263] methods = [

[0264] ('Sharp Scaling', norm(bf.bf_to_voc_sharp_scaling(muted=False))),

[0265] ('Muted Scaling', norm(bf.bf_to_voc_sharp_scaling(muted=True))),

[0266] ('Linear Scaling', norm(bf.bf_to_voc_scaling)),

[0267] ('Meta', meta_voc) ]

[0269] true_set_size = 4096

[0270] true_intersection_size = 1000

[0271] run_experiment("VoC Size", num_trials, voc_size_range, methods, [bf],

[0272] [true_intersection_size], [true_set_size], [bf_size], voc_size_range)

[0273] As shown in Figure 6 , as the size of the VoC sketch (number of registers) increases, the meta estimate generally outperforms the other techniques for Bloom filter to VoC conversion in terms of error and bias.

[0274] Reference is now made to Figure 7 , which depicts a graph showing example experimental data including error and bias as the size of the Bloom filter increases for an exponential Bloom filter implementation.

[0275] num_trials = 250

[0276] size_bounds = (2048 15, 2048 100)

[0277] bf_size_range = list(range( size_bounds, 2048 2))

[0278] voc_size = 2048

[0279] ebf = ExpBloomFilterOps(10)

[0280] bf = BloomFilterOps(1)

[0281] methods = [('Sharp Scaling',

[0282] exp_bloom_filter(ebf.ebf_to_voc_sharp_scaling(muted=False), meta=False)),

[0283] ('Linear Scaling',

[0284] exp_bloom_filter(ebf.ebf_to_voc_modded_transform, meta=False)),

[0285] ('Meta',

[0286] exp_bloom_filter(bf.bf_to_voc_meta, meta=True)) ]

[0288] true_intersection_size = 1000

[0289] true_set_size = 4096

[0290] run_experiment("Bloom Filter Size", num_trials, bf_size_range,methods, [ebf],

[0291] [true_intersection_size], [true_set_size], bf_size_range, [voc_size])

[0292] As Figure 7 As shown in the middle, as the size of the bloom filter sketch (number of registers) increases, the meta-estimate of the exponential bloom filter is competitive in terms of error and bias compared to other techniques for exponential bloom filter to VoC conversion. However, the meta-estimate is more computationally efficient than the other implementations, such that the difference in relative error is negligible for large datasets.

[0293] Embodiments of the subject matter and operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more components of computer program instructions encoded on a computer storage medium for execution by, or to control the operation of, data processing apparatus. The program instructions can be encoded on a propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0294] The features disclosed herein can be implemented on a smart television module (or connected television module, hybrid television module, etc.), which can include a processing module configured to integrate Internet connectivity with more traditional television programming sources (e.g., via cable, satellite, wireless, or other signal reception). The smart television module can be physically integrated into a television set or can include a separate device such as a set-top box, Blu-ray or other digital media player, game console, hotel television system, and other companion device. The smart television module can be configured to allow viewers to search and find videos, movies, photos and other content of their choosing, whether on the Internet, local cable TV channels, satellite TV channels, or stored on a local hard drive. A set-top box (STB) or set-top unit (STU) can include an information appliance device that can contain a tuner and connect to a television, and an external source of signals, turning the signals into content which is then displayed on the television or other display device. The smart television module can be configured to provide a home screen or top level screen including icons for a number of different applications, such as a web browser and a number of streaming media services, connected cable or satellite media sources, other network "channels", etc. The smart television module can be further configured to provide an electronic program guide to the user. Companion applications for the smart television module can operate on a mobile computing device to provide additional information to a user about available programming, to allow the user to control the smart television module, etc. In alternative embodiments, the features can be implemented on a laptop computer or other personal computer, a smart phone, other mobile telephone, a handheld computer, a tablet PC, or other computing device.

[0295] The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0296] The terms "data processing apparatus," "data processing system," "client device," "computing platform," "computing device," or "device" encompass all kinds of apparatus, devices, and machines for processing data, including by way of example, a programmable processor, a computer, a system on a chip, or multiple chips, or combinations of the foregoing. The apparatus can include special purpose logic, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer programs in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of the foregoing, where a combination can include one or more of the foregoing combined with hardware. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing infrastructures, and grid computing infrastructures.

[0297] A computer program, which can also be referred to or referred to as a program, software, a software application, an app, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0298] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0299] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The elements of a computer include a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), for example. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0300] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), plasma, or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can include any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending web pages to a web browser on a user’s client device in response to requests received from the web browser.

[0301] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0302] The computing system of FIG. 1 can include clients and servers. For example, data processing system 105 can include one or more servers in one or more data centers or server farms. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) can be received from the client device at the server (e.g., as a result of the client device interacting with the server or any other computing device).

[0303] Although this specification contains many specifics, these should not be construed as limiting the scope of any invention or application of what can be claimed, but as merely describing features that can be incorporated into a specific embodiment of the systems and methods described herein. The scope of the specification is not limited to the embodiments described herein, but rather includes all alternatives consistent with the principles of the systems and methods described herein. Some features described in the context of separate embodiments can be implemented in a combined embodiment. Conversely, various features described in the context of a combined embodiment can also be implemented in separate embodiments or in any suitable sub-combination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a sub-combination or a variation of a sub-combination.

[0304] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring such order nor that all illustrated operations be performed, to achieve desirable results. In some cases, the recited order is an example order in which the actions can be performed in other orders, and still achieve desirable results. Additionally, the processes depicted in the drawings do not necessarily require the particular order shown, or sequential order, to achieve the desired results.

[0305] In some cases, multitasking and parallel processing can be advantageous. Additionally, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated in a single software product or packaged into multiple software products. For example, the data processing system 105 can be a single module, a logical device with one or more processing modules, one or more servers, or a portion of a search engine.

[0306] Having now described some illustrative implementations, it is apparent that the foregoing is illustrative and not limiting, having been presented by way of example. In particular, although many of the examples presented herein involve specific combinations of method acts or system elements, those acts and those elements can be combined in other ways to accomplish the same objectives. Acts, elements and features discussed only in connection with one implementation are not intended to be excluded from combination with other implementations or elements from other implementations.

[0307] The phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," "having," "containing," "involving," "characterized by," "characterized in that," "characterized by," and variations thereof herein, is meant to encompass the item listed thereafter, equivalents thereof, and additional items not specifically listed. In one implementation, the systems and methods described herein consist of one, each combination of more than one, or all of the described elements, acts, or components.

[0308] Any reference to an implementation or element or act of a system and method herein using a singular convolution can also encompass implementations including a plurality of such elements, and any plural reference to an implementation or element or act herein can also encompass implementations including only a single element. References in the singular or plural form are not intended to limit the presently disclosed systems or methods, their components, acts, or elements, to the singular or plural configuration. Any reference to an action or element based on any information, action, or element can encompass implementations where the action or element is based at least in part on any information, action, or element.

[0309] Any embodiment disclosed herein can be combined with any other embodiment, and the references to "an embodiment," "some embodiments," "other embodiments," "various embodiments," "one embodiment," etc. do not necessarily refer to the same embodiment and are not necessarily mutually exclusive, and the various scope of embodiments can include modifications, combinations, and / or permutations of the embodiments described herein. These terms do not require a granting reference to be the prior art. Any embodiment can include any of the features, aspects and / or features described in any of the embodiments disclosed herein.

[0310] References to "or" can be construed as inclusive so that any term can be

[0311] Where a technical feature is followed by a reference sign in the drawings, the sole purpose of the reference sign is to increase the intelligibility of the drawings, detailed description and claims. Therefore, neither the presence nor the absence of a reference sign has any limiting effect on the scope of any claim element.

[0312] The systems and methods described herein can be embodied in other specific forms without departing from the characteristics thereof. While the examples provided can be useful for estimating the metadata of a data structure representing an identifier, the systems and methods described herein can be applied to other environments. The foregoing embodiments are illustrative rather than limiting, and the described systems and methods are not limited to the embodiments disclosed herein. The scope of the systems and methods described herein can be indicated by the appended claims, rather than by the foregoing description, and changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein.

Claims

1. A system for meta-estimation of data structures, comprising: One or more processors; as well as One or more memory devices storing computer-readable instructions, which, when executed, cause the one or more processors to perform operations, including: Retrieve one or more generation parameters corresponding to a first initial sketch comprising a plurality of first initial sketch registers, wherein a subset of the plurality of first initial sketch registers is populated based on the one or more generation parameters and at least in part based on a hash of a plurality of data records; Retrieve a first-ary estimation sketch comprising a plurality of first-ary estimation sketch registers, the plurality of first-ary estimation sketch registers being populated at least in part based on a hash of the index values ​​of each of the subsets of the plurality of first initial sketch registers; and Using the one or more generation parameters, the first-variant estimation sketch and the second-variant estimation sketch are used to determine one or more set properties.

2. The system according to claim 1, wherein, The second-ary estimation sketch includes a plurality of second-ary estimation sketch registers, which are filled at least in part based on a hash of the index values ​​of each of a subset of the plurality of second-initial sketch registers of the second initial sketch.

3. The system according to claim 2, wherein, The one or more set properties include a cardinality estimate of the union of the first initial sketch and the second initial sketch.

4. The system according to claim 1, wherein, Determining the one or more set properties includes: determining the intersection of the first-dimensional estimation sketch and the second-dimensional estimation sketch.

5. The system according to claim 1, wherein, Determining the one or more set properties includes: determining the union of the first meta-estimation sketch and the second meta-estimation sketch.

6. The system according to claim 1, wherein, The first initial sketch is a Bloom filter sketch.

7. The system according to claim 1, wherein, The first initial sketch is an exponential Bloom filter sketch.

8. The system according to claim 1, wherein, The multiple data records correspond to multiple identifiers associated with one or more attributes.

9. The system according to any one of claims 1-8, wherein, The operation further includes: The first meta-estimation sketch is generated by the following operation: for each of the subsets of the plurality of first initial sketch registers, the corresponding index value among the index values. The corresponding first-ary estimation sketch index is determined based on hashing the corresponding index values; and Increment one of the plurality of first-ary estimation sketch registers associated with the corresponding first-ary estimation sketch index.

10. A system for meta-estimation of data structures, comprising: One or more processors; as well as One or more memory devices storing computer-readable instructions, which, when executed, cause the one or more processors to perform operations, including: Generate a first initial sketch comprising a plurality of first initial sketch registers, wherein a subset of the plurality of first initial sketch registers is populated based on one or more generation parameters and at least in part on a hash of a plurality of data records; and A first-ary estimation sketch is generated, comprising a plurality of first-ary estimation sketch registers, the plurality of first-ary estimation sketch registers being filled at least in part based on a hash of the index values ​​of each of the subsets of the plurality of first initial sketch registers.

11. The system according to claim 10, wherein, The first initial sketch is a Bloom filter sketch.

12. The system according to claim 10, wherein, The multiple data records correspond to multiple identifiers associated with one or more attributes.

13. The system of claim 10, wherein generating the first meta-estimation sketch comprises, for each of the subsets of the plurality of first initial sketch registers, a corresponding index value among the index values: The corresponding first-ary estimation sketch index is determined based on hashing the corresponding index values; and Increment one of the plurality of first-ary estimation sketch registers associated with the corresponding first-ary estimation sketch index.

14. The system according to claim 12, wherein, The operation further includes: Receive a meta-estimation sketch request from the computing device; and In response to the meta-estimation sketch request, the first meta-estimation sketch is transmitted to the computing device.

15. The system according to claim 14, wherein, The meta-estimation sketch request identifies at least one parameter of the identifier or attribute to be represented in the first initial sketch.

16. The system according to claim 14, wherein, The meta-estimation sketch request includes one or more generation parameters of the first initial sketch.

17. The system according to any one of claims 10 to 16, wherein, The one or more generation parameters include at least one parameter selected from the type of hash function used, the number of hash functions used, and the number of registers.

18. A method for meta-estimation of data structures, comprising: A data processing system comprising one or more processors and memory maintains multiple data records, the multiple data records including multiple identifiers and one or more attributes; The data processing system generates a first data structure representing the plurality of data records, the first data structure having a plurality of first registers, wherein at least one of the plurality of first registers is filled based on a hash of each of the plurality of identifiers; The data processing system identifies a subset of the plurality of first registers that are equal to a predetermined value, each of the subsets of the plurality of first registers being identified by an index value; The data processing system generates a second data structure representing a subset of the plurality of first registers, the second data structure having a plurality of second registers, wherein at least one of the plurality of second registers is filled based on a hash of the index values ​​of each of the subsets of the plurality of first registers; The data processing system stores the second data structure in the memory, the second data structure representing the subset of the plurality of first registers.

19. The method of claim 18, further comprising: The data processing system retrieves a third data structure having a plurality of third registers, each of the plurality of third registers corresponding to a corresponding second register among the plurality of second registers of the second data structure; as well as The data processing system uses the second data structure and the third data structure to determine the intersection value of the information represented by the second data structure and the information represented by the third data structure.

20. The method of claim 19, further comprising: The data processing system uses the intersection value to determine the union value of the information represented by the second data structure and the information represented by the third data structure.

21. The method according to claim 18, wherein, The first data structure is a Bloom filter sketch.

22. The method according to claim 18, wherein, The first data structure is an exponential Bloom filter sketch.

23. The method according to claim 18, wherein, The generation of the second data structure by the data processing system includes, for each of the subsets of the plurality of first registers, a corresponding index value: The data processing system determines the corresponding second data structure index based on hashing the corresponding index value; as well as The data processing system increments one of the plurality of second registers associated with the corresponding second data structure index.

24. The method according to any one of claims 18 to 23, further comprising: The data processing system receives a meta-estimation sketch request from the computing device; as well as The data processing system transmits the second data structure to the computing device in response to the meta-estimation sketch request.

25. The method according to claim 24, wherein, The meta-estimation sketch request identifies at least one parameter of the identifier or attribute to be represented in the first data structure.

26. The method according to claim 24, wherein, The meta-estimation sketch request includes one or more generation parameters of the first data structure.

27. The method according to claim 26, wherein, The one or more generation parameters include at least one parameter selected from the type of hash function used, the number of hash functions used, and the number of registers.

Images