Hyperloglog Custom Audience Segment Cardinality Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques require hours or days to generate a custom audience segment by combining multiple audience segments while excluding overlapping entries, which is inefficient and inaccurate, especially when dealing with large audience segments.
Innovation Solution
A system that utilizes a non-transitory computer-readable medium with instructions to process circuitry to receive a request for a custom segment, generate a second Boolean expression based on the first Boolean expression and segments, and determine the number of distinct elements in the custom segment using Hyperloglog data structures and algorithms, enabling rapid and accurate estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional techniques are used to generate custom audience segments by combining multiple segments and excluding overlaps, then accurate distinct element counting is achieved, but processing time increases to hours or days
Solution Approach 1:
The patent changes the parameter of how segment combinations are evaluated by transforming Boolean expressions into alternative forms that enable efficient cardinality estimation. Specifically, it converts complex Boolean expressions involving intersections and exclusions into equivalent expressions that can be evaluated using Hyperloglog algorithms, thereby maintaining accuracy while dramatically reducing processing time from hours/days to milliseconds.
Solution Approach 2:
The patent introduces Hyperloglog data structures as an intermediary mechanism between the Boolean expression evaluation and the final distinct element count. These data structures enable approximate counting of distinct elements in large datasets without requiring full materialization of segment intersections, thus resolving the time-accuracy contradiction by providing a efficient intermediate representation.
2Measurement precision
If exact distinct element counting is performed for large audience segments, then accurate measurements are obtained, but computational complexity increases significantly
Solution Approach 1:
The patent changes the computational approach by transforming the parameter evaluation method from exact set operations to probabilistic estimation using Hyperloglog. This allows the system to handle large audience segments with millions of viewers by estimating distinct element counts through algorithmic approximation rather than exhaustive computation, thereby reducing computational complexity while maintaining sufficient accuracy for advertising decisions.
Solution Approach 2:
The patent substitutes the mechanical system of exact set operations and Boolean evaluations with a more efficient algorithmic approach using Hyperloglog data structures. This replacement eliminates the need for materializing and comparing large datasets directly, instead using hashing and probabilistic counting mechanisms that are computationally lighter and scale better to large audiences.
3Reliability
If Boolean expressions are directly evaluated for custom segments, then exact segment definitions are maintained, but processing efficiency decreases to hours or days
Solution Approach 1:
The patent applies preliminary action by pre-processing and transforming Boolean expressions into equivalent forms that are optimized for efficient evaluation. Before performing the actual segment combination and counting operations, the system rewrites the Boolean expressions into alternative forms that expose opportunities for efficient computation using Hyperloglog, thereby maintaining segment definition accuracy while enabling rapid processing.
Solution Approach 2:
The patent changes the evaluation parameter from direct Boolean expression computation to transformed expression evaluation that leverages Hyperloglog algorithms. This parameter change allows the system to maintain the semantic meaning and accuracy of the original segment definitions while executing the evaluation in milliseconds rather than hours, thus resolving the reliability-productivity contradiction.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A non-transitory, computer-readable medium includes instructions, that, when executed, cause processing circuitry to receive, from an electronic device, a request regarding a custom segment defined by a first Boolean expression of a plurality of segments having elements corresponding to, or representative of, one or more electronic devices and/or one or more users of the one or more electronic devices. The instructions, when executed, also cause the processing circuitry to generate a different, second Boolean expression based on the first Boolean expression and the plurality of segments. Additionally, the instructions, when executed, cause the processing circuitry to determine a number of distinct elements of the custom segment at least partially utilizing the second Boolean expression and a plurality of data structures corresponding to the plurality of segments. Furthermore, the instructions, when executed, cause the processing circuitry to send an indication of the number of distinct elements to the electronic device.