Real-Time Variable Predictiveness via Offline Mutual Information
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face computational and time-intensive challenges in dynamically computing mutual information for a large number of variables in real-time, limiting their ability to provide variable predictiveness in real-time, especially when handling extensive data sets and multiple concurrent queries.
Innovation Solution
Computing mutual information offline in parts for logical data sets, such as daily data, and using partial mutual information to generate conditional mutual information in real-time, reducing the number of data access attempts and calculations required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If mutual information is computed dynamically for each variable in real-time, then variable predictiveness can be accurately determined, but computational time and processor utilization increase significantly
Solution Approach 1:
The patent pre-computes mutual information values for all variables against the metric and stores them in a data structure before real-time queries arrive. This preliminary computation phase separates the intensive calculation work from the query response phase, allowing real-time queries to simply retrieve and aggregate pre-computed values rather than performing full mutual information calculations.
Solution Approach 2:
The patent divides the computation into two distinct segments: an offline pre-computation segment where mutual information is calculated and stored, and an online query segment where pre-computed values are retrieved and aggregated. This segmentation allows each segment to be optimized independently - the pre-computation can use all available resources while queries get fast responses.
2Adaptability or versatility
If mutual information is computed for thousands of variables concurrently, then comprehensive variable predictiveness is provided, but the number of data access attempts and logarithm computations becomes unmanageable
Solution Approach 1:
The patent performs all complex mutual information calculations including data access and logarithm computations in advance, storing results in a pre-computed data structure. This eliminates the need to perform these complex operations during concurrent query processing, making the system manageable even with thousands of variables.
Solution Approach 2:
The patent creates a copied representation of mutual information values in a pre-computed data structure that can be quickly accessed and aggregated. Instead of computing mutual information from raw data during queries, the system uses this copied pre-computed representation, dramatically reducing computation complexity during concurrent access.
3Speed
If real-time computation is performed for multiple concurrent queries, then responsive service is maintained, but processor utilization becomes excessive
Solution Approach 1:
The patent moves the computationally intensive mutual information calculations to a preliminary offline phase, storing results in a data structure optimized for quick access. During concurrent query processing, the system only needs to retrieve and aggregate pre-computed values, maintaining fast response times with minimal processor utilization.
4Productivity
If mutual information is computed offline in parts, then real-time computation efficiency is improved, but additional storage requirements are introduced
Solution Approach 1:
The patent pre-computes and stores mutual information values in a compact data structure that captures the essential information needed for query responses. This stored representation is much more space-efficient than storing raw data, achieving a favorable trade-off between storage requirements and real-time computation efficiency.
Data Source
AI summary
Embodiments of the present invention relate to efficiently computing variable predictiveness such that an indication of variable predictiveness can be provided in real time. In this regard, aspects of the present invention enable a user (e.g., digital marketer) to input a query and, in response, receive an indication of variable predictiveness. To efficiently compute variable predictiveness in response to a submitted user query, mutual information is computed offline and, thereafter, used to generate, in real time, conditional mutual information of variables for a specified date range. The concept of conditional mutual information can be utilized to represent variable predictiveness or otherwise indication variable predictiveness, such as to identify a set of variables that accurately predict a metric. Using such an approach effectively reduces a number of data access attempts and calculations performed in real time thereby reducing utilization of a processor(s).