Utility-Based Inquiry Selection in Streaming Data Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data collection methods in the digital marketing industry lack the ability to collect specifically targeted data, leading to less accurate results in determining user preferences and utility, particularly in cases where traditional methods cannot determine a user's health conditions or treatment preferences.
Innovation Solution
An intelligent decision engine with algorithms and models is employed in a streaming data pipeline to collect historically non-observable user data, such as preferences and interests, using a data discovery platform that dynamically serves inquiries to users, allowing for the delivery of personalized user experiences like advertisements based on calculated utility and user interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional data collection methods are used, then data collection is simple and straightforward, but the ability to collect specifically targeted data and determine user utility is insufficient
Solution Approach 1:
The patent segments the data collection process into multiple specialized components: event processing services that handle specific data types, utility determination services that calculate user utility for different data points, and inquiry selection services that choose which questions to ask next. This segmentation enables precise targeted data collection while managing system complexity through modular design.
Solution Approach 2:
The patent introduces intermediary services between the user and the data collection system. Event processing services act as intermediaries that translate user actions into structured events, while utility determination services mediate between collected events and the overall user profile, calculating which additional information would be most valuable to gather.
2Measurement precision
If comprehensive user data is collected to improve targeting accuracy, then user preference determination improves, but user tolerance for information provision is exceeded
Solution Approach 1:
The patent applies partial action by using utility determination to identify and collect only the most valuable data points rather than comprehensively gathering all possible user information. The system determines which specific user attributes or preferences would provide the greatest utility for advertising targeting and focuses collection efforts there, asking fewer questions while achieving better targeting accuracy.
Solution Approach 2:
The patent makes the data collection process dynamic by continuously recalculating utility based on events already observed. The inquiry selection service dynamically adjusts which questions to ask next based on the user's evolving profile and the marginal utility of potential data points, adapting to user tolerance by stopping collection when additional questions would provide diminishing returns.
3Productivity
If real-time user data processing is implemented to deliver personalized experiences, then user engagement improves, but processing time and computational resources increase
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing utility determination logic and user profile structures before actual data collection begins. The system pre-establishes the framework for processing events and determining utility, allowing real-time operations to simply query pre-computed values rather than performing complex calculations during user interactions, thus reducing processing time while maintaining personalization.
Data Source
AI summary
Systems and methods disclosed herein relate to utility based selection of inquiries in a streaming data pipeline. An indication of user identification data and a data source may be received by a server in communication with a distributed streaming data pipeline. This pipeline may be in communication with asynchronous event processing services. Those event processing services may determine an inquiry rule that includes an inquiry threshold. This rule may be based on the data source and retrieval of information related to that data source from memory in a database server. The rule may prompt a comparison of an inquiry count to an inquiry threshold. If the count is below the threshold, an inquiry may be sent to a user based in part upon expected utility that is itself based upon several indications of expected utility calculated by an event processing service.