Household Link Persistence via Temporal Feedback Loops
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current household-based marketing systems face challenges in accurately identifying and maintaining representative households due to stale or incorrect personal identifiable information (PII) and transcription errors, which complicates the identification of changes in social and economic relationships, leading to poor computational efficiency and accuracy.
Innovation Solution
A computationally efficient contextual framework using a Hadoop cluster and LiveRamp's Entity Graph Resolution Repository (EGRR) that leverages both point-in-time and temporal data to construct and maintain accurate household representations, employing a champion-challenger methodology and feedback loops to ensure persistence and accuracy of household links and addresses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional PII data collection methods are used, then data coverage is achieved, but data accuracy deteriorates due to stale or incorrect information
Solution Approach 1:
The system performs preliminary actions by collecting temporal data points before final household identification. Multiple data collection points over time are gathered and processed to pre-validate PII information, ensuring accuracy is established before the final household representation is created. This allows the system to filter out stale or incorrect information proactively.
Solution Approach 2:
The system implements feedback loops that continuously validate and update household representations. By comparing temporal data points and using feedback mechanisms, the system can identify and correct stale or incorrect PII information, maintaining high data accuracy while preserving comprehensive data coverage through iterative validation.
2Measurement precision
If comprehensive PII validation is performed, then data accuracy improves, but computational time increases
Solution Approach 1:
The validation process is segmented into multiple stages: initial data collection, temporal pattern analysis, and final household identification. By dividing comprehensive validation into sequential segments, the system achieves thorough accuracy checking without requiring all validation steps to complete simultaneously, thereby reducing overall computational time while maintaining high data accuracy.
Solution Approach 2:
Preliminary filtering and validation steps are performed on temporal data points before final household representation is established. This preliminary action pre-processes and validates data in advance, reducing the computational burden during final household identification and decreasing overall processing time while preserving accuracy.
3Loss of information
If household representations are updated frequently, then data freshness improves, but system stability deteriorates
Solution Approach 1:
The system uses feedback loops to monitor changes in household representations and only updates when validated changes are detected. This feedback mechanism ensures data freshness by continuously monitoring for legitimate changes while maintaining stability by filtering out spurious or erroneous updates, allowing the system to adapt to real changes without unnecessary fluctuations.
Solution Approach 2:
The household representation system is designed to be dynamic, allowing updates when legitimate changes occur, while incorporating stability mechanisms through temporal data validation. The system adapts to genuine changes in household composition or PII information while resisting unnecessary or erroneous updates, achieving a balance between freshness and stability through controlled dynamics.
Data Source
AI summary
A system and method for the creation of household links (HHLs) associates each household with particular consumers associated with a consumer link (CL) and an address link (AL). The system and method utilize a feedback loop system to maintain persistence of HHLs over time and more accurately resolve HHLs. Top-down and bottom-up clustering methods are applied to the data, and the best results are taken to generate the final association of HHLs with particular ALs and CLs. By more accurately identifying the households associated with particular consumer data, the invention significantly reduces the storage requirements and time required for processing very large consumer data sets.