Data Linkage Orchestration for Privacy-Safe Prediction Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating accurate computerized predictions is challenging when different data sets are stored in disparate data storage by disparate parties, each having their own data security and data privacy obligations, compounded by high uncertainty and complexity of interrelationships, especially in probabilistic linkages and spurious relationships.
Innovation Solution
A data processing orchestrator device or service securely interoperates with data sets to establish data linkages, using machine learning and reinforcement learning mechanisms, such as multi-armed bandit approaches, to improve the relevance of offers by analyzing user intent and purchase behavior, and dynamically adjusting offer parameters based on user interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data sets are stored in disparate data storage by disparate parties, then data security and data privacy are maintained, but generating accurate computerized predictions becomes challenging
Solution Approach 1:
The patent introduces an intermediary system that acts as a mediator between disparate data storage systems. This intermediary establishes probabilistic linkages between data sets without requiring direct access to the underlying data, thereby maintaining data security and privacy while enabling prediction generation. The intermediary translates and connects data representations across different storage systems through probabilistic relationships.
2Measurement precision
If holistic usage of multiple data sets is used to generate accurate predictions, then prediction accuracy improves, but the complexity of modelling interrelationships increases
Solution Approach 1:
The patent transforms the complex problem of modelling interrelationships by changing the parameters from direct causal relationships to probabilistic linkages. Instead of attempting to model complex causal chains between multiple data sets, the system represents relationships in terms of probabilities and statistical associations, significantly reducing modelling complexity while maintaining prediction accuracy.
Solution Approach 2:
The patent replaces the mechanical approach of directly linking data sets through explicit causal models with a statistical/probabilistic system. This substitution allows the system to handle holistic usage of multiple data sets without the computational burden of modelling all interrelationships explicitly, using probabilistic methods to capture the essential connections.
3Loss of information
If a large number of potential paths and spurious relationships are considered, then comprehensive analysis is achieved, but establishing the system becomes increasingly difficult
Solution Approach 1:
The patent applies partial action by focusing on establishing probabilistic linkages for the most significant relationships rather than attempting to model all potential paths and spurious relationships. This selective approach maintains comprehensive analysis of key data connections while avoiding the prohibitive complexity of exhaustively modelling every possible relationship path.
Data Source
AI summary
Systems and methods for establishing data linkages are described in various embodiments. A system architecture is described which provides a data processing orchestrator device or service which securely interoperates with data sets at various points in time associated with a set of interactions a user may have with computer systems. The data sets are obtained from different data repositories, and are combined together for analysis such that a first data set representing intents (e.g., web search/browse history) can be combined together with a second data set representing outcomes (e.g., purchase transaction history, web site shopping carts).