Extensible Data Mapping System for Unified Identity Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud-based data management systems face challenges in integrating multiple data sources into a cohesive data model, leading to difficulties in user identity resolution, data segmentation, and insights generation due to siloed data sources and limited support for custom or standard data models.
Innovation Solution
An extensible and scalable data mapping and modeling system that integrates multiple data sources into a shared data lake, allowing for customizable data models, preserving original data, and updating data mappings without re-ingesting source data, thereby supporting accurate user insights and flexible data analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data sources are maintained separately without integration, then data source independence is preserved, but user identity resolution and data segmentation become difficult
Solution Approach 1:
The patent introduces a data lake as an intermediary component that receives and stores data from multiple separate data sources. This data lake serves as a mediator that enables unified data access and user identity resolution without requiring direct integration between individual data sources, thus resolving the contradiction by maintaining source independence while enabling cohesive data operations.
Solution Approach 2:
The patent creates a universal data model that can represent multiple data sources and their relationships in a unified framework. This universal model enables various operations (user identity resolution, data segmentation, analytics) to be performed across integrated data from multiple sources without requiring source-specific implementations, thereby improving versatility while managing complexity through standardization.
2Adaptability or versatility
If standard data models are used, then implementation simplicity is improved, but functional flexibility and customization are limited
Solution Approach 1:
The patent implements a dynamic data model system where the data lake schema can be flexibly configured and extended based on specific needs. The system allows for both standard data models and custom data models to be defined and integrated, enabling the data model structure to adapt dynamically to different requirements while maintaining a consistent operational framework through the unified data lake interface.
3Manufacturing precision
If data mappings are updated, then data accuracy is improved, but processing time increases due to re-ingestion requirements
Solution Approach 1:
The patent implements a data lake that performs preliminary data ingestion and storage in a unified format from multiple sources. By pre-processing and standardizing data during the initial ingestion phase, the system eliminates the need for time-consuming re-ingestion operations when updates are required. Subsequent data mapping updates can be applied directly to the existing unified data structure, significantly reducing processing time while maintaining accuracy.
Data Source
AI summary
Methods, systems, and devices are described that support extensible data mapping. A data mapping server may receive an indication of a source schema for a data source and may receive a user input indicating creation of a custom data object to handle the source schema. The server may create the custom data object based on the user input. The data mapping server may automatically map one or more data fields for a source data object (e.g., based on the source schema) to one or more custom data fields for the custom data object (e.g., based on a custom schema). The server may import a set of data records stored at the data source and may store the data records in a database system according to the custom schema based on the data mapping. The stored custom data objects may be used for segmentation, activation, analysis, or some combination thereof.