Enterprise Data Model Schema for Incompatible Source Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems struggle to integrate and analyze data from different entities with incompatible data models, leading to incompatibility issues and the inability to view and analyze data collectively.
Innovation Solution
A data management system that automatically integrates data from different entities by defining a uniform enterprise data model schema, mapping and transforming data to a common format, and providing consolidated data views and collective analyses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data from different entities with incompatible data models is integrated, then data collectivity and analysis capability are improved, but data model compatibility and system complexity increase
Solution Approach 1:
The patent introduces an enterprise data model schema as an intermediary layer between diverse entity data models and the data lake. This schema acts as a standardized mediator that receives data from multiple entities with different data models, transforms them into a common format, and stores them in a unified structure. The intermediary schema layer isolates the complexity of data model incompatibility from the analysis systems, enabling data collectivity without proportionally increasing system complexity.
Solution Approach 2:
The patent segments the data integration architecture into distinct layers: entity-specific data models, enterprise data model schema (standardized layer), and data lake (storage layer). This segmentation allows each layer to operate independently with its own data model, reducing the complexity burden on the entire system while enabling comprehensive data integration across entities.
2Productivity
If a uniform enterprise data model schema is imposed on diverse entity data models, then data access efficiency is improved, but data model flexibility and transformation complexity increase
Solution Approach 1:
The patent performs preliminary data transformation by establishing an enterprise data model schema before data is stored in the data lake. Data from various entities is pre-transformed into the standardized schema format during the ingestion process, rather than transforming it at query time. This preliminary action ensures data access efficiency is improved while the transformation complexity is managed during the initial data loading phase rather than during operational queries.
3Loss of information
If data is consolidated into a centralized data lake, then data analysis capability is improved, but data security risks and compliance challenges increase
Solution Approach 1:
The patent applies local quality by allowing different security and compliance policies to be applied to different portions of data within the data lake based on their sensitivity and regulatory requirements. The enterprise data model schema enables identification and classification of data elements, allowing targeted security measures to be applied where needed rather than uniformly across all data, thus maintaining analysis capability while managing security risks proportionally.
Data Source
AI summary
Methods and systems are presented for collectively storing, managing, and analyzing data associated with different data sources. A data management system defines an enterprise data model schema based on different data model schemas associated with the different data sources. The data management system generates, for each data source, an enterprise data model instance based on the enterprise data model schema. Data is ingested from the different data sources, and then transformed and stored in a corresponding enterprise data model instance based on a mapping between a corresponding data model schema and the enterprise data model schema. Upon ingesting the data from the data sources, one or more consolidated data views are generated that combine at least portions of data from different enterprise data model instances. The data arranged according to the one or more consolidated data views is presented on a device and/or further analyzed to produce an analysis outcome.