Knowledge-Driven Semantic Layer Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of a semantic layer in enterprise computing systems is resource-intensive and often requires significant human intervention, and the generated layer may inadequately describe the characteristics and dependencies of attributes.
Innovation Solution
A 'knowledge vault' is used to provide dependencies and semantic information of entities identified in the data source, facilitating the efficient generation of a semantic layer, which can be modified by users and updated to generate another semantic layer based on different data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional methods are used to generate a semantic layer, then the semantic layer can be created, but the process is resource-intensive and requires significant human intervention
Solution Approach 1:
The system performs preliminary actions by automatically identifying physical entities, their characteristics, and dependencies before semantic layer generation. This pre-processing includes crawling data sources to extract entity information, determining relationships between entities, and preparing structured data that can be directly used in semantic layer creation, thereby reducing both resource consumption and human intervention requirements
Solution Approach 2:
The system enables self-service by automatically generating semantic layers without requiring significant human intervention. The automated entity identification, relationship determination, and semantic layer construction processes allow the system to serve itself in creating and maintaining semantic layers, eliminating the need for manual configuration and reducing operational resource requirements
2Loss of information
If traditional methods are used to generate a semantic layer, then the semantic layer can be created, but it may inadequately describe characteristics and dependencies of the attributes
Solution Approach 1:
The system segments the semantic layer into distinct components: physical entities, their characteristics, dependencies, and relationships. Each entity is independently identified and described with its specific attributes, allowing for comprehensive documentation of characteristics and dependencies without creating an overly complex monolithic structure
Solution Approach 2:
The system uses an intermediary approach by introducing a structured representation layer between the physical data sources and the semantic layer. This intermediary structure systematically captures and preserves entity characteristics and dependencies, ensuring that no information is lost during the transformation process while maintaining organizational clarity
Data Source
AI summary
A system includes acquisition of data from a first data source, identification of logical entities of a first data model of the first data source based on the data of the first data source, identification, for one or more of the logical entities of the first data model, of a respective knowledge vault entity described by first metadata, and generation of second metadata describing a plurality of semantic layer entities, the second metadata describing a semantic layer entity comprising at least one semantic attribute of a knowledge vault entity corresponding to the semantic layer entity.


