Semantic Layer for Document-Oriented Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Document-oriented databases, such as those using JSON/BSON, face challenges in representing and processing unstructured or semi-structured data, particularly in systems designed for traditional relational databases, where data types can vary widely and schema complexity makes data understanding and management difficult.
Innovation Solution
A mechanism and process are introduced to automatically determine data types for JSON objects/documents by sampling data and using a hierarchical representation, enabling efficient storage, management, and processing of unstructured data in a managed hierarchical manner, compatible with existing systems like SAP BusinessObjects, without altering their behavior significantly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If document-oriented databases use flexible data structures to store unstructured or semi-structured data, then data storage flexibility and adaptability are improved, but data type identification and schema management become more difficult
Solution Approach 1:
The patent introduces a semantic layer as an intermediary between the document-oriented database and the application layer. This semantic layer automatically infers data types, relationships, and schemas from the unstructured or semi-structured data, translating it into a standardized format that is easy to manage and query. The semantic layer acts as a mediator that resolves the contradiction by maintaining storage flexibility while providing structured data understanding.
Solution Approach 2:
The system performs preliminary processing by automatically inferring data types and creating semantic representations before the data needs to be queried or processed by applications. This preliminary action of data type inference and schema generation eliminates the need for manual data type identification, resolving the contradiction between flexible storage and easy data management.
2Ease of operation
If relational databases use strongly typed data structures with fixed schemas, then data management and querying are simplified, but adaptability to varied data formats is reduced
Solution Approach 1:
The patent implements a dynamic schema inference mechanism that automatically adapts to different data formats without requiring predefined schemas. The system dynamically creates data models based on the actual data structure, allowing the database to maintain simplicity in data management while achieving high adaptability to varied data formats. This dynamic approach resolves the contradiction by making the schema flexible rather than fixed.
Solution Approach 2:
The semantic layer provides a universal interface that can handle multiple data formats and structures through a single standardized representation. This multi-functionality allows the system to manage diverse data types uniformly, simplifying operations while maintaining adaptability. The semantic layer serves as a universal translator that converts various data formats into a common representation language.
3Productivity
If NoSQL databases are used to support SQL-like query languages for big data applications, then query capability is improved, but compatibility with traditional relational database systems is reduced
Solution Approach 1:
The semantic layer serves as an intermediary that translates between NoSQL query languages and traditional SQL dialects. This allows applications written for relational databases to query NoSQL document stores without requiring changes to the application code. The semantic layer maintains compatibility with traditional RDBMS while enabling the productivity benefits of NoSQL query processing, resolving the contradiction between query efficiency and system compatibility.
Data Source
AI summary
A system and a method, the method including receiving a set of documents, the set including at least one instance of an unstructured document; parsing the set of documents to obtain information regarding the set of documents; determining a map of a map for the set of documents based on the information obtained by the parsing; generating a hierarchical semantic representation of the set of documents based on the map of the map; and storing the hierarchical semantic representation of the set of documents.