Dynamic Data Guide for Hierarchical Schema Automation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Defining explicit schemas for schema-less data is a complex, time-consuming, and error-prone process, especially when dealing with hierarchically marked-up data, as it requires manual examination and writing of complicated path expressions, delaying the ability to query such data using powerful querying capabilities of RDBMS.
Innovation Solution
A dynamic data guide is generated to describe the structure of hierarchical data objects, allowing for automatic schema definition by parsing and updating the guide as data is added, using a data guide tree to efficiently detect changes and minimize resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If schema-discovery utilities are run to automatically generate schemas for schema-less data, then the manual effort and time required is reduced, but the processing of the entire body of data entails significant expenditure of time and computing resources
Solution Approach 1:
The patent applies preliminary action by maintaining a data guide that is continuously updated in the background as hierarchical data objects are added to the database. Instead of waiting until all data is collected before generating the schema, the system proactively builds and updates the data guide incrementally, so that schema information is already available when needed for querying.
Solution Approach 2:
The patent segments the schema discovery process into incremental updates based on individual hierarchical data objects or batches of data. Rather than processing the entire body of data at once, the system processes data in smaller units and updates the data guide progressively, reducing the computational burden at any given moment.
2Reliability
If schema-discovery utilities process the entire body of schema-less data to capture schema changes, then complete schema information is obtained, but significant computing resources are consumed
Solution Approach 1:
The patent extracts only the necessary structural information from hierarchical data objects as they are added to the database. Instead of re-processing the entire data body to detect schema changes, the system extracts and captures only the new or modified path expressions and data types from the incoming data, updating the data guide with these specific changes.
Solution Approach 2:
The system implements feedback by continuously monitoring and comparing the structure of newly added hierarchical data objects against the existing data guide. When changes are detected, the data guide is updated accordingly, creating a feedback loop that maintains schema accuracy without requiring full re-processing of all data.
3Manufacturing precision
If manual examination and writing of path expressions is performed for schema-less data, then schema accuracy can be ensured, but the process is complex, time-consuming, and error-prone
Solution Approach 1:
The patent applies self-service by enabling the system to automatically generate and update schema definitions through the data guide without requiring manual intervention. The system itself performs the task of examining hierarchical data objects, extracting path expressions, and updating schema information, eliminating the need for manual schema writing while maintaining accuracy.
Solution Approach 2:
The data guide serves as an intermediary between the hierarchical data objects and the querying system. It automatically captures and stores path expressions and data type information from the data, translating the complex hierarchical structure into a usable schema format without requiring manual path expression writing.
4Adaptability or versatility
If explicit schema is developed after schema-less data is added to database, then iterative development is facilitated, but the ability to query data using powerful RDBMS capabilities is delayed
Solution Approach 1:
The patent applies preliminary action by maintaining the data guide in advance as hierarchical data objects are added. The data guide is built incrementally and is ready before full schema processing would traditionally occur, enabling query capabilities to be activated sooner without compromising the iterative development approach.
Solution Approach 2:
The system implements dynamics by making the schema definition process adaptive and incremental rather than static and all-at-once. The data guide evolves dynamically as data is added, allowing the schema to be partially functional earlier and fully functional later, bridging the gap between iterative development and query capability availability.
Data Source
AI summary
A data guide is dynamically generated. The data guide describes the structures of hierarchical data objects added to a collection of hierarchical data objects. Examples of hierarchical data objects are documents that conform to XML (Extensible Mark-up Language) or data objects that conform to JSON (JavaScript Object Notation). The data guide may be created and/or updated as hierarchical data objects are added to the collection.