Tabular DB Interface for Unstructured Data via Metadata Catalog
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database integration methods face difficulties in accessing unstructured data due to its flexible or non-existent schema, which is incompatible with relational databases that rely on fixed schema definitions, making it challenging to employ SQL-based interfaces effectively.
Innovation Solution
A system and method that maps SQL commands to unstructured data stores by generating unique 'flat file' field names from nested hierarchy field names, creating metadata for accessing unstructured data through a relational interface, allowing conditional comparisons and maintaining the data in its original unstructured form without conversion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If unstructured data is accessed through a relational interface, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The patent introduces a catalog as an intermediary layer between the relational database management system and the unstructured data store. The catalog contains metadata that maps relational queries to unstructured data fields, enabling SQL-based access without direct complexity exposure. This mediator translates structured query requirements into unstructured data access patterns while preserving the simplicity of relational interface usage.
2Adaptability or versatility
If metadata is generated for unstructured data, then adaptability is improved, but loss of information increases
Solution Approach 1:
The patent creates a metadata catalog that copies structural information from the unstructured data store into a relational format. This catalog contains field names, data types, and hierarchical relationships as metadata entries. The copying process preserves the essential structural information needed for querying while maintaining the original unstructured data intact, thus providing adaptability without information loss.
3Manufacturing precision
If unstructured data is converted to tabular form, then manufacturing precision is improved, but loss of substance increases
Solution Approach 1:
The patent extracts only the necessary structural metadata from the unstructured data store into a catalog, rather than converting the entire data set to tabular form. The metadata includes field names, types, and hierarchical relationships needed for query mapping. This extraction approach provides the precision of structured data access while avoiding the substantial data loss that would occur with full conversion.
4Reliability
If schema enforcement is applied to unstructured data, then reliability is improved, but ease of manufacture worsens
Solution Approach 1:
The patent implements a dynamic metadata catalog that adapts to changes in the unstructured data structure. When new fields or data types are introduced into the unstructured data store, the catalog automatically updates its metadata representations. This dynamic approach maintains reliability through consistent query interfaces while preserving the ease of data insertion characteristic of unstructured stores.
Data Source
AI summary
A tabular (relational) DB interface is responsive to SQL commands for accessing unstructured data bases. An application receives a relational data command or query and maps fields from the relational query to fields in an unstructured data store including documents arranged in a hierarchy and unbounded by fixed types or field lengths. The application generates field names by concatenating nested hierarchy field names to define unique “flat file” field names in a tabular form. The application generates a catalog defining the mapping which is used as metadata for accessing the unstructured data to satisfy the relational query. Use of the metadata avoids copying or translating the unstructured data store to a tabular form because the unstructured data collection remains unmodified, and is accessed via the catalog.