Catalogued Database Metadata Search for Faster Data Location
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large organizations face challenges in efficiently searching through numerous databases to locate specific data due to non-specific machine names and the tedious process of examining transaction logs, which is time-consuming and resource-intensive.
Innovation Solution
Implementing an enhanced database search system that utilizes catalogued metadata, including table names, schema information, indexed columns, and PII, to facilitate faster and more precise searches across multiple databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional machine names are used to identify databases, then the system is simple to operate, but the search precision is insufficient and locating specific data becomes time-consuming
Solution Approach 1:
The system performs preliminary actions by generating and storing metadata records for all databases before searches are conducted. These metadata records include table names, column names, data types, and other structural information. When a search is initiated, the system queries these pre-generated metadata records instead of examining entire databases or transaction logs, dramatically reducing search time while improving precision through structured indexable data.
Solution Approach 2:
The patent introduces metadata records as an intermediary layer between the database system and the search function. These metadata records serve as a mediator that contains pre-extracted structural information about databases, tables, and columns. The search system queries this intermediary metadata layer rather than directly searching through raw database contents or transaction logs, enabling fast and precise data location without compromising system simplicity.
2Reliability
If transaction logs are examined to locate specific data, then comprehensive search coverage is achieved, but the process becomes resource-intensive and time-consuming
Solution Approach 1:
The system extracts only the essential structural metadata from databases (table names, column names, data types, relationships) and stores this extracted information in separate metadata records. This extraction process separates the search-critical information from the full database contents and transaction logs. When searching, the system queries only this extracted metadata rather than processing entire databases or examining transaction logs, achieving reliable search coverage with minimal computing resources.
Solution Approach 2:
Instead of performing exhaustive searches through entire databases or transaction logs, the system applies partial action by searching only through the metadata records which contain representative structural information. This partial search approach focuses computational effort on the most relevant data (metadata that describes database structure) rather than processing all possible data, significantly reducing resource consumption while maintaining search effectiveness.
3Productivity
If comprehensive database metadata is catalogued for enhanced searching, then search precision and speed improve, but the system complexity increases
Solution Approach 1:
The patent segments the database system into distinct components: the original databases, the metadata extraction layer, and the search query layer. Metadata records are segmented into specific categories (table metadata, column metadata, relationship metadata). This segmentation allows each component to operate independently and efficiently, with the metadata layer serving as a simplified interface between databases and search functions, improving productivity without proportionally increasing overall system complexity.
Data Source
AI summary
In one embodiment, a method comprises receiving a search request from a user; responsive to the search request, searching a plurality of metadata records corresponding to a plurality of databases, a metadata record of the plurality of metadata records comprising a plurality of searchable metadata items of a database of the plurality of databases, wherein the plurality of searchable metadata items of the metadata record includes a name of a table of the database; and providing search results responsive to the search request, the search results comprising indications of a subset of the plurality of metadata records that match the search request.