Non-Relational Database Metadata for Distributed Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed databases with multiple storage devices often experience increased retrieval times due to the dispersion of data across multiple computers or networks, which can negatively impact data access efficiency.
Innovation Solution
A computer-implemented method generates metadata based on the location, state, and data in a non-relational database, using this metadata to execute queries and generate result sets, allowing for efficient data retrieval from multiple devices and updating data accordingly, with features including full text indexing and shard management across computing devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in multiple computers or storage devices to provide more storage capacity, then storage capacity is improved, but retrieval time increases
Solution Approach 1:
The system generates and stores metadata about data locations, states, and content before actual retrieval operations. This preliminary metadata creation enables the system to quickly locate and retrieve data from distributed storage devices without scanning the entire distributed storage system, thus reducing retrieval time while maintaining high storage capacity through distribution.
Solution Approach 2:
Metadata acts as an intermediary layer between the query system and the distributed storage devices. Instead of directly querying multiple storage devices, the system uses metadata to identify the precise locations and states of data, then retrieves only the needed data from specific devices, significantly improving retrieval efficiency in distributed environments.
2Quantity of substance
If data is dispersed over a network of interconnected computers to provide more storage, then storage capacity is improved, but data access efficiency deteriorates
Solution Approach 1:
The system performs preliminary actions by generating and maintaining metadata that describes data locations, states, and characteristics before access is needed. This pre-computed metadata enables efficient data access operations across the distributed network, allowing the system to quickly identify and retrieve data without traversing the entire network, thus improving data access efficiency while utilizing distributed storage capacity.
Solution Approach 2:
Metadata serves as an intermediary that bridges the query system and distributed storage resources. It provides a centralized or cached view of data locations and states, enabling efficient data access decisions without directly querying multiple networked computers, thereby improving data access efficiency while maintaining the benefits of distributed storage capacity.
3Speed
If metadata is generated and stored to improve data retrieval efficiency, then retrieval speed is improved, but system complexity increases
Solution Approach 1:
The system extracts and stores only the essential metadata information (location, state, and data characteristics) rather than copying the entire data structure. This selective extraction reduces the overhead and complexity of the metadata management system while still enabling fast data retrieval operations, as only the necessary metadata needs to be processed and stored.
Solution Approach 2:
Instead of creating complex copies of the entire database structure, the system creates simplified metadata representations that capture the essential information needed for retrieval operations. These lightweight metadata copies enable fast querying and retrieval without replicating the full database complexity across multiple systems.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, computer program product, and computer system for a database system and method. In some embodiments, metadata associated with data stored in a non-relational database is generated. The metadata is based upon, at least in part, at least one of a location of the data, a state of data, and the data. The metadata is stored in a data structure in memory. A query for data stored in the non-relational database is received. The query is executed using the generated metadata to generate a metadata result set. A result set including data in the non-relational database is generated using the generated metadata result set.