Multi-database System for Large Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large data sets are difficult to access and manage due to their size and varying formats, leading to inefficiencies in processing and network resource utilization.
Innovation Solution
A multi-database system that utilizes metadata to construct graph-based and relational databases, prioritizing processing tasks based on data set importance, allowing for efficient data access and reduced network bandwidth usage by storing metadata separately and performing processing tasks during low computational loads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large data sets are stored and processed directly, then data completeness is maintained, but access difficulty and management complexity increase
Solution Approach 1:
The patent segments large data sets into smaller chunks and creates multiple database representations (original format database, normalized database, and summary database). Each segment serves specific access patterns, making data management more manageable while maintaining complete data availability through the original format database.
Solution Approach 2:
The patent introduces intermediate database structures (normalized database and summary database) that act as mediators between the raw data and user queries. These intermediate structures pre-process and organize data, reducing the complexity of direct access to large data sets while preserving data completeness.
2Adaptability or versatility
If data sets are processed in multiple formats, then data versatility is improved, but processing complexity increases
Solution Approach 1:
The patent divides data processing into separate database structures, each handling specific format requirements. The normalized database handles structured queries, while the summary database handles aggregate operations, allowing the system to support multiple data formats and access patterns without requiring complex processing logic in a single database.
Solution Approach 2:
The patent creates a multi-functional database system where different database structures serve different purposes. The same underlying data set can be queried through multiple database representations, each optimized for specific types of operations, thereby achieving data versatility through a universal multi-database architecture.
3Productivity
If processing tasks are performed on large data sets, then data utility is enhanced, but network resource usage increases
Solution Approach 1:
The patent performs preliminary processing actions by creating normalized and summary databases in advance. These pre-processed databases contain data in formats optimized for specific query types, allowing rapid response to user requests without requiring real-time processing of large data sets, thereby reducing network resource usage during actual data access operations.
Data Source
AI summary
A method may include obtaining metadata associated with first and second data sets that may be stored at remote computing devices. The method may also include constructing a first database based on the metadata, and performing a first processing task on the first data set. The first processing task may include obtaining raw data of the first data set, and importing the raw data of the first data set into a second database that may be configured as a relational database. The first processing task may also include generating relationships in the second database between entries of the first data set to facilitate anticipated responses to queries. The method may additionally include performing a second processing task different from the first processing task on the second data set, and after performing the first processing task, receiving a query that invokes the first data set, and responding to the query.