Transparency Layer for Multi-Repository Data Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As data repositories grow in size, their performance decreases, leading to slower query times and backup/recovery issues, and existing applications often fail to access data moved to different repositories, requiring costly modifications or user configuration changes.
Innovation Solution
A transparency layer is created that allows multiple data repositories to be manifested as a single repository, using additional containers and constructs to combine data from different repositories, enabling unmodified applications to access data as if it were stored in a single repository.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is moved from a first repository to a second repository to maintain manageable size, then the first repository's performance is improved, but applications that need to access the moved data fail to find it
Solution Approach 1:
The patent introduces a transparency layer that acts as an intermediary between applications and multiple repositories. This layer intercepts data access requests from applications, translates them into appropriate repository-specific queries, and returns unified results. The transparency layer includes components such as a repository manager, query translator, and data mapper that work together to maintain data accessibility across distributed repositories without requiring application modifications.
Solution Approach 2:
The patent divides a single large repository into multiple smaller repositories, each storing a subset of the total data. This segmentation improves performance by reducing the size of individual repositories while maintaining data accessibility through the transparency layer that presents a unified view to applications. The data is partitioned across repositories based on criteria such as data type, access frequency, or organizational structure.
2Reliability
If applications are modified to access multiple repositories, then data accessibility is improved, but development cost and time increase
Solution Approach 1:
The transparency layer serves as a mediator that shields applications from the complexity of accessing multiple repositories. Applications continue to use standard single-repository access methods while the transparency layer handles the complexity of data distribution, repository management, and query translation. This eliminates the need for expensive application modifications while maintaining full data accessibility.
Solution Approach 2:
The transparency layer provides universal access to data across multiple repositories through a single standardized interface. It supports multiple access patterns, data formats, and repository types while presenting a consistent API to applications. This multi-functionality allows the system to handle diverse repository configurations without requiring application-specific modifications for each scenario.
3Quantity of substance
If a single large repository is used to store all data, then data completeness is maintained, but query performance and backup time increase
Solution Approach 1:
The patent segments the large repository into multiple smaller repositories, each containing a subset of the total data. This segmentation reduces query execution time by limiting the search space to relevant portions of data while maintaining data completeness through the transparency layer's ability to aggregate results from multiple repositories. Queries are automatically distributed to appropriate repositories based on data location information.
Solution Approach 2:
The patent applies local quality by optimizing each individual repository for specific access patterns and data types. Different repositories can be tuned with different parameters, indexing strategies, and storage configurations based on their specific data characteristics and access requirements. This localized optimization improves overall system performance while maintaining complete data availability through the transparency layer.
Data Source
AI summary
Techniques are provided for manifesting a multiple repository system as a single repository to an application that is designed to use a single repository. According to one aspect, a particular container, which is used by the application to store data, is identified. One or more additional containers are created in one or more different repositories. Each of the additional containers is formatted to store the data that the application stores in the particular container. A construct is created. The construct is associated with the particular container's identifier. The construct causes information from the particular container and each of the additional containers to be combined to answer database commands that reference the particular container's identifier. As a result, data stored in a single repository can be distributed among multiple repositories, and the unmodified application can continue to access that data as though the data was stored in the single repository.