Dynamic API Allocation for Data Tagging and Cloud Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Retrieving data from a single stream can lead to performance degradation as systems evolve, and existing data storage methods do not efficiently manage highly utilized data, causing resource bottlenecks and performance issues.
Innovation Solution
A dynamic API allocation method based on data-tagging, where the system identifies highly used fields, creates an index, and migrates data to cloud-based object storage when a usage threshold is exceeded, establishing a bi-directional pipeline for seamless data transfer and management, allowing local requests to be satisfied by cloud-based data while conserving local resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored and retrieved from a single local stream, then data access is simple and direct, but system performance degrades as systems evolve and data grows
Solution Approach 1:
The patent segments data storage into two distinct locations: local system storage for recently accessed or frequently modified data, and cloud-based object storage for archival or less frequently accessed data. This segmentation allows the system to maintain high performance for active operations while offloading bulk storage to the cloud, thereby improving overall system productivity without excessive local complexity
Solution Approach 2:
The patent introduces an intermediary data migration service that acts as a mediator between local system storage and cloud-based object storage. This intermediary automatically migrates data between locations based on usage patterns, accessibility requirements, and storage policies, enabling seamless data access while optimizing performance and resource utilization without requiring application-level complexity
2Speed
If all data is kept at the local system, then data access speed is fast, but local resources are consumed and bottlenecked
Solution Approach 1:
The patent applies local quality by keeping recently accessed or frequently modified data at the local system where fast access is needed, while storing archival or less frequently accessed data in cloud-based object storage. This ensures that local resources are dedicated to data requiring high-speed access, while cloud storage handles bulk data capacity requirements, optimizing both speed and storage quantity
3Productivity
If data is migrated to cloud-based object storage, then local resources are conserved, but data access requires additional network operations
Solution Approach 1:
The patent implements preliminary action by pre-migrating data to cloud-based object storage based on predicted usage patterns, accessibility requirements, and storage policies before actual access needs arise. The system proactively manages data placement, so when data is needed, it is already in the optimal location (local or cloud), minimizing retrieval time while maintaining efficient resource utilization
Data Source
AI summary
Dynamic API allocation based on data-tagging is provided. Data that is stored in a local system is parsed and normalized. One or more highly used fields is identified and tagged. A counter corresponding to each highly used field is incremented upon each reference. Upon exceeding a threshold, data is migrated to object storage. An index is created for each highly used field. A bi-directional pipeline is created between the local system and the cloud-based system. The data structure is created in object storage in the cloud-based system. Data is dynamically migrated through the pipeline from the local system to cloud-based object storage. Cloud-based system sends an API endpoint to local system. Future data accesses to local data are redirected to object storage using the API endpoint. Local system continues monitoring data utilization. Upon utilization dropping below a threshold, data accesses are redirected to local system, using the local pointer.