Object Schema API Query Coalescing for Faster Data Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing APIs generate a large number of queries for one-to-many data relationships, leading to inefficient resource usage and increased processing time for read and write operations.
Innovation Solution
Optimizing queries by generating a graph projection of the API, identifying data caching opportunities, and coalescing subqueries to reduce the number of queries through parallel execution and data denormalization across cloud locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional API query generation is used for one-to-many data relationships, then data can be retrieved from multiple sources, but the number of queries increases significantly leading to inefficient resource usage
Solution Approach 1:
The patent merges multiple subqueries into a single optimized query by identifying common data access patterns across multiple nodes in the graph projection. Instead of executing separate queries for each node, the system combines them into one query that retrieves all necessary data in a single operation, thereby reducing the total number of queries while maintaining comprehensive data retrieval.
Solution Approach 2:
The system performs preliminary analysis of the graph projection to identify caching opportunities and data denormalization possibilities before query execution. By pre-processing the query plan to detect patterns and prepare optimized execution strategies, the system reduces the number of actual queries that need to be executed against data sources.
2Loss of information
If multiple subqueries are generated for each node in the navigable path, then complete data can be retrieved, but processing time increases due to sequential query execution
Solution Approach 1:
Multiple subqueries targeting different nodes but accessing the same or overlapping data sources are merged into a single query. This combination ensures that all required data is retrieved in one operation rather than through multiple sequential queries, maintaining data completeness while reducing processing time.
Solution Approach 2:
The optimized query execution maintains continuous data retrieval operations by eliminating idle time between sequential query executions. The system processes all necessary data access operations in a single continuous query execution flow, preventing gaps and delays that would occur with traditional sequential query processing.
3Reliability
If data is accessed from multiple cloud locations, then data availability is improved, but resource usage increases due to redundant data retrieval
Solution Approach 1:
The system applies local quality optimization by identifying which data can be cached at specific cloud locations based on access patterns. Instead of uniformly retrieving data from all locations, the system optimizes data placement and retrieval strategies for each location, caching frequently accessed data locally to reduce redundant cross-location data transfers and associated resource consumption.
4Ease of manufacture
If traditional query execution is used without optimization, then implementation is simple, but the number of queries and processing overhead are high
Solution Approach 1:
The system introduces an intermediary optimization layer between the API layer and the data sources. This intermediary component automatically analyzes query patterns, identifies optimization opportunities, and generates optimized queries without requiring changes to the existing API interface or data source structures. The intermediary handles the complexity of query optimization transparently, maintaining ease of implementation while achieving performance improvements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to processing read requests to retrieve data from a plurality of data sources. According to one embodiment, an example method includes determining a navigable path of nodes accessed to satisfy a read request based on a graph projection of an application programming interface (API). An API service generates a plurality of subqueries, each subqueries being associated with a node in the navigable path. While traversing the navigable path to satisfy the read request, the API service identifies data associated with lower level nodes that is cached at a data source associated with a current node, replaces subqueries directed to data stored at the current node and the identified data with a single subquery executed against the data source associated with the current node, and executes the single subquery at the current node. The API service returns data accessed during traversal of the navigable path.