Hierarchical Data Attribute Retrieval to Reduce API Duplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data retrieval processes face challenges such as heterogeneity in data formats, distribution across multiple sources, and query performance optimization, leading to increased latency and resource consumption due to duplicative communications and API calls.
Innovation Solution
The system generates dependency trees for data attributes, identifies an anchor dependency tree with the longest length, and iteratively queries data sources based on the current leaf node of this tree, optimizing the retrieval process by reducing duplicative communications and conserving resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional data retrieval processes are used to query multiple data sources, then data attributes can be obtained from heterogeneous sources, but latency increases and processing resources are consumed due to duplicative communications and API calls
Solution Approach 1:
The patent combines multiple independent queries into a single optimized query execution plan. The query processor merges queries targeting different data sources into one coordinated execution plan that retrieves multiple data attributes in a single pass, eliminating duplicative communications and reducing retrieval latency while maintaining complete data coverage
Solution Approach 2:
The system performs preliminary analysis of query dependencies and data source relationships before execution. The query processor pre-processes queries to identify overlapping data sources and attributes, building an optimized execution plan in advance that prevents redundant API calls and communications during actual data retrieval
2Reliability
If traditional data retrieval processes are used to query multiple data sources, then data attributes can be obtained from heterogeneous sources, but processing resources and network resources are consumed due to duplicative communications and API calls
Solution Approach 1:
The query processor merges multiple independent queries into one coordinated execution plan that targets multiple data sources simultaneously. By combining queries that access the same or overlapping data sources, the system reduces the total number of API calls and communications, thereby conserving processing resources, computing power, and network bandwidth while still retrieving all required data attributes
3Reliability
If traditional data retrieval processes are used to query multiple data sources, then data attributes can be obtained from heterogeneous sources, but network resources are consumed due to duplicative communications and API calls
Solution Approach 1:
The system merges multiple queries into a single optimized execution plan that consolidates network communications. By identifying and combining queries that access the same data sources, the system reduces duplicative API calls and network traffic, conserving network resources and reducing energy consumption while maintaining complete data retrieval from heterogeneous sources
Data Source
AI summary
In some implementations, a device may receive a request to obtain a set of data attributes. The device may generate one or more hierarchical lists for respective data attributes included in the set of data attributes. The device may identify an anchor hierarchical list, from the one or more hierarchical lists, that is associated with a highest order among the one or more hierarchical lists. The device may generate a first query to a first data source indicated by a first data attribute in a top position of the anchor hierarchical list, wherein the first query requests a first one or more data attributes including the first data attribute and any other data attributes, associated with the first data source, that are in top positions of respective hierarchical lists of the one or more hierarchical lists. The device may transmit, to the first data source, the first query.