Entity API Query Segmentation for Scalable Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face inefficiencies in accessing and managing vast amounts of data, particularly when dealing with millions or billions of data entity instances, as they require users to have detailed knowledge of the underlying database structure and incur high computational burdens during attribute value retrieval.
Innovation Solution
The entity API processes queries in two stages: identification and enrichment, generating separate executable queries for identifying data entities and obtaining attribute values, allowing efficient access to data entity instances without requiring knowledge of the underlying storage structure and minimizing computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a data processing system manages vast amounts of data using traditional interfaces (GUI or API), then users can access data entities, but users must have detailed knowledge of the underlying database structure and the system incurs high computational burdens during attribute value retrieval
Solution Approach 1:
The patent introduces an intermediary layer between the user and the database that automatically translates high-level data entity queries into database-specific queries. This intermediary handles the complexity of database structures internally, allowing users to access data entities without needing detailed knowledge of underlying database schemas, tables, or columns.
2Productivity
If a data processing system retrieves attribute values for millions or billions of data entity instances, then comprehensive data access is achieved, but the computational burden and time required for query processing increase significantly
Solution Approach 1:
The patent segments the query processing into distinct stages: identifying data entity instances in a first stage, and retrieving attribute values in a second stage. This segmentation allows the system to process queries in a streaming manner, where results from the first stage feed into the second stage, enabling efficient handling of large volumes of data entities without requiring all data to be loaded into memory simultaneously.
Solution Approach 2:
The patent performs preliminary identification of data entity instances before retrieving their attribute values. By first identifying which data entity instances match the query criteria and then subsequently retrieving only the relevant attribute values for those identified instances, the system avoids unnecessary computational work and reduces overall query processing time.
3Adaptability or versatility
If a data processing system provides direct access to underlying database structures, then detailed data manipulation is possible, but the system becomes tightly coupled to specific storage mechanisms reducing scalability
Solution Approach 1:
The patent introduces an intermediary layer that abstracts the data access logic from the underlying storage mechanism. This intermediary translates high-level data entity queries into database-specific queries, allowing the system to maintain adaptability for different data manipulation needs while remaining decoupled from specific storage implementations. The intermediary handles the complexity of database structures internally, allowing users to access data entities without needing detailed knowledge of underlying database schemas.
4Productivity
If a data processing system processes queries for millions or billions of data entity instances in a traditional manner, then complete data retrieval is achieved, but the computational resources required exceed practical limits
Solution Approach 1:
The patent implements a two-stage query processing approach where the system first identifies data entity instances that match query criteria, then retrieves attribute values only for those identified instances. This partial action approach ensures that computational resources are expended only on retrieving data that actually matches the query conditions, rather than processing all possible data entities, thereby reducing overall computational resource consumption while maintaining complete data retrieval for matching instances.
Data Source
AI summary
Techniques for obtaining information about data entity instances managed by a data processing system using at least one data store. The techniques include obtaining a query comprising a first portion comprising information for identifying instances of a first data entity stored in at least one data store; and a second portion indicating at least one attribute of the first data entity; generating, from the query, a plurality of executable queries including a first set of one or more executable queries and a second set of one or more executable queries, the generating comprising: generating, using the first portion, the first set of executable queries for identifying instances of the first data entity, and generating, using the second portion, the second set of executable queries for obtaining attribute values for instances of the first data entity; and executing the plurality of executable queries to obtain results for the query.