Asynchronous Remote Data Query Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large enterprises face challenges in querying remote data stored in distributed systems due to high data loads and strict rules such as availability and consistency, especially for statistical queries that do not require all data to be transferred to a central system.
Innovation Solution
A system and method that allows a central data requesting system to send remote data queries asynchronously to multiple remote systems, enabling analysis even if only some systems respond, using asynchronous communication protocols and independent query results processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all remote data is transferred to the central system for querying, then complete data availability is achieved, but data load on the central system increases significantly
Solution Approach 1:
The patent extracts only the necessary query results from remote systems rather than transferring all remote data to the central system. The query is sent to remote systems and only the resulting data (not the entire dataset) is transferred back, significantly reducing data load while maintaining data availability for the specific query needed.
Solution Approach 2:
The patent segments the data retrieval process by keeping data distributed across multiple remote systems rather than consolidating it all at the central system. Each remote system independently processes queries and returns results, allowing the central system to access needed data without holding all data centrally, thus reducing central system data load while maintaining availability.
2Reliability
If distributed databases with strict rules are used, then data consistency is maintained, but query flexibility is reduced
Solution Approach 1:
The patent applies partial action by implementing consistency rules only where necessary (in distributed databases) while allowing flexibility in other areas. For statistical queries, the system accepts that not all data needs to be consistently updated across all systems, enabling flexible querying without requiring full consistency for all data types.
Solution Approach 2:
The patent applies different quality requirements to different data and query types. Strict consistency rules are applied locally to distributed databases where they are needed, while statistical and analytical queries operate with more flexible requirements. This allows the system to maintain consistency where critical while providing flexibility for analytical workloads.
3Reliability
If synchronous query processing is used, then all remote systems must be available, but system reliability is reduced
Solution Approach 1:
The patent introduces an intermediary asynchronous communication mechanism between the central system and remote systems. Instead of requiring synchronous confirmation from all remote systems, the central system sends queries asynchronously and processes results as they arrive. This intermediary approach allows queries to proceed even if some remote systems are temporarily unavailable, improving both reliability and efficiency.
4Measurement precision
If complete data transfer is performed, then accurate statistical analysis is achieved, but data transfer time increases
Solution Approach 1:
The patent extracts only the specific data needed for statistical analysis from remote systems rather than transferring complete datasets. By sending targeted queries and retrieving only the resulting data, the system achieves sufficient analysis accuracy while dramatically reducing data transfer time compared to transferring all available data.
Data Source
AI summary
Various embodiments of systems and methods for communicating with a plurality of remote systems are described herein. A remote data query is sent from a central data requesting system to remote systems. A query result is received asynchronously from at least one of the remote systems. The remote systems may include data pools which are independent of each other. In one embodiment, the remote systems include a responsive remote system and an unresponsive remote system, and the query result is received from the responsive remote system. In one embodiment, a predetermined timeout period is set at the central data requesting system and the query result obtained after the pre-determined timeout period is ignored by the central data requesting system.