Source-Adapted Data Retrieval for Multi-Tenant Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-tenant data processing systems face performance issues and costly migration challenges when accessing data from multiple distinct deployments, as existing solutions disrupt single-tenant deployments and complicate data retrieval and formatting.
Innovation Solution
An integration computing device maintains scheduling parameters for source computing devices, receives modification indicators, and periodically requests and retrieves subsets of source data records using bulk record requests to mitigate performance disruptions and optimize data importation into a multi-tenant repository.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is accessed from multiple distinct deployments in a multi-tenant system, then data versatility and integration capability are improved, but system performance and reliability deteriorate due to disruptions in single-tenant deployments
Solution Approach 1:
The system segments data retrieval operations by maintaining separate scheduling parameters for each source computing device, allowing independent control of data pull operations for different tenants. This segmentation enables the system to access data from multiple deployments without creating uniform disruption across all tenants, as each tenant's data retrieval can be scheduled independently based on their specific performance requirements and usage patterns.
Solution Approach 2:
The system performs preliminary actions by pre-establishing scheduling parameters and modification indicator mechanisms before actual data retrieval operations. Modification indicators are generated in advance when data changes occur, allowing the integration computing device to prepare bulk record requests efficiently without causing disruptions during critical operational periods. This preliminary setup enables low-impact data synchronization.
2Measurement precision
If data retrieval operations are performed frequently to maintain up-to-date information, then data freshness and accuracy are improved, but computational load on source systems increases causing performance degradation
Solution Approach 1:
The system implements periodic action through scheduling parameters that define optimal intervals for data retrieval from each source computing device. Instead of continuous or frequent polling, the system retrieves data at predetermined intervals that balance freshness requirements with computational load constraints. This periodic approach allows the integration computing device to maintain data freshness while preventing excessive computational demands on source systems.
Solution Approach 2:
The system uses feedback mechanisms through modification indicators that trigger data retrieval only when actual data changes occur. This feedback-driven approach ensures data freshness by responding to real changes while avoiding unnecessary retrieval operations that would increase computational load. The modification indicators provide precise feedback about what needs to be retrieved, enabling efficient bulk record requests.
3Ease of operation
If individual record requests are made to retrieve data from source systems, then data retrieval flexibility is improved, but system complexity and migration costs increase
Solution Approach 1:
The system merges multiple individual record requests into a single bulk record request containing multiple record identifiers. This combining approach maintains data retrieval flexibility by allowing selective retrieval of specific records through the bulk request mechanism, while significantly reducing system complexity by consolidating multiple operations into one standardized interface call. The bulk request format simplifies the migration and integration process compared to managing numerous individual requests.
Data Source
AI summary
A method includes: maintaining, at an integration computing device, a scheduling parameter corresponding to a source computing device maintaining a plurality of source data records with record identifiers; receiving, from the source computing device, a plurality of modification indicators, each modification indicator generated at the source computing device in response to modification of a source data record, and containing the record identifier of the source data record; storing the modification indicators in a repository; according to the scheduling parameter, periodically obtaining a subset of the source data records from the source computing device, by: retrieving the modification indicators from the repository; generating a bulk record request containing the plurality of record identifiers from the retrieved modification indicators; transmitting the bulk record request to the source computing device; and responsive to transmitting the bulk record request, receiving the subset of source data records from the source computing device.