Dynamic Data Migration Throttling via Traffic Light Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual organization migration in cloud-based systems is inefficient, error-prone, and costly due to human supervision, and automated migrations can interfere with ongoing operations like Data Guard services, leading to suboptimal throttling and potential service disruptions.
Innovation Solution
Implementing a dynamic traffic light system that monitors and adjusts migration speed based on predicted Data Guard apply lag and catch-up time to throttle migration effectively, ensuring minimal disruption to Data Guard services by using concurrency controllers and migration schedulers to manage resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated migration is employed to improve efficiency and reduce human supervision costs, then productivity and ease of operation are improved, but migration may interfere with ongoing operations like Data Guard service, worsening reliability and service stability
Solution Approach 1:
The system dynamically adjusts migration speed based on real-time monitoring of Data Guard service performance metrics. The migration process transitions from static automated execution to dynamic control, where the migration rate is continuously adapted according to the apply lag observed in Data Guard operations, resolving the contradiction between automation efficiency and service stability
Solution Approach 2:
The system implements a feedback mechanism that monitors Data Guard apply lag and uses this information to control migration speed. The migration process receives feedback about service performance and automatically adjusts its behavior, ensuring that productivity gains from automation do not compromise the reliability of ongoing Data Guard operations
2Duration of action of moving object
If migration speed is increased to reduce migration time, then duration of action is reduced, but impact on Data Guard service increases, worsening harmful factors generated by the migration
Solution Approach 1:
The migration speed is made dynamic rather than fixed, allowing the system to optimize the balance between migration duration and service impact. The speed adjusts in real-time based on monitored Data Guard performance, enabling the system to maintain acceptable migration timelines while minimizing harmful effects on ongoing operations
Solution Approach 2:
The system changes the parameter of migration speed based on observed Data Guard apply lag. By adjusting this critical parameter in response to service performance metrics, the system reduces the harmful impact on Data Guard operations while still achieving migration objectives within reasonable timeframes
3Object-generated harmful factors
If throttling is applied to reduce impact on Data Guard service, then harmful factors are reduced, but migration duration increases, worsening the duration of action
Solution Approach 1:
Rather than applying static throttling, the system uses dynamic speed adjustment based on real-time monitoring. The migration speed is optimized at each moment based on current Data Guard performance, preventing excessive throttling when unnecessary and applying it only when service impact is observed, thus minimizing both harmful factors and total migration duration
Solution Approach 2:
The feedback mechanism enables the system to apply throttling only when Data Guard apply lag indicates service impact. This responsive approach avoids unnecessary throttling that would extend migration time, while still protecting service stability when needed, optimally balancing both concerns
Data Source
AI summary
Migration results in specific action requests to move data from a source system instance to a target system instance. Migration may consume many resources. In an effort to monitor migration effects on source and/or target performance, one or more traffic lights are determined to monitor utilization of resources of the source and/or target. Based on the one or more traffic lights, migration is dynamically throttled. The one or more traffic light may be assigned a status based on how the migration affects performance of another data operation which may be contemporaneously operating on either the source and/or the target.