Campaign Batch Monitoring for Spot Instance Retry Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current notification systems using AWS spot instances face challenges with unreliable delivery due to instance reclamation, high costs with on-demand instances, lack of automatic retry mechanisms, and resource inefficiencies, leading to incomplete or missed campaigns.
Innovation Solution
A system that segments users into batches, processes campaigns using auto-scaling virtual machine instances, and implements an auto-healing process to monitor and automatically retry interrupted batches, ensuring reliable delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If spot instances are used for cost efficiency, then operational costs are reduced, but delivery reliability deteriorates due to instance reclamation
Solution Approach 1:
The system performs preliminary actions by saving campaign state information (processed batch identifiers, user device information) in a database before instance reclamation occurs. This allows the system to resume processing from the saved state rather than losing progress, thereby maintaining delivery reliability while using cost-effective spot instances.
Solution Approach 2:
The system implements feedback mechanisms by monitoring campaign processing status and detecting interruptions. When a spot instance is reclaimed, the system receives feedback about the interruption, retrieves saved state information, and automatically resumes processing, thus maintaining reliability without sacrificing cost efficiency.
2Reliability
If on-demand instances are used for stability, then delivery reliability is improved, but operational costs increase significantly
Solution Approach 1:
The system dynamically adjusts resource usage by implementing automatic retry mechanisms that can utilize available instances (whether spot or on-demand) to complete interrupted campaigns. This dynamic approach maintains reliability without requiring permanent commitment to expensive on-demand instances, thereby reducing operational costs.
3Productivity
If spot instances are used, then resource utilization is optimized, but campaign processing continuity deteriorates due to interruptions
Solution Approach 1:
The system performs preliminary actions by continuously saving campaign processing state (batch identifiers, user device information) to a database before interruptions occur. This ensures that when spot instances are reclaimed, the system can resume processing from the saved state, maintaining continuity without sacrificing resource utilization efficiency.
Solution Approach 2:
The system ensures continuity of useful action by implementing automatic retry mechanisms that resume campaign processing from saved state information. This allows the system to maintain continuous progress on campaigns despite spot instance interruptions, thereby preserving both resource utilization efficiency and processing continuity.
4Reliability
If manual intervention is used for missed batches, then delivery completeness can be achieved, but operational complexity and time increase
Solution Approach 1:
The system implements self-service by automatically detecting interrupted campaigns, retrieving saved state information from the database, and resuming processing without human intervention. This automated approach achieves delivery completeness while reducing operational complexity and eliminating the need for manual monitoring and intervention.
Solution Approach 2:
The system uses feedback mechanisms to automatically monitor campaign processing status, detect interruptions, and trigger retry operations. This closed-loop feedback system ensures delivery completeness while minimizing operational complexity by eliminating manual intervention requirements.
Data Source
AI summary
A method and system for sending a campaign to a set of users is disclosed. The method comprises receiving a request to send the campaign to the set of users. The set of users into are segmented into a plurality of batches. Each batch comprises at least a predetermined number of users from the set of users. The campaign is processed for the plurality of batches using one or more virtual machine instances. Furthermore, an auto-healing process is implemented to periodically monitor a processing status of the campaign across the plurality of batches. Further, one or more pending batches are identified from the plurality of batches for which the campaign has not been processed. Further re-initiating processing of the campaign for the identified one or more pending batches is performed. The campaign to the pending one or more batches of the plurality of batches is transmitted.