Dynamic Execution Environment Upgrade for Event-Driven Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Event-driven computing services, such as AWS Lambda, face limitations when executing program code in response to events, leading to timeouts and failures if the processing is not completed within a specified time, especially when dealing with large data files that exceed the virtual machine's specifications.
Innovation Solution
The system implements an asynchronous process code execution method that dynamically switches to a higher-specification execution environment when processing exceeds the initial environment's limitations, ensuring that the event processing can continue reliably by monitoring execution time and retry limits, and dynamically adjusting the environment to complete tasks efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If program code is executed in a fixed virtual machine environment with specified CPU and memory resources, then the service can be simplified and billed by the hour, but processing timeouts occur when data size exceeds the environment's capacity
Solution Approach 1:
The system dynamically adjusts the virtual machine environment specifications during program code execution based on monitored data processing progress. When the initial environment is insufficient, the system automatically upgrades to a higher-specification environment, making the execution environment flexible and adaptive rather than fixed, thus preventing timeouts while maintaining service simplicity
Solution Approach 2:
The system changes the execution environment parameters (CPU specification, memory capacity) during runtime based on the actual data processing requirements. By monitoring processing progress and comparing it against timeout thresholds, the system modifies environment parameters to match the actual workload, resolving the contradiction between fixed simplicity and flexible reliability
2Reliability
If the virtual machine environment specifications are increased to handle larger data files, then processing reliability improves, but the cost and resource consumption increase
Solution Approach 1:
Instead of always providing maximum specifications, the system provides only the necessary resources by monitoring processing progress in real-time. When the initial environment is sufficient, it maintains that level; only when processing indicates insufficiency does it upgrade to a higher environment, avoiding unnecessary resource consumption while ensuring reliability when needed
Solution Approach 2:
The system continuously monitors the progress of data processing and uses this feedback to determine whether the current virtual machine environment is adequate. By comparing processing progress against timeout thresholds and capacity limits, the system dynamically decides whether to maintain the current environment or upgrade, optimizing resource usage based on actual processing needs
3Productivity
If the execution time limit is extended to handle complex processing, then processing completeness improves, but the service timeout limitation is exceeded
Solution Approach 1:
The system dynamically extends the execution time limit during program code execution based on monitored processing progress. When the initial time limit is approached and processing is not complete, the system automatically extends the timeout threshold, allowing complex processing to finish without being artificially truncated, thus resolving the contradiction between processing completeness and time limits
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In the case where an indication that a limit of an event-driven computing service will be reached is detected during the execution of asynchronous process code(S1104), the same asynchronous process code is started in an execution environment having higher specifications from the asynchronous process code currently being executed, and processing is continued (S1107).