Intelligent Execution Environment Selection via ML Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face difficulties in selecting the appropriate execution environment for computing jobs due to lack of experience or complex execution flows, leading to resource wastage or prolonged execution times if the environment is either too large for lightweight jobs or too small for heavyweight jobs.
Innovation Solution
A method that uses machine learning models to predict execution times across multiple environments by matching execution context and environment information, allowing for intelligent selection based on aggregated predictions and continuous model updating for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large execution environment is selected for a lightweight computing job, then the job can be executed, but resources are wasted and costs increase
Solution Approach 1:
The system performs preliminary analysis of the computing job's execution flow characteristics before job submission. By pre-evaluating the job's resource requirements and matching them with available execution environments, the system predicts execution times and recommends optimal environments, preventing resource waste before it occurs
Solution Approach 2:
The system collects actual execution data from completed jobs and uses it to train machine learning models. This feedback loop continuously improves the accuracy of execution time predictions, enabling better matching between job requirements and execution environment capabilities, thereby reducing resource waste
2Loss of energy
If a small execution environment is selected for a heavyweight computing job, then resources are conserved, but the execution time becomes too long
Solution Approach 1:
The system performs preliminary analysis of the computing job's execution flow characteristics before job submission. By pre-evaluating the job's resource requirements and matching them with available execution environments, the system predicts execution times and recommends optimal environments, preventing excessive execution time before it occurs
Solution Approach 2:
The system dynamically adjusts the selection criteria for execution environments based on the specific characteristics of the computing job. By changing the matching parameters according to job complexity and resource requirements, the system optimizes the balance between resource conservation and execution time
3Ease of operation
If users select execution environments based on experience and expectations, then the process is simple, but the selection accuracy decreases for complex jobs or inexperienced users
Solution Approach 1:
The system introduces an intelligent intermediary layer that automatically analyzes job characteristics and matches them with suitable execution environments. This intermediary uses machine learning models to provide accurate recommendations, bridging the gap between simple user interaction and complex selection criteria
Solution Approach 2:
The system enables execution environments to 'self-select' by automatically analyzing job requirements and identifying the most suitable environment. This self-service approach eliminates the need for user expertise while maintaining high selection accuracy through automated analysis
Data Source
AI summary
Mechanisms are provided for intelligently identifying an execution environment to execute a computing job. An execution time of the computing job in each execution environment of a plurality of execution environments is predicted by applying a set of existing machine learning models matching execution context information and key parameters of the computing job and execution environment information of the execution environment. The predicted execution time of the machine learning models is aggregated. The aggregated predicted execution times of the computing job are summarized for the plurality of execution environments. Responsive to a selection of an execution environment from the plurality of execution environments based on the summary of the aggregated predicted execution times of the computing job, the computing job is executed in the selected execution environment. Related data during the execution of the computing job in the selected execution environment is collected.