Self-Learning Smart NIC Service Scheduling for Predictive Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional schedulers in cloud-based data centers lack awareness of future resource utilization trends, leading to inefficient scheduling that can result in degraded service performance or underutilization of server resources due to high resource utilization services being scheduled onto servers facing resource deficiencies, while low resource utilization services are scheduled onto servers with available resources.
Innovation Solution
Implementing a self-learning scheduler that uses a machine learning model to predict future resource requirements and availabilities, allowing services with high future resource needs to be scheduled on servers with high future resource availability, thereby optimizing resource utilization and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional schedulers are used to allocate services to servers, then the scheduling process is simple and fast, but resource utilization efficiency deteriorates due to lack of awareness of future resource utilization trends
Solution Approach 1:
The system performs preliminary actions by training machine learning models on historical utilization data before actual scheduling decisions are needed. The models predict future resource requirements and server availability in advance, allowing the scheduler to make informed decisions about service placements before the actual deployment occurs, thereby improving resource utilization efficiency without adding complex real-time scheduling logic.
Solution Approach 2:
The patent introduces machine learning models as intermediary components between historical data and scheduling decisions. These models act as mediators that translate historical utilization patterns into predictive insights, which then guide the scheduler. This intermediary layer enables sophisticated resource allocation without requiring the scheduler itself to become complex, as the ML models handle the analytical complexity.
2Reliability
If services are scheduled based on current resource availability, then scheduling decisions can be made quickly, but service performance deteriorates when future resource requirements are not met
Solution Approach 1:
The system performs preliminary analysis by training machine learning models on historical data to predict future resource requirements and server availability. This advance preparation allows the scheduler to identify suitable servers for future service deployments before actual scheduling decisions are needed, ensuring that performance requirements are met while maintaining quick decision-making capability.
Solution Approach 2:
The patent implements dynamic scheduling by continuously updating machine learning models with new historical utilization data. The models adapt to changing patterns in service resource requirements and server availability over time, allowing the system to maintain high service performance reliability even as workloads and infrastructure evolve, without requiring manual reconfiguration.
3Measurement precision
If more historical data is collected and analyzed for better predictions, then prediction accuracy improves, but data processing time and computational resources increase
Solution Approach 1:
The system applies partial action by selectively collecting and analyzing only the most relevant historical utilization features rather than processing all available data. The machine learning models focus on key predictors of resource requirements and server availability, achieving sufficient prediction accuracy without the computational overhead of analyzing every possible data point, thus balancing accuracy with resource efficiency.
Data Source
AI summary
An example method comprises determining, by an edge services controller, based on a respective predicted resource utilization value for each of a plurality of servers, a corresponding server weight for each of the plurality of servers; the plurality of servers comprising respective network interface cards (NICs), wherein each NIC of the plurality of NICs comprises an embedded switch and a processing unit coupled to the embedded switch; determining, by the edge services controller, based on a respective predicted resource utilization value for each of a plurality of services, a corresponding application weight for each of the plurality of services; and scheduling, by the edge services controller, based on the respective server weight for a server of the plurality of servers and the respective application weight for the service, a service of the plurality of services on the server.