AI Inference Task Scheduling for Resource-Aware Embedded Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for executing multiple AI models on embedded devices face limitations in hardware and platform scalability, and fail to consider resource availability and failure modes, leading to inefficiencies and reliability issues.
Innovation Solution
A scheduling method and apparatus that generate resource lists and identify available resources, determine task conditions based on state information, and perform or alternate tasks accordingly to ensure efficient execution of AI models while maintaining reliability and optimizing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple AI models are executed simultaneously on embedded devices, then the cognitive and decision-making capabilities are improved, but the hardware resource management complexity increases
Solution Approach 1:
The patent segments AI model execution into separate task threads, each independently managed. The scheduling system divides resource allocation decisions into discrete units, assigning specific models to specific hardware resources based on predefined criteria, thereby reducing overall management complexity while enabling multiple models to run simultaneously
Solution Approach 2:
The patent implements dynamic resource allocation where the scheduling system continuously monitors hardware resource availability and adjusts task assignments in real-time. This dynamic approach allows the system to adapt to changing resource states, maintaining optimal performance across multiple AI models without fixed, complex static configurations
2Productivity
If conventional task distribution methods are used, then task parallelization is achieved, but resource availability and failure modes are not considered
Solution Approach 1:
The patent performs preliminary resource availability checks and failure mode assessments before assigning tasks to hardware resources. The scheduling system pre-evaluates potential failures and resource constraints, allowing it to preemptively adjust task assignments to ensure both parallelization and reliability
Solution Approach 2:
The patent implements feedback mechanisms where the scheduling system continuously monitors resource usage, task execution status, and hardware state. This feedback loop enables real-time adjustments to task assignments based on actual resource availability and detected failures, maintaining both productivity and reliability
3Reliability
If AI models are executed with high reliability requirements, then functional safety is improved, but software optimization complexity increases
Solution Approach 1:
The patent changes key scheduling parameters to prioritize functional safety requirements. The system adjusts task priority levels, resource allocation weights, and execution timing based on the safety-critical nature of specific AI models, thereby achieving high reliability without requiring overly complex optimization algorithms
Data Source
AI summary
An embodiment relates to a method for scheduling an inference task performed by at least one processor. The method comprises: generating a memory resource list and a processing unit list for the inference task based on a configuration file, and creating task threads for the inference task based on the processing unit list; confirming a first required resource available from the memory resource list and a second required resource available from the processing unit list in response to a start request for the inference task, and receiving state information of the first required resource and the second required resource; and determining whether preset task conditions are satisfied in consideration of the state information of the first required resource and the second required resource, and performing the inference task or performing an alternative task for the inference task according to whether the task conditions are satisfied.