Inference Processing Unit Task Scheduling for Neural Network Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to perform inference processing using multiple neural network (NN) models within a limited calculation environment, such as IoT devices, without exceeding the required processing time.
Innovation Solution
An information processing apparatus and method that includes an obtainer for sensing data, an inference processing unit to input data into multiple inference models, a determiner to schedule tasks based on processing time information, and a controller to process tasks according to the schedule, allowing for efficient processing even in limited environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple inference models are used to improve processing accuracy and functionality, then the inference capability and task coverage are improved, but the calculation load and processing time increase beyond the limited environment's capacity
Solution Approach 1:
The patent applies preliminary action by predicting the processing time of each subsequent task before actual execution. The prediction unit estimates how long each task will take based on the inference result, allowing the system to pre-plan the execution schedule and avoid time-consuming trial-and-error during actual processing
Solution Approach 2:
The patent implements dynamics by creating an adaptable task schedule that can be dynamically adjusted based on predicted processing times. The schedule generation unit continuously optimizes the execution order of multiple tasks according to real-time conditions and resource availability, making the system flexible rather than rigid
2Reliability
If multiple inference models are executed sequentially to ensure accurate processing, then the inference accuracy is maintained, but the total processing time exceeds the required deadline
Solution Approach 1:
The system performs preliminary prediction of processing times for all subsequent tasks before execution begins. This allows the schedule generation unit to pre-calculate the optimal execution sequence that maintains accuracy requirements while minimizing total processing time
Solution Approach 2:
The patent changes the parameter of task execution from fixed sequential order to dynamically optimized ordering based on predicted processing times. By adjusting the execution sequence parameter according to predictions, the system achieves both high reliability and improved productivity
3Power
If the calculation capability is increased to handle multiple inference models simultaneously, then the processing speed and throughput are improved, but the device complexity and resource requirements increase
Solution Approach 1:
The patent segments the inference processing into distinct stages: prediction stage and execution stage. The prediction unit separately estimates processing times, while the schedule generation unit separately optimizes task ordering. This segmentation allows complex multi-model processing to be managed through simpler, modular components
Solution Approach 2:
The prediction unit acts as an intermediary between the inference models and the task execution system. It translates the complex output of multiple inference models into simplified processing time estimates that the schedule generation unit can use to create efficient execution plans, reducing the complexity burden on the execution system
Data Source
AI summary
An information processing apparatus includes: an obtainer that obtains sensing data; a common neutral network (NN) that inputs the sensing data into an inference model to obtain a result of inference and information on a processing time for a plurality of tasks subsequent to the processing performed by the inference model; and an NN inference computation management unit that determines a task schedule for a task processing unit that processes the plurality of subsequent tasks to process the plurality of subsequent tasks on the basis of the information on the processing time for the plurality of subsequent tasks and inputs the result of the inference into the task processing unit to process the plurality of subsequent tasks according to the determined task schedule.