Split Inference Policy Control for Adaptive Neural Network Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining device selection in split inference of artificial neural networks are often inappropriate for the current device state and task requirements, making it difficult for users to correct these policies effectively.
Innovation Solution
A method and device for controlling inference task execution through split inference of an artificial neural network by determining a task execution policy based on requirements and failure rates, updating policies based on execution records, and training AI models to optimize device selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a fixed task execution policy is used for split inference, then device selection is simplified, but the policy becomes inappropriate for varying device states and task requirements
Solution Approach 1:
The patent implements dynamic task execution policies that automatically adjust based on real-time device states (CPU usage, memory availability, battery level) and task characteristics. The system transitions from static policy selection to dynamic policy generation, where policies are continuously adapted to match current operating conditions, resolving the contradiction between operational simplicity and adaptability.
Solution Approach 2:
The system changes key parameters including device selection criteria, inference split ratios, and policy priorities based on varying device states and task requirements. By dynamically adjusting these parameters rather than using fixed values, the system achieves both ease of operation through automated parameter management and adaptability to changing conditions.
2Measurement precision
If users directly correct task execution policies, then policy accuracy improves, but operational complexity increases significantly
Solution Approach 1:
The patent implements self-service mechanisms where the system automatically monitors execution results, detects failures, analyzes causes, and updates policies without user intervention. The autonomous policy optimization system includes automated feedback loops that continuously improve policy accuracy while keeping the user interface simple, eliminating the need for users to directly correct complex policies.
Solution Approach 2:
The system establishes feedback mechanisms that collect execution results and failure information, then automatically process this feedback to refine task execution policies. The feedback loop includes failure cause analysis, policy evaluation, and automated policy updates, achieving high policy accuracy through systematic feedback processing rather than manual user correction.
3Power
If multiple devices are used for split inference, then computational capacity increases, but coordination and management complexity increases
Solution Approach 1:
The patent creates a universal task execution policy framework that can be applied across multiple diverse devices with different capabilities. The standardized policy structure and unified coordination mechanism enable the system to manage heterogeneous devices (mobile phones, tablets, PCs, servers) through a single framework, increasing computational capacity while controlling coordination complexity through standardization.
Data Source
AI summary
A method of controlling execution of an inference task includes determining a task execution policy based on requirements of the inference task or a correction index, wherein the correction index is determined based on failure rates of task execution policies; determining, based on the task execution policy, devices to execute split inference; obtaining an updated correction index corresponding to the task execution policy, based on result information indicating the split inference has failed; and obtaining an updated task execution policy based on execution records of the first split inference obtained from the one or more first devices, wherein the execution records include failure cause information of the first split inference, wherein a task execution policy from among the first plurality of task execution policies includes a priority of device conditions for selecting a device to execute the split inference, or a number of devices used for the split inference.