Subtask-Adaptive Neural Network Pruning for Efficient Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network architectures require significant computational resources even when performing only a subset of tasks due to the need to run the entire network, which is inefficient for resource-constrained devices and applications.
Innovation Solution
A subtask adaptable neural network that includes a base network and a pruning mask generator, allowing for dynamic and controllable pruning to create a smaller network tailored to specific tasks, reducing computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large unified neural network is used to serve multiple applications, then the model can recognize a large number of categories, but it leads to inference-time inefficiency when only a subset of categories is needed
Solution Approach 1:
The patent segments the large unified neural network into multiple smaller subtask-specific networks, each trained to recognize a specific subset of categories. This allows the system to deploy only the necessary smaller network for each application, improving inference-time efficiency while maintaining the capability to handle multiple categories through selective deployment.
Solution Approach 2:
The patent introduces dynamic task routing that automatically selects and activates only the relevant subtask-specific network based on the input data characteristics. This dynamic selection mechanism ensures that the system adapts its computational resources to match the actual task requirements, avoiding the inefficiency of running a large network for simple subtasks.
2Measurement precision
If a significantly larger network is used for recognizing more categories, then the model performance improves, but computational budget and resource constraints are exceeded
Solution Approach 1:
The patent applies local quality by training each subtask-specific network to achieve high recognition accuracy for its designated subset of categories. Each smaller network is optimized locally for its specific task, ensuring that recognition precision is maintained for the target categories without requiring the computational resources of a full large-scale network.
Solution Approach 2:
The patent implements partial action by deploying only the necessary portion of the neural network capacity required for each specific application. Instead of always running the full large network, the system activates only the relevant subtask-specific network, consuming minimal computational resources while achieving sufficient accuracy for the given task.
3Adaptability or versatility
If the entire neural network is run for every application, then all categories can be recognized, but resource constraints prevent deployment on resource-constrained devices
Solution Approach 1:
The patent creates a universal framework where multiple subtask-specific networks share a common architecture and training methodology. Each smaller network is designed to be deployed on resource-constrained devices, and the collection of these networks provides universal coverage for recognizing all categories across different applications through selective activation.
Data Source
AI summary
At training time, a base neural network can be trained to perform each of a plurality of basis subtasks included in a total set of basis subtasks (e.g., individually or some combination thereof). Next, a description of a desired combined subtask can be obtained. Based on the description of the combined subtask, a mask generator can produce a pruning mask which is used to prune the base neural network into a smaller combined-subtask-specific network that performs only the two or more basis subtasks included in the combined subtask.