Sub-Kernel Searching Module for Adaptive Edge Neural Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network models on edge devices face limitations due to restricted computational capacity and uniform parameter usage across varying image difficulty levels, leading to inconsistent performance and requiring significant human intervention for improvement.
Innovation Solution
The method employs a Sub-kernel Searching Module (SSM) to generate sub-architectures of neural networks tailored to edge device capabilities and environmental constraints, optimizing computational load and sample difficulty levels through a Scene Overviewing Module and Sub-Kernel Network Generator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the neural network model uses uniform parameters for all images, then the device complexity is reduced, but the measurement precision of image classification deteriorates for difficult images
Solution Approach 1:
The patent implements dynamic architecture selection by introducing a gate network that adaptively chooses between different sub-architectures based on input image characteristics. The system transitions from static uniform parameters to dynamic parameter selection, allowing the model complexity to vary according to the difficulty level of each input image, thereby resolving the contradiction between device complexity and measurement precision.
Solution Approach 2:
The patent divides the single neural network into multiple sub-architectures with different capacities, where each sub-architecture is optimized for specific types of images. Instead of using the same uniform parameters for all inputs, the system applies different local architectures (sub-networks) to different regions of the input space based on image difficulty, improving classification accuracy for difficult images while maintaining simplicity for easy images.
2Measurement precision
If the neural network model capacity is increased to improve detection performance, then the measurement precision improves, but the device complexity and computational load increase
Solution Approach 1:
The patent segments the large neural network into multiple smaller sub-architectures with different capacities. Instead of deploying one large complex model that handles all cases, the system divides the model into manageable segments (sub-networks) that can be selectively activated. The gate network routes inputs to appropriate sub-architectures, reducing the average computational load while maintaining high detection accuracy for difficult images that require larger model capacity.
Solution Approach 2:
The patent applies partial action by using only the necessary model capacity for each specific input. For easy images, smaller sub-architectures suffice, avoiding the excessive computational load of large models. For difficult images, the system can activate larger sub-architectures or combine multiple sub-architectures to provide the excessive capacity needed for high-accuracy detection, thereby optimizing the trade-off between measurement precision and device complexity.
3Measurement precision
If manual expert intervention is used to search for hard samples and optimize the model, then the measurement precision improves, but the loss of time and productivity decrease
Solution Approach 1:
The patent implements self-service by enabling the neural network system to automatically identify hard samples and optimize its own architecture without external expert intervention. The gate network and sub-architecture selection mechanism allow the system to autonomously determine which samples are difficult and route them to appropriate sub-architectures for improved learning. This automated self-optimization process eliminates the need for manual expert time while maintaining or improving measurement precision.
Solution Approach 2:
The patent incorporates feedback mechanisms where the performance on hard samples is continuously monitored and used to adjust the model architecture and training process. The system uses feedback from classification results to identify hard samples, adjust the gating mechanism, and refine sub-architecture selections. This automated feedback loop enables continuous improvement of measurement precision without requiring ongoing manual expert intervention, thereby reducing time loss and increasing productivity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for optimizing an on-device neural network model by using a Sub-kernel Searching Module is provided. The method includes steps of a learning device (a) if a Big Neural Network Model having a capacity capable of performing a targeted task by using a maximal computing power of an edge device has been trained to generate a first inference result on an input data, allowing the Sub-kernel Searching Module to identify constraint and a state vector corresponding to the training data, to generate architecture information on a specific sub-kernel suitable for performing the targeted task on the training data, (b) optimizing the Big Neural Network Model according to the architecture information to generate a specific Small Neural Network Model for generating a second inference result on the training data, and (c) training the Sub-kernel Searching Module by using the first and the second inference result.