DNN Modularization for Parallel Loading and Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for loading and executing Deep Neural Networks (DNNs) on mobile devices result in increased latency and inefficient hardware utilization due to sequential loading and execution of multiple layers, leading to prolonged memory occupation and battery usage.
Innovation Solution
The method involves DNN modularization by splitting the DNN model into sub-models based on device and model parameters, determining the performance level of each sub-model, and selecting sub-models for execution that meet a performance threshold, allowing for parallel loading and execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the DNN model is loaded and executed sequentially layer by layer, then the model structure is simple to implement, but the loading time and execution time increase significantly causing latency
Solution Approach 1:
The patent divides the DNN model into multiple sub-models (first sub-model, second sub-model, etc.) that can be loaded and executed independently in parallel. This segmentation allows different parts of the neural network to process data simultaneously, dramatically reducing both loading time and execution time while maintaining the original model architecture through proper integration of sub-model outputs
2Reliability
If the complete DNN model is loaded into memory, then the model can be executed, but memory is occupied for a longer time causing inefficient operation of other applications
Solution Approach 1:
The patent segments the DNN model into multiple sub-models that can be loaded separately into memory. Only the necessary sub-models are loaded at any given time based on the specific task requirements, reducing overall memory occupation duration and allowing other applications to operate more efficiently while maintaining reliable DNN execution
Solution Approach 2:
The patent performs preliminary splitting of the DNN model into sub-models before execution. This preliminary action allows the system to prepare and load only the required sub-models in advance, reducing memory occupation time during actual operation and decreasing battery consumption while ensuring reliable model execution
3Reliability
If the processor operates for a longer time to execute the DNN model, then the model can be executed completely, but battery is over used
Solution Approach 1:
The patent divides the DNN model into multiple sub-models that can be executed in parallel, significantly reducing the total execution time and thereby decreasing battery consumption while ensuring complete and reliable model execution through coordinated processing of all sub-models
Solution Approach 2:
The patent enables continuous parallel execution of multiple sub-models simultaneously rather than sequential execution. This continuity of useful action maximizes processing efficiency, reducing the overall time the processor needs to operate and thereby conserving battery energy while maintaining complete model execution
Data Source
AI summary
A method of deep neural network (DNN) modularization for optimal loading includes receiving, by an electronic device, a DNN model for execution, obtaining, by the electronic device, a plurality of parameters associated with the electronic device and a plurality of parameters associated with the DNN model, determining, by the electronic device, a number of sub-models of the DNN model and a splitting index, based on the obtained plurality of parameters associated with the electronic device and the obtained plurality of parameters associated with the DNN model, and splitting, by the electronic device, the received DNN model into a plurality of sub-models, based on the determined number of sub-models of the DNN model and the determined splitting index.