Neural Network Layer Reuse for Lower-Cost Large Model Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The deployment and inference of large models in various scenarios face high costs, harsh training conditions, and significant deployment difficulties due to their large parameter scale, leading to inefficient use of storage and computational resources.
Innovation Solution
A method involving the retrieval and cyclic reuse of hyperparameters from a storage unit to execute sequential computational subtasks using an operator unit, simulating deep processing of network layers, thereby reducing storage space and computational overhead while maintaining model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large model parameter scale is used to improve model accuracy and processing capability, then model performance is improved, but storage space occupation and deployment costs increase significantly
Solution Approach 1:
The patent creates a virtual copy of network layer parameters through operator reuse. Instead of storing multiple physical copies of parameters for different network layers, the system stores a single parameter set and virtually replicates it by reusing the same parameters across multiple computational subtasks, thereby reducing storage space while maintaining the functional equivalence of large-scale models
Solution Approach 2:
The patent makes parameters universal by enabling a single set of network layer parameters to serve multiple functions. The same parameters are reused across different computational subtasks and network layer simulations, allowing one parameter set to perform the work of multiple parameter sets, thus reducing overall storage requirements while maintaining model accuracy
2Productivity
If large model parameter scale is used to improve model performance, then processing capability is enhanced, but deployment difficulty increases
Solution Approach 1:
The patent uses virtual copying to simplify deployment. By creating a virtual representation of multiple network layers through parameter reuse rather than physically deploying multiple complete model copies, the system reduces the complexity of model deployment while maintaining processing capability equivalent to large-scale models
Solution Approach 2:
The patent segments the computational task into multiple subtasks that can be executed sequentially using the same parameters. This segmentation allows the complex processing capability of large models to be achieved through coordinated simple operations, reducing deployment complexity while maintaining overall processing power
3Measurement precision
If traditional model execution is used to maintain model accuracy, then computational precision is preserved, but computational overhead and resource consumption increase
Solution Approach 1:
The patent eliminates redundant computational copies by reusing parameters across multiple subtasks. Instead of performing separate computational passes with duplicate parameter sets, the system virtually replicates parameter functionality through reuse, reducing computational overhead while maintaining the precision that would result from multiple processing passes
Solution Approach 2:
The patent maintains continuous useful computation by seamlessly reusing parameters across sequential subtasks. The computational process continues without interruption or redundant recalculations, as each subtask builds upon the previous one using the same parameter set, thereby reducing total computational overhead while preserving accuracy through continuous processing
Data Source
AI summary
The task execution method includes: retrieving, from a storage unit, a hyperparameter of a target network layer in a target model; executing, using an operator unit, a first computational subtask in a computational task, according to the hyperparameter of the target network layer, so as to obtain a first feature output by the target network layer; executing, in response to reusing the hyperparameter of the target network layer, a second computational subtask in the computational task using the operator unit based on the first feature retrieved from the storage unit, so as to obtain a second feature output by the target network layer, where the first computational subtask and the second computational subtask are subtasks sequentially executed in the computational task; and determining a model output result of the target model using the operator unit based on the second feature retrieved from the storage unit.