Shared Tensors Across Neural Network Portions to Reduce Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural networks require significant computing resources for storage and processing, leading to latency issues and resource constraints, and existing methods to reduce weight numbers, such as pruning and quantization, often compromise accuracy or resource usage.
Innovation Solution
Implementing a neural network training method that uses an original neural network to teach a target network to approximate polynomial functions using tensors in multiple portions, reducing the number of weights and memory requirements while maintaining accuracy through knowledge distillation and linear combinations of tensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks use a large number of weights to maintain accuracy, then model precision is improved, but computing resource consumption increases
Solution Approach 1:
The patent merges multiple weight matrices into a single tensor representation. Instead of storing and processing separate weight matrices for different layers, the invention combines them into a unified tensor structure that can be processed more efficiently, reducing memory bandwidth requirements and computational overhead while preserving the functional equivalence of the original multi-matrix structure
Solution Approach 2:
The patent creates a universal tensor representation that can serve multiple functions simultaneously. The same tensor structure is used for both storage and processing operations, eliminating the need for separate data structures for different operations. This multi-functional approach reduces the overall computing resource requirements while maintaining model accuracy
2Reliability
If neural networks store more weight parameters to maintain performance, then model capability is improved, but memory resource usage increases
Solution Approach 1:
The patent combines multiple weight matrices into a single tensor, reducing the total memory footprint. By merging the storage representation of weights across different layers into one unified tensor structure, the invention eliminates redundant memory allocations and reduces the overall quantity of stored parameters while maintaining model performance through efficient tensor operations
3Use of energy by moving object
If pruning and quantization are used to reduce weight numbers, then resource usage is reduced, but model accuracy deteriorates
Solution Approach 1:
The patent creates a simplified tensor copy that represents the full-weight model more efficiently. Instead of pruning or quantizing the actual weights, the invention generates a compressed tensor representation that captures the essential information needed for accurate inference, allowing the model to maintain its accuracy while using fewer computational resources
Data Source
AI summary
Apparatuses, techniques, and/or software to reduce a number of weight parameter values for one or more neural networks by causing said one or more neural networks to use one or more tensors in two or more portions of the one or more neural networks. For example, apparatuses, techniques, processors, and/or software to generate a linear combination of tensors to approximate two or more layers of a neural network, where said same tensors can be used or repeated to approximate different portions of a neural network. In one or more embodiments, said two or more portions are generated and provided to said one or more neural networks using knowledge distillation techniques.