Large Model Collaborative Weight Processing for Lower-Cost Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large models require significant computing resources and face high training and inference costs due to their large number of parameters, making deployment challenging.
Innovation Solution
The method involves dividing the weight parameters of large models into basic and collaborative weights, using a matrix multiplication mechanism to determine smaller sub-weights for collaborative tasks, and fusing these with basic features to reduce computing overhead and improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large models with many parameters are used to process complex tasks, then model capability and task performance are improved, but computing costs and energy consumption increase significantly
Solution Approach 1:
The patent segments the model parameters into two distinct parts: basic parameters that remain fixed and collaborative parameters that are dynamically adjusted. This segmentation allows the model to maintain high capability through the basic parameters while reducing computing costs by only processing collaborative parameters during inference, thus resolving the contradiction between model capability and computing cost.
Solution Approach 2:
The patent changes the parameter representation by decomposing full model parameters into basic parameters and collaborative parameters. The collaborative parameters are derived through matrix multiplication of two smaller parameter matrices, enabling the model to adapt to different tasks without processing all original parameters, thereby reducing energy consumption while maintaining adaptability.
2Measurement precision
If large models with many parameters are deployed, then model accuracy and performance are improved, but deployment difficulty increases due to high resource requirements
Solution Approach 1:
By segmenting parameters into basic and collaborative components, the patent enables deployment of accurate models on devices with limited resources. The basic parameters provide the foundation for model accuracy, while the smaller collaborative parameters can be efficiently processed on resource-constrained devices, thus reducing deployment difficulty without sacrificing accuracy.
Solution Approach 2:
The patent creates a simplified copy of the full model parameters through collaborative parameters that are generated on-demand via matrix multiplication. This copying mechanism allows the model to achieve high accuracy using compact representations that are easier to deploy, reducing the computational burden on deployment devices.
3Productivity
If collaborative weight is processed using matrix multiplication mechanism, then computing overhead is reduced, but the complexity of weight processing increases
Solution Approach 1:
The patent segments the collaborative weight processing into two separate matrix multiplications instead of one large matrix multiplication. This segmentation reduces the computational complexity from O(n³) for a single large matrix to O(nk²) for two smaller matrices, improving computing efficiency while the modular structure makes the processing more manageable and less complex.
Data Source
AI summary
A task execution method and apparatus for a large model, an electronic device, and a storage medium are provided, which relate to a field of artificial intelligence, and in particular to fields of deep learning and large model technologies. The method includes: executing, according to a target feature to be processed, a collaborative computing task using a target computing unit, where the collaborative computing task includes a first collaborative task and a second collaborative task, the first collaborative task is used to process the target feature to be processed and a first collaborative sub-weight to obtain an intermediate collaborative feature, the second collaborative task is used to process the intermediate collaborative feature and a second collaborative sub-weight to obtain a target collaborative feature; and fusing a target basic feature and the target collaborative feature to obtain a next target feature to be processed