LLM Tensor Network Compression for Resource Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) require significant computational resources for training and operation, making it challenging to run them on classical computing devices due to high memory and energy demands.
Innovation Solution
The method involves integrating pre-trained layers of LLM architectures into tensor network layers, which reduces the parameter space by representing some parameters as compilations of others, using tensor networks to efficiently store and process model parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LLM uses massive amounts of parameters for training, then language understanding and generation abilities are improved, but computational resources and energy consumption increase
Solution Approach 1:
The patent segments the LLM parameters into tensor network components, decomposing the weight matrices into a set of tensors connected by contraction operations. This segmentation allows the model to represent parameters more efficiently by sharing computational structures across different layers, reducing the total number of parameters needed while maintaining language understanding and generation capabilities.
Solution Approach 2:
The patent changes the representation of parameters from traditional dense weight matrices to tensor network formulations. By transforming parameters into a different mathematical representation with shared structures, the model achieves parameter efficiency without sacrificing performance, directly addressing the contradiction between model capability and resource consumption.
2Reliability
If LLM uses massive amounts of parameters, then model accuracy is improved, but storage requirements and device complexity increase
Solution Approach 1:
The patent merges parameter representations across different layers by using shared tensors in the tensor network. Instead of storing independent weight matrices for each layer, the model uses a unified tensor network structure where tensors are shared and contracted across layers, significantly reducing storage requirements while maintaining model accuracy.
Solution Approach 2:
The patent transitions from a traditional matrix-based parameter representation to a multi-dimensional tensor network structure. This dimensional transformation allows for more efficient parameter organization and storage, reducing the complexity of device requirements while preserving the model's accuracy through the tensor contraction operations.
3Device complexity
If LLM parameters are compressed using traditional methods, then storage requirements are reduced, but model accuracy deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing the tensor network decomposition of weight matrices. This preliminary processing allows the model to achieve compression benefits during inference without sacrificing accuracy, as the tensor network structure preserves the essential computational relationships needed for accurate language processing.
Solution Approach 2:
The patent creates a compressed representation of the model parameters through tensor network copying, where the essential computational information is replicated in a more efficient format. This copying process maintains model accuracy by preserving the functional relationships between parameters while reducing storage requirements through the tensor network's shared structure.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer implemented method for compressing pre-trained of a large language model (LLM) comprising identifying (S101) layers of the LLM (47) with the weight matrices (48), decomposing (S 104a) the weight matrices (48) of the LLM (47) into a tensor network (49), compressing (S104b) the tensor network (49), and storing (S104c) the compressed tensor network (49) in a data storage unit (40).