Large model pruning method, device and equipment for end side model training acceleration

By pruning the Transformer model on the end-side device, constructing sparse graphs and generating a self-attention mask matrix, the computing and storage bottlenecks of large-model training on resource-constrained devices are solved, and efficient model training and inference are achieved.

CN119740624APending Publication Date: 2025-04-01NAT UNIV OF DEFENSE TECH
0 Cites 0 Cited by

Patent Information

Application Number
CN202411894221.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-20
Publication Date
2025-04-01

AI Technical Summary

Technical Problem

On resource-constrained end-side devices, large-scale model training faces limitations in computing resources, storage space and energy consumption. In particular, the multi-head self-attention mechanism of the Transformer model leads to a sharp increase in computing complexity and memory consumption.

Method used

A large-model pruning method is adopted to construct sparse graphs based on token sequences, prune unimportant parameters in the Transformer model, generate a self-attention mask matrix, and retrain the model to reduce the computational amount and storage requirements.

Benefits of technology

Implement efficient model training and inference under limited resources, significantly reduce computational complexity and memory consumption, and improve the efficiency of end-side model training.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to a large model pruning method, device and equipment for end-side model training acceleration. The method points out that the main bottleneck of end-side model training is all-to-all attention operation on an input sequence. A complete graph is constructed through all-to-all operation, that is, directed edges are established between the token of each sequence and all other tokens, the complete graph is of a dense structure, but different tokens have differentiated similarity, according to the differentiated similarity of the token sequences, a serialized sparse graph is constructed, serialized and universal Transform sparse pruning is achieved, and the sparse pruning efficiency is improved. A set of flexible sparse pruning tool is provided for Transform model trainers, and the calculation amount and storage requirements of the model are reduced by cutting off unimportant parameters in the model, so that efficient model training and reasoning are realized under limited resources, and acceleration of end side model training is realized.
Need to check novelty before this filing date? Find Prior Art