This invention relates to the field of
deep learning technology, particularly to a
Transformer optimization method and
system based on a token graph-based query propagation mechanism. The method comprises an input
processing layer, a dynamic graph construction layer, a propagation
control layer, a computational optimization layer, a feedback adjustment layer, and an output integration layer. This invention captures the
contextual similarity or relationship between tokens by constructing a token-graph (dynamic or
static structure) and introduces a stochastic propagation of Q vectors mechanism. Only the seed token generates a complete Q vector, while the remaining tokens generate approximate Q vectors through graph propagation. Furthermore, K and V are projected only onto the
central node, and a local error control feedback module is included. This solves the problems of
computational resource constraints, memory limitations, and accuracy stability in long
sequence processing of Transformers. It provides a practical and feasible technical path for its application in complex tasks such as long document understanding, multi-turn dialogue, and
code generation.