The invention belongs to the technical field of computers, and particularly discloses a method and
system for integrating a code structure into a
large model based on multiple modalities, and the method comprises the steps: obtaining a code attribute graph corresponding to a code text; inputting the code attribute graph to a graph
encoder, and obtaining a graph structure representation output by the graph
encoder; the code text and the graph structure representation are input to a large
language model, a code
processing result output by the large
language model is obtained, and the large
language model is used for
processing the code according to the code
processing task based on the code text and the graph structure representation; wherein the graph structure representation and the
serialization language representation are kept aligned in a representation space, and the
serialization language representation is obtained by performing
serialization processing on a code text by a text
encoder in a large language model. According to the method, the multi-
modal encoder is introduced, so that different
modal information, such as grammar,
semantics,
control flow and data flow, of the code can be fused and processed, and the understanding ability of a
large model to a code structure can be remarkably improved.