The application discloses a chart visual
question answering method based on dynamic routing and low rank mixing and a related device, relates to the technical field of visual
question answering, and comprises the following steps: acquiring
original data containing a chart to be processed and a text question, and generating a corresponding answer by
processing through a pre-trained double-channel fusion model. In the model, a visual
encoder is provided to extract semantic features of a complex
layout of the chart, a text
encoder is used to accurately convert a text question vector, and the two improve the
feature matching degree of the chart and the question; a chart mixed connection structure dynamically fuses visual features of a
data graph and a data table, and enhances the cross-type chart adaptability; a chart low rank mixing structure fuses related vectors by means of a LoRA routing network, realizes low rank
adaptation and dynamic fusion in combination with a large
language model layer output, reduces training parameters, and improves zero sample
question answering capability; and a two-stage training strategy reduces the memory loss. The method effectively improves the generality, accuracy and zero sample performance of chart visual question answering, and reduces the training cost.