The invention discloses a self-
adaptive learning question answering system and method based on multi-
modal interaction, and particularly relates to the technical field of self-
adaptive learning question answering. By constructing a
modal recognition and preprocessing module,
standardization and structuralization of multi-
modal input of texts, voices, images and the like are realized; establishing a cross-round modal
memory map through
time sequence coding and map modeling; dynamically updating a user portrait in combination with user
interaction history and current modal characteristics; in the question and answer
generation process, current input, a user portrait and a historical graph are fused, intermediate
semantic representation is generated through a context enhancement module, and the intermediate
semantic representation is combined with a
knowledge base to generate answers; meanwhile, a modal confidence degree dynamic evaluation mechanism is introduced, and weights are distributed according to the input quality, the user
adaptation degree and the context correlation; and finally, incremental optimization is carried out on the graph structure, the user portrait and the question and answer strategy through a
user feedback driving
system, and multi-round, multi-mode and self-adaptive intelligent question and answer interaction is realized.