A tutoring question-answering method and device based on multimodal interaction
By collecting and encoding multimodal data, inputting it into a large model of fine-tuning data in the education field, the teaching reply content is generated, which solves the problem of insufficient interactiveness of traditional tutoring Q&A systems and improves interactivity and teaching effect.
CN118503496BActive Publication Date: 2024-11-08DMAI (GUANGZHOU) CO LTD
View PDF 0 Cites 0 Cited by
Patent Information
- Application Number
- CN202410931652.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-12
AI Technical Summary
Technical Problem
The traditional tutoring Q&A system is limited to text interaction and cannot make full use of multimodal information such as video and audio, limiting the richness and efficiency of the interaction.
Method used
By collecting user video, audio and active input data, these multimodal data are encoded and processed, feature vectors are generated, and inputted into a pre-trained multimodal model that has been fine-tuned by education data, obtain the execution strategy, and generate Targeted teaching replies.
Benefits of technology
Make full use of multimodal information to improve the interactivity and teaching effect of the tutoring Q&A system.
✦ Generated by Eureka AI based on patent content.
Smart Images

Figure CN118503496B_ABST
Abstract
The present invention discloses a tutoring question-and-answer method and device based on multimodal interaction, comprising: firstly, by collecting the user's video, audio and active input data, encoding and processing the multimodal data, and generating corresponding feature vectors. These vectors are input into a multimodal large model that is pre-trained and fine-tuned with education field data to obtain an execution strategy for the current problem. Finally, targeted teaching reply content is generated based on the strategy. Such a design makes full use of multimodal information and improves the interactivity and teaching effect of the tutoring question-and-answer system.
Need to check novelty before this filing date? Find Prior Art
