Multi-person interactive teaching system and method based on LLM and virtual reality large-space technology

Abstract

The invention belongs to the technical field of teaching, and particularly relates to a multi-person interactive teaching system and method based on LLM and virtual reality large space technology, and the system comprises a teaching scene creation and management module which is used for creating and managing a teaching scene; the intelligent teaching auxiliary module is used for enabling students to intelligently learn in the teaching scene created by the teaching scene creation and management module through virtual reality equipment; and the multi-person cooperation interaction module is used for enabling a plurality of students to carry out cooperation interaction in the virtual teaching scene. Through the virtual reality large-space technology, students can immerse in a vivid teaching scene in the whole body and naturally interact with the virtual environment, the participation degree and the initiative of learning are greatly improved, compared with a traditional teaching mode, the students are no longer passive knowledge receivers but active explorers and practitioners, and the teaching experience of the students is greatly improved. And the learning effect is more obvious.

Description

Technical Field

[0001] This invention relates to the field of teaching technology, specifically to a multi-user interactive teaching system and method based on LLM and virtual reality large space technology. Background Technology

[0002] An interactive teaching system is a platform that integrates various online teaching functions, supporting real-time classroom interaction between teachers and students via the internet. It typically includes features such as live video streaming, courseware presentation, real-time Q&A, online quizzes, and classroom management, aiming to enhance students' learning experience and help teachers better impart knowledge through technological means.

[0003] However, existing interactive teaching systems typically rely solely on students' imagination to engage with the text, failing to achieve an immersive experience and thus making the teaching somewhat monotonous and limited. Therefore, this paper proposes a multi-user interactive teaching system and method based on LLM and virtual reality large-space technology. Summary of the Invention

[0004] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution:

[0005] A multi-user interactive teaching system based on LLM and virtual reality large-space technology, comprising:

[0006] The teaching scenario creation and management module is used to create and manage teaching scenarios;

[0007] The intelligent teaching assistance module enables students to engage in intelligent learning within teaching scenarios created by the teaching scenario creation and management module using virtual reality devices.

[0008] The multi-person collaborative interaction module is used to enable multiple students to collaborate and interact in a virtual teaching scenario.

[0009] The teaching assessment and feedback module is used to record and assess students' learning status in virtual teaching scenarios;

[0010] The intelligent teaching assistance module includes:

[0011] The intelligent question-and-answer module is used to answer students' questions when they are obtained through voice and sign language.

[0012] The knowledge explanation module is used to provide detailed explanations and descriptions of key and difficult knowledge points in teaching, enabling LLM to provide detailed explanations and descriptions.

[0013] The learning path planning module is used to enable LLM to create personalized learning paths for each student based on their learning progress, knowledge acquisition, and learning goals.

[0014] As a preferred embodiment of the multi-user interactive teaching system based on LLM and virtual reality large-space technology described in this invention, the teaching scene creation and management module includes:

[0015] The scene editing module provides teachers with a visual scene editing tool, enabling them to add virtual objects, set environmental parameters, and write interactive scripts to customize personalized teaching scenes based on teaching content and objectives.

[0016] The scenario resource library is used to establish a teaching scenario resource library, storing various created teaching scenario templates and materials. Teachers can select suitable scenarios from the scenario resource library for reuse and modification, and can also upload their own excellent scenarios to the resource library to achieve resource sharing.

[0017] As a preferred embodiment of the multi-user interactive teaching system based on LLM and virtual reality large-space technology described in this invention, the teaching scene creation and management module further includes:

[0018] The scene publishing module is used to allow teachers to publish scenes to the teaching management system after they have completed scene creation and editing, so that students can access them.

[0019] The scenario update module enables teachers to update and maintain published scenarios, ensuring the timeliness and accuracy of teaching content.

[0020] As a preferred embodiment of the multi-user interactive teaching system based on LLM and virtual reality large-space technology described in this invention, the intelligent question-answering module includes:

[0021] The sound acquisition module is used to collect the students' voices;

[0022] The text conversion module is used to convert the sound captured by the sound acquisition module into text;

[0023] The text storage module is used to store various questions;

[0024] The text comparison module is used to compare the text converted by the text conversion module with the text stored in the text storage module.

[0025] The text judgment module is used to determine whether the text converted by the text conversion module exists in the storage module. If it does, the student question will be obtained.

[0026] As a preferred embodiment of the multi-user interactive teaching system based on LLM and virtual reality large-space technology described in this invention, the intelligent question-answering module further includes:

[0027] The gesture capture module is used to capture students' gestures.

[0028] The sign language storage module is used to store various sign languages;

[0029] The sign language comparison module is used to compare the gestures collected by the gesture acquisition module with the sign language stored in the sign language storage module.

[0030] The sign language judgment module is used to determine whether the gestures collected by the gesture acquisition module are similar to those in the sign language storage module. If they are similar, the meaning expressed by the gestures will be obtained.

[0031] As a preferred embodiment of the multi-user interactive teaching system based on LLM and virtual reality large-space technology described in this invention, the intelligent question-answering module further includes:

[0032] A knowledge base is used to store various questions and their corresponding answers.

[0033] The playback module is used to play the corresponding answers from the knowledge base aloud based on the student's question.

[0034] As a preferred embodiment of the multi-user interactive teaching system based on LLM and virtual reality large-space technology described in this invention, the multi-user collaborative interaction module includes:

[0035] The multi-person synchronization module is used to ensure that the positions, actions and states of multiple students in the virtual reality scene are synchronized in real time according to the multi-person interaction framework, so that students can see the real-time dynamics of other students in the scene, and realize multiple people to learn and collaborate in the same virtual space at the same time.

[0036] The collaborative task module is used to design various collaborative learning tasks so that students can divide the work and cooperate to complete the task objectives together.

[0037] The social interaction module supports voice chat, emoticon communication, and action interaction between students to enhance the fun of social interaction.

[0038] As a preferred embodiment of the multi-user interactive teaching system based on LLM and virtual reality large-space technology described in this invention, the teaching evaluation and feedback module includes:

[0039] The learning process recording module is used to automatically record various behavioral data of students during the learning process, including learning time, movement trajectory, operation steps, number of interactions with virtual objects, questions and answers, so as to provide rich information for teaching evaluation.

[0040] The automatic assessment and feedback module is used to automatically assess students' learning outcomes based on set assessment indicators and students' learning process data, and generate detailed assessment reports. At the same time, it provides students with personalized feedback suggestions to help them understand their learning situation and identify areas for improvement.

[0041] A multi-user interactive teaching method based on LLM and virtual reality large-space technology includes the following specific steps:

[0042] Step 1: The scene editing module provides teachers with a visual scene editing tool, allowing them to add virtual objects, set environmental parameters, and write interactive scripts to customize personalized teaching scenes based on teaching content and objectives. Next, a teaching scene resource library is established, storing various created teaching scene templates and materials. Teachers can select suitable scenes from the resource library for reuse and modification, and upload their own excellent scenes to achieve resource sharing. Then, the scene publishing module allows teachers to publish their scenes to the teaching management system for student access after creation and editing. After publishing, the scene update module enables teachers to update and maintain the published scenes, ensuring the timeliness and accuracy of the teaching content.

[0043] Step Two: The student's voice is captured by the sound acquisition module. After acquisition, the voice is converted into text by the text conversion module. Then, the text comparison module compares the converted text with the text stored in the text storage module. Following the comparison, the text judgment module determines if the converted text exists in the storage module. If a similar text exists, the student's question is obtained. Simultaneously, if a student cannot speak, the gesture acquisition module captures the student's gestures. After acquisition, the gestures are compared with the sign language stored in the sign language storage module by the sign language comparison module. Following the comparison, the sign language judgment module... The block determines whether the gestures collected by the gesture acquisition module are similar to those in the sign language storage module. If they are, the meaning of the gestures is obtained. Then, the text judgment module determines whether the meaning of the gestures is similar to those in the storage module. If they are, the student's question is obtained. After obtaining the question, the corresponding answer is searched in the knowledge base. Next, the playback module plays the corresponding answer in the knowledge base according to the student's question. Then, the knowledge explanation module provides detailed explanations and descriptions of the key and difficult knowledge in the teaching, enabling LLM to provide detailed explanations and descriptions. Finally, the learning path planning module creates a personalized learning path for each student based on their learning situation, knowledge mastery level and learning goals.

[0044] Step 3: Through the multi-person synchronization module, based on the multi-person interaction framework, the positions, actions, and states of multiple students in the virtual reality scene are synchronized in real time, so that students can see the real-time dynamics of other students in the scene, realizing that multiple people can learn and collaborate in the same virtual space at the same time. Next, through the collaborative task module, various collaborative learning tasks will be designed so that students can divide the work and cooperate to complete the task objectives together. After that, through the social interaction module, voice chat, facial expression communication, and action interaction between students will be supported to enhance the fun of social interaction.

[0045] Step 4: The learning process recording module automatically records various behavioral data of students during the learning process, providing rich information for teaching evaluation. Then, the automatic evaluation and feedback module automatically evaluates students' learning outcomes based on the set evaluation indicators and students' learning process data, and generates detailed evaluation reports. At the same time, it provides students with personalized feedback suggestions to help them understand their learning situation and clarify the direction for improvement.

[0046] Compared with existing technologies:

[0047] 1. High immersion and interactivity: Through virtual reality large-space technology, students can be fully immersed in realistic teaching scenarios and interact naturally with the virtual environment, which greatly improves their participation and initiative in learning. Compared with traditional teaching methods, students are no longer passive recipients of knowledge, but active explorers and practitioners, resulting in more significant learning outcomes.

[0048] 2. Intelligent teaching support: The introduction of LLM provides powerful intelligent support for the teaching process. It can understand students' questions, provide accurate and detailed answers, and develop personalized learning paths based on students' learning situations to achieve individualized instruction. This not only reduces the teaching burden on teachers, but also improves the pertinence and effectiveness of teaching.

[0049] 3. Multi-person collaborative learning mode: It supports multiple people to conduct collaborative learning in a virtual reality environment at the same time, breaking the time and space limitations of the traditional classroom; students can form groups in the virtual space to complete learning tasks together, and cultivate teamwork spirit and communication skills; this collaborative learning mode is more in line with the needs of modern society for talents and helps students better adapt to future work and life.

[0050] 4. Real-time teaching assessment and feedback: It can record students' learning process data in real time and automatically assess students' learning outcomes according to preset assessment indicators; at the same time, students can also understand their learning status based on the assessment report, make targeted improvements, and form a good learning loop.

[0051] 5. Wide applicability: Supports both voice and sign language for questioning, catering to both general students and special needs students, enabling them to participate in the questioning process, opening their hearts, and fostering their learning confidence. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0054] This invention provides a multi-user interactive teaching system based on LLM and virtual reality large-space technology. Please refer to [link / reference]. Figure 1 ;

[0055] It includes: a teaching scenario creation and management module for creating and managing teaching scenarios; an intelligent teaching assistance module for enabling students to conduct intelligent learning in teaching scenarios created by the teaching scenario creation and management module through virtual reality devices; a multi-person collaborative interaction module for enabling multiple students to collaborate and interact in virtual teaching scenarios; and a teaching evaluation and feedback module for recording and evaluating students' learning status in virtual teaching scenarios.

[0056] The teaching scenario creation and management module includes: a scenario editing module, which provides teachers with visual scenario editing tools, enabling them to add virtual objects, set environmental parameters, and write interactive scripts to customize personalized teaching scenarios based on teaching content and objectives; a scenario resource library, which stores various created teaching scenario templates and materials, allowing teachers to select suitable scenarios for reuse and modification, and upload their own excellent scenarios to the resource library for resource sharing; a scenario publishing module, which allows teachers to publish scenarios to the teaching management system for student access after creation and editing; and a scenario update module, which enables teachers to update and maintain published scenarios to ensure the timeliness and accuracy of teaching content.

[0057] The intelligent teaching support module includes: an intelligent question-and-answer module, which can answer students' questions when they are obtained through voice and sign language; a knowledge explanation module, which enables LLM to provide detailed explanations and elaborations on key and difficult knowledge points in teaching; and a learning path planning module, which enables LLM to create personalized learning paths for each student based on their learning situation, knowledge mastery, and learning goals.

[0058] The intelligent question-and-answer module includes: a voice acquisition module for capturing students' voices; a text conversion module for converting the voices captured by the voice acquisition module into text; a text storage module for storing various questions; a text comparison module for comparing the text converted by the text conversion module with the text stored in the text storage module; a text judgment module for determining whether the text converted by the text conversion module is similar to the text stored in the storage module, and if so, obtaining the student's question; a gesture acquisition module for capturing students' gestures; a sign language storage module for storing various sign languages; a sign language comparison module for comparing the gestures captured by the gesture acquisition module with the sign languages ​​stored in the sign language storage module; a sign language judgment module for determining whether the gestures captured by the gesture acquisition module are similar to the sign language stored in the sign language storage module, and if so, obtaining the meaning expressed by the gesture; a knowledge base for storing various questions and their corresponding answers; and a playback module for playing the corresponding answers in the knowledge base aloud based on the student's question.

[0059] The multi-person collaborative interaction module includes: a multi-person synchronization module, which ensures that the positions, actions, and states of multiple students in the virtual reality scene are synchronized in real time according to the multi-person interaction framework, so that students can see the real-time dynamics of other students in the scene and realize multiple people learning and collaborating in the same virtual space at the same time; a collaborative task module, which is used to design various collaborative learning tasks so that students can divide the work and cooperate to complete the task objectives together; and a social interaction module, which supports voice chat, facial expression communication, and action interaction between students to enhance the fun of social interaction.

[0060] The teaching assessment and feedback module includes: a learning process recording module, which automatically records various behavioral data of students during the learning process, including learning time, movement trajectory, operation steps, number of interactions with virtual objects, and questions and answers, so as to provide rich information for teaching assessment; and an automatic assessment and feedback module, which automatically assesses students' learning outcomes based on set assessment indicators and students' learning process data, generates detailed assessment reports, and provides students with personalized feedback suggestions to help them understand their learning situation and clarify areas for improvement.

[0061] A multi-user interactive teaching method based on LLM and virtual reality large-space technology includes the following specific steps:

[0062] Step 1: The scene editing module provides teachers with a visual scene editing tool, allowing them to add virtual objects, set environmental parameters, and write interactive scripts to customize personalized teaching scenes based on teaching content and objectives. Next, a teaching scene resource library is established, storing various created teaching scene templates and materials. Teachers can select suitable scenes from the resource library for reuse and modification, and upload their own excellent scenes to achieve resource sharing. Then, the scene publishing module allows teachers to publish their scenes to the teaching management system for student access after creation and editing. After publishing, the scene update module enables teachers to update and maintain the published scenes, ensuring the timeliness and accuracy of the teaching content.

[0063] Step Two: The student's voice is captured by the sound acquisition module. After acquisition, the voice is converted into text by the text conversion module. Then, the text comparison module compares the converted text with the text stored in the text storage module. Following the comparison, the text judgment module determines if the converted text exists in the storage module. If a similar text exists, the student's question is obtained. Simultaneously, if a student cannot speak, the gesture acquisition module captures the student's gestures. After acquisition, the gestures are compared with the sign language stored in the sign language storage module by the sign language comparison module. Following the comparison, the sign language judgment module... The block determines whether the gestures collected by the gesture acquisition module are similar to those in the sign language storage module. If they are, the meaning of the gestures is obtained. Then, the text judgment module determines whether the meaning of the gestures is similar to those in the storage module. If they are, the student's question is obtained. After obtaining the question, the corresponding answer is searched in the knowledge base. Next, the playback module plays the corresponding answer in the knowledge base according to the student's question. Then, the knowledge explanation module provides detailed explanations and descriptions of the key and difficult knowledge in the teaching, enabling LLM to provide detailed explanations and descriptions. Finally, the learning path planning module creates a personalized learning path for each student based on their learning situation, knowledge mastery level and learning goals.

[0064] Step 3: Through the multi-person synchronization module, based on the multi-person interaction framework, the positions, actions, and states of multiple students in the virtual reality scene are synchronized in real time, so that students can see the real-time dynamics of other students in the scene, realizing that multiple people can learn and collaborate in the same virtual space at the same time. Next, through the collaborative task module, various collaborative learning tasks will be designed so that students can divide the work and cooperate to complete the task objectives together. After that, through the social interaction module, voice chat, facial expression communication, and action interaction between students will be supported to enhance the fun of social interaction.

[0065] Step 4: The learning process recording module automatically records various behavioral data of students during the learning process, providing rich information for teaching evaluation. Then, the automatic evaluation and feedback module automatically evaluates students' learning outcomes based on the set evaluation indicators and students' learning process data, and generates detailed evaluation reports. At the same time, it provides students with personalized feedback suggestions to help them understand their learning situation and clarify the direction for improvement.

[0066] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A multi-user interactive teaching system based on LLM and virtual reality large-space technology, characterized in that, include: The teaching scenario creation and management module is used to create and manage teaching scenarios; The intelligent teaching assistance module enables students to engage in intelligent learning within teaching scenarios created by the teaching scenario creation and management module using virtual reality devices. The multi-person collaborative interaction module is used to enable multiple students to collaborate and interact in a virtual teaching scenario. The teaching assessment and feedback module is used to record and assess students' learning status in virtual teaching scenarios; The intelligent teaching assistance module includes: The intelligent question-and-answer module is used to answer students' questions when they are obtained through voice and sign language. The knowledge explanation module is used to provide detailed explanations and descriptions of key and difficult knowledge points in teaching, enabling LLM to provide detailed explanations and descriptions. The learning path planning module is used to enable LLM to create personalized learning paths for each student based on their learning progress, knowledge acquisition, and learning goals.

2. The multi-user interactive teaching system based on LLM and virtual reality large-space technology according to claim 1, characterized in that, The teaching scenario creation and management module includes: The scene editing module provides teachers with a visual scene editing tool, enabling them to add virtual objects, set environmental parameters, and write interactive scripts to customize personalized teaching scenes based on teaching content and objectives. The scenario resource library is used to establish a teaching scenario resource library, storing various created teaching scenario templates and materials. Teachers can select suitable scenarios from the scenario resource library for reuse and modification, and can also upload their own excellent scenarios to the resource library to achieve resource sharing.

3. The multi-user interactive teaching system based on LLM and virtual reality large-space technology according to claim 2, characterized in that, The teaching scenario creation and management module also includes: The scene publishing module is used to allow teachers to publish scenes to the teaching management system after they have completed scene creation and editing, so that students can access them. The scenario update module enables teachers to update and maintain published scenarios, ensuring the timeliness and accuracy of teaching content.

4. The multi-user interactive teaching system based on LLM and virtual reality large-space technology according to claim 1, characterized in that, The intelligent question-answering module includes: The sound acquisition module is used to collect the students' voices; The text conversion module is used to convert the sound captured by the sound acquisition module into text; The text storage module is used to store various questions; The text comparison module is used to compare the text converted by the text conversion module with the text stored in the text storage module. The text judgment module is used to determine whether the text converted by the text conversion module exists in the storage module. If it does, the student question will be obtained.

5. A multi-user interactive teaching system based on LLM and virtual reality large-space technology according to claim 4, characterized in that, The intelligent question-answering module also includes: The gesture capture module is used to capture students' gestures. The sign language storage module is used to store various sign languages; The sign language comparison module is used to compare the gestures collected by the gesture acquisition module with the sign language stored in the sign language storage module. The sign language judgment module is used to determine whether the gestures collected by the gesture acquisition module are similar to those in the sign language storage module. If they are similar, the meaning expressed by the gestures will be obtained.

6. A multi-user interactive teaching system based on LLM and virtual reality large-space technology according to claim 5, characterized in that, The intelligent question-answering module also includes: A knowledge base is used to store various questions and their corresponding answers. The playback module is used to play the corresponding answers from the knowledge base aloud based on the student's question.

7. A multi-user interactive teaching system based on LLM and virtual reality large-space technology according to claim 1, characterized in that, The multi-user collaborative interaction module includes: The multi-person synchronization module is used to ensure that the positions, actions and states of multiple students in the virtual reality scene are synchronized in real time according to the multi-person interaction framework, so that students can see the real-time dynamics of other students in the scene, and realize multiple people to learn and collaborate in the same virtual space at the same time. The collaborative task module is used to design various collaborative learning tasks so that students can divide the work and cooperate to complete the task objectives together. The social interaction module supports voice chat, emoticon communication, and action interaction between students to enhance the fun of social interaction.

8. A multi-user interactive teaching system based on LLM and virtual reality large-space technology according to claim 1, characterized in that, The teaching assessment and feedback module includes: The learning process recording module is used to automatically record various behavioral data of students during the learning process, including learning time, movement trajectory, operation steps, number of interactions with virtual objects, questions and answers, so as to provide rich information for teaching evaluation. The automatic assessment and feedback module is used to automatically assess students' learning outcomes based on set assessment indicators and students' learning process data, and generate detailed assessment reports. At the same time, it provides students with personalized feedback suggestions to help them understand their learning situation and identify areas for improvement.

9. A multi-user interactive teaching method based on LLM and virtual reality large-space technology, characterized in that, The specific steps are as follows: Step 1: The scene editing module provides teachers with a visual scene editing tool, allowing them to add virtual objects, set environmental parameters, and write interactive scripts to customize personalized teaching scenes based on teaching content and objectives. Next, a teaching scene resource library is established, storing various created teaching scene templates and materials. Teachers can select suitable scenes from the resource library for reuse and modification, and upload their own excellent scenes to achieve resource sharing. Then, the scene publishing module allows teachers to publish their scenes to the teaching management system for student access after creation and editing. After publishing, the scene update module enables teachers to update and maintain the published scenes, ensuring the timeliness and accuracy of the teaching content. Step Two: The student's voice is captured by the sound acquisition module. After acquisition, the voice is converted into text by the text conversion module. Then, the text comparison module compares the converted text with the text stored in the text storage module. Following the comparison, the text judgment module determines if the converted text exists in the storage module. If a similar text exists, the student's question is obtained. Simultaneously, if a student cannot speak, the gesture acquisition module captures the student's gestures. After acquisition, the gestures are compared with the sign language stored in the sign language storage module by the sign language comparison module. Following the comparison, the sign language judgment module... The block determines whether the gestures collected by the gesture acquisition module are similar to those in the sign language storage module. If they are, the meaning of the gestures is obtained. Then, the text judgment module determines whether the meaning of the gestures is similar to those in the storage module. If they are, the student's question is obtained. After obtaining the question, the corresponding answer is searched in the knowledge base. Next, the playback module plays the corresponding answer in the knowledge base according to the student's question. Then, the knowledge explanation module provides detailed explanations and descriptions of the key and difficult knowledge in the teaching, enabling LLM to provide detailed explanations and descriptions. Finally, the learning path planning module creates a personalized learning path for each student based on their learning situation, knowledge mastery level and learning goals. Step 3: Through the multi-person synchronization module, based on the multi-person interaction framework, the positions, actions, and states of multiple students in the virtual reality scene are synchronized in real time, so that students can see the real-time dynamics of other students in the scene, realizing that multiple people can learn and collaborate in the same virtual space at the same time. Next, through the collaborative task module, various collaborative learning tasks will be designed so that students can divide the work and cooperate to complete the task objectives together. After that, through the social interaction module, voice chat, facial expression communication, and action interaction between students will be supported to enhance the fun of social interaction. Step 4: The learning process recording module automatically records various behavioral data of students during the learning process, providing rich information for teaching evaluation. Then, the automatic evaluation and feedback module automatically evaluates students' learning outcomes based on the set evaluation indicators and students' learning process data, and generates detailed evaluation reports. At the same time, it provides students with personalized feedback suggestions to help them understand their learning situation and clarify the direction for improvement.

Images