Immersive teaching system and method based on virtual reality technology

Abstract

The invention relates to the technical field of teaching systems, in particular to an immersive teaching system based on a virtual reality technology and a method thereof, the immersive teaching system is fused with the virtual reality technology, and a brand new teaching mode is created. The system captures gestures and voices of students through a data acquisition module to form a real-time behavior data stream; the cloud data processing module intelligently analyzes the data, and accurately judges the learning and emotional states of the students based on a personalized emotion recognition model; the central control module comprehensively analyzes the result and generates a control instruction; the virtual interaction module switches a simulation mode or a practical operation mode according to an instruction, and the hardware equipment module provides a corresponding teaching environment, creates a virtual immersion sense during simulation, and provides a real operation platform during practical operation; the system realizes individuation and interaction of teaching, improves the learning effect, and brings revolutionary change to the education field.

Description

Technical Field

[0001] The present invention relates to the technical field of teaching systems, and particularly to an immersive teaching system based on virtual reality technology and its method. Background Art

[0002] With the rapid development of technology, virtual reality technology has been increasingly widely used in the field of education. Traditional teaching methods are often limited to the classroom environment and are difficult to provide students with an immersive learning experience. The teaching system based on virtual reality technology provides an innovative solution to this problem.

[0003] Existing virtual reality teaching systems usually include basic functions such as virtual environment construction, interaction devices, and content display. These systems can improve students' learning interest and participation to a certain extent. However, there are still many deficiencies in the current technical solutions.

[0004] First of all, most systems lack in-depth consideration of students' individual differences. They often adopt unified teaching content and progress, and it is difficult to adapt to the learning abilities and styles of different students. This one-size-fits-all approach leads to uneven learning effects and cannot fully unleash the potential of virtual reality technology.

[0005] Secondly, there are obvious shortcomings in the existing systems in terms of real-time monitoring and analysis of students' learning status. They usually can only collect simple operation data and cannot comprehensively capture students' emotional changes and learning dynamics. This limitation in information acquisition makes it difficult for the system to adjust teaching strategies in a timely manner and cannot provide students with a personalized learning experience.

[0006] Furthermore, current virtual reality teaching systems often focus on content presentation and neglect in-depth analysis and guidance of the learning process. They lack effective learning assessment mechanisms and feedback systems and are difficult to help students accurately grasp their learning progress and deficiencies. This one-way information transmission mode greatly limits the improvement of teaching effects.

[0007] In addition, the existing systems are not flexible enough in the conversion between the virtual environment and actual operations. Most systems either focus on the construction of the virtual environment or on the simulation of actual operations, and it is difficult to achieve seamless connection between the two. This sense of fragmentation not only affects the coherence of learning but also reduces the efficiency of knowledge transfer from virtual to reality.

[0008] Finally, existing virtual reality teaching systems often lack systematicness and integrity. The cooperation between various functional modules is insufficient, and it is difficult to form an organic teaching ecosystem. This fragmented design makes it difficult to effectively integrate teaching resources and cannot fully utilize the comprehensive advantages of virtual reality technology. Summary of the Invention

[0009] In view of the above problems, the present invention proposes an immersive teaching system and method based on virtual reality technology. The system aims to construct an intelligent and personalized virtual teaching environment, and provide a customized learning experience for students through multi-dimensional data collection, real-time analysis and adaptive adjustment.

[0010] The present invention proposes an immersive teaching system based on virtual reality technology, including:

[0011] A data acquisition module, configured to:

[0012] Acquire gesture, voice and action data of students through a variety of sensors;

[0013] Form a real-time dynamic data stream of students' behaviors;

[0014] A cloud data processing module, communicatively connected to the data acquisition module, configured to:

[0015] Receive the student behavior data uploaded by the data acquisition module;

[0016] Based on a pre-constructed personalized emotion recognition model, intelligently analyze the student behavior data;

[0017] Generate analysis results of the learning status and emotions of individual students;

[0018] A central control module, communicatively connected to the cloud data processing module, configured to:

[0019] Receive the analysis results sent by the cloud data processing module;

[0020] Based on the analysis results, make a comprehensive judgment;

[0021] Generate corresponding control instructions;

[0022] A virtual interaction module, communicatively connected to the central control module, configured to:

[0023] Receive the control instructions sent by the central control module;

[0024] According to the control instructions, switch between the simulation mode and the practical operation mode;

[0025] A hardware device module, communicatively connected to the virtual interaction module, configured to:

[0026] In the simulation mode, provide a virtualized immersive teaching environment;

[0027] In the practical operation mode, provide a teaching environment for actual operation.

[0028] Preferably, the cloud data processing module includes:

[0029] A personalized emotion recognition model construction unit, configured to:

[0030] Set teaching contents with different difficulty levels for different students in the virtual simulation teaching platform;

[0031] Group the students according to their mastery of the teaching contents and their actual learning abilities;

[0032] Match teaching contents with different difficulty levels for the grouped students;

[0033] Analyze the acceptance and mastery of different teaching contents by different students;

[0034] Construct an adaptation network between the teaching contents and different individual students;

[0035] An intelligent analysis unit, communicatively connected to the personalized emotion recognition model construction unit, configured to:

[0036] Receive the student behavior data uploaded by the data acquisition module;

[0037] Analyze the student behavior data based on the personalized emotion recognition model;

[0038] Generate the learning status and emotion analysis results of individual students.

[0039] Preferably, the central control module includes:

[0040] A judgment unit, configured to:

[0041] Receive the analysis results sent by the cloud data processing module;

[0042] Judge the learning status and emotion of the students based on the analysis results;

[0043] An instruction generation unit, communicatively connected to the judgment unit, configured to:

[0044] Generate corresponding control instructions according to the judgment results of the judgment unit;

[0045] When the judgment result is that the student's emotion is relatively low or the learning status is not good, generate an instruction to enter the simulation mode;

[0046] When the judgment result is that the student's emotion is relatively excited or the learning status is relatively excited, generate an instruction to enter the practical operation mode.

[0047] Preferably, the virtual interaction module includes:

[0048] A mode switching unit, configured to:

[0049] Receive the control instructions sent by the central control module;

[0050] Switch between the simulation mode and the practical operation mode according to the control instruction;

[0051] A simulation environment generation unit, communicatively connected to the mode switching unit, for:

[0052] In the simulation mode, generate a virtual classroom environment, including an indoor environment, a laboratory environment, and an experimental equipment environment;

[0053] A practical operation environment generation unit, communicatively connected to the mode switching unit, for:

[0054] In the practical operation mode, convert the devices in the hardware device module into an operable actual teaching environment.

[0055] Preferably, the system further includes:

[0056] A curriculum design module, communicatively connected to the central control module, for:

[0057] Receive the student learning status and emotion analysis results sent by the central control module;

[0058] Based on the analysis results, dynamically adjust the curriculum content;

[0059] Retrieve basic knowledge content and professional knowledge content from the database;

[0060] Design personalized curriculum content according to the students' learning situations;

[0061] A curriculum analysis module, communicatively connected to the curriculum design module and the central control module, for:

[0062] Receive the curriculum content information sent by the curriculum design module;

[0063] Receive the student learning status and emotion analysis results sent by the central control module;

[0064] Analyze the students' learning situations, curriculum effects, and user satisfaction;

[0065] Feed back the analysis results to the curriculum design module for further optimizing the curriculum content.

[0066] Preferably, the curriculum analysis module includes:

[0067] A learning emotion recognition unit for recognizing and recording the facial expressions of students;

[0068] A learning content analysis unit for recognizing and recording the actions of students and determining the students' learning progress;

[0069] The course content analysis unit is used to identify and record the answering situations in students' homework exercises and determine the students' understanding of the course;

[0070] The homework analysis unit is used to obtain the action and emotion records of students on the virtual reality device and analyze the students' homework completion situation.

[0071] Preferably, the hardware device module includes:

[0072] The virtual reality helmet is used to display the virtual teaching environment and content;

[0073] The spatial locator is used to track the operation actions of students;

[0074] The virtual reality glove is used to capture the hand actions of students;

[0075] The somatosensory clothing is used to capture the body actions of students;

[0076] Among them, the virtual reality helmet, the spatial locator, the virtual reality glove and the somatosensory clothing are communicatively connected to the virtual interaction module by wired or wireless means.

[0077] Preferably, the system further includes:

[0078] The multimedia data storage module is used to store the multimedia resources required by the system, including pictures, audio, video and animations;

[0079] The multimedia playback module is communicatively connected to the multimedia data storage module and the virtual interaction module and is used for:

[0080] Receiving the playback instruction sent by the virtual interaction module;

[0081] Retrieving the corresponding multimedia resources from the multimedia data storage module;

[0082] Playing the multimedia resources in the virtual teaching environment.

[0083] Preferably, the system further includes:

[0084] The homework module is communicatively connected to the central control module and the virtual interaction module and is used for:

[0085] Receiving the homework setting instruction sent by the central control module;

[0086] Generating the corresponding homework content in the virtual teaching environment;

[0087] Recording the students' homework completion situation;

[0088] Feeding back the homework completion situation to the central control module;

[0089] Among them, the operation module includes an operation setting unit, an answer management unit, and an operation analysis unit. The immersive teaching method based on virtual reality technology, using the said system, includes the following steps:

[0090] S1: Construct a personalized emotion recognition model, specifically including:

[0091] Set teaching contents with different difficulty levels for different students in the virtual simulation teaching platform;

[0092] Divide the students into different groups according to the students' mastery of the teaching contents and their actual learning abilities;

[0093] Match teaching contents with different difficulty levels for the said student groups;

[0094] Analyze the acceptance and mastery of different teaching contents by different students;

[0095] Construct an adaptation network between the teaching contents and different individual students;

[0096] S2: Obtain students' behavior data, specifically including:

[0097] Collect students' gesture information and voice information in real time through multiple sensors;

[0098] Form a real-time dynamic behavior data stream of students in the virtual simulation environment;

[0099] S3: Analyze students' behavior data, specifically including:

[0100] Input the said students' behavior data into the personalized emotion recognition model;

[0101] Generate an analysis result of the learning emotion state of individual students, including positive state and negative state;

[0102] S4: Control the virtual simulation teaching environment, specifically including:

[0103] Send corresponding control instructions to the virtual interaction module according to the analysis result of the said behavior data;

[0104] When the analysis result shows that the students' emotion state is not good and the learning effect is not good, control the hardware device module to enter the simulation mode, so that the students enter the immersive virtual teaching environment;

[0105] When the analysis result shows that the students' emotion state is good and the learning effect is good, control the hardware device module to enter the practical operation mode, so that the students enter the actual teaching environment for operation;

[0106] S5: Dynamically adjust the course content, specifically including:

[0107] Retrieve the corresponding basic knowledge content and professional knowledge content from the database based on the analysis results of the students' learning status and emotions;

[0108] Design personalized course content according to the students' learning situations;

[0109] S6: Analyze the learning effects, specifically including:

[0110] Record and analyze the students' learning behaviors and homework completion situations in the virtual teaching environment;

[0111] Evaluate the course effects and students' satisfaction;

[0112] S7: Optimize the teaching strategies, specifically including:

[0113] Based on the learning effect analysis results, adjust the course content and teaching methods;

[0114] Update the personalized emotion recognition model to improve the accuracy and adaptability of the model.

[0115] Compared with the prior art, the present invention has the following beneficial effects:

[0116] The system of the present invention realizes the all-round optimization of the teaching process through modular design and the application of intelligent algorithms. First, the data acquisition module of the system can comprehensively capture the students' behavior and emotion data, providing a rich information basis for subsequent personalized teaching. The cloud data processing module then uses advanced artificial intelligence technology to deeply analyze these data, so as to accurately grasp the learning status and needs of each student.

[0117] As the decision-making center of the system, the central control module can make intelligent judgments based on the analysis results and timely adjust the teaching strategies. This adaptive mechanism ensures that the teaching content and methods are always in the best match with the actual situation of the students. The collaborative work of the virtual interaction module and the hardware device module creates a highly immersive and interactive learning environment for the students, greatly improving the interest and effect of learning.

[0118] In addition, the system of the present invention also includes modules such as course design and course analysis, forming a complete teaching closed-loop. This comprehensive design can not only evaluate the learning effects in real time, but also continuously optimize the teaching content and methods, realizing the continuous improvement of teaching quality.

[0119] Generally speaking, the system of the present invention constructs an intelligent and personalized teaching ecosystem by integrating cutting-edge technologies such as virtual reality technology, artificial intelligence, and big data analysis. It not only solves many problems existing in the existing virtual reality teaching systems, but also makes remarkable breakthroughs in teaching effects, learning experiences, knowledge transfer, etc. This innovative solution opens up a new direction for the application of technology in the education field and is expected to play an important role in improving teaching quality and promoting educational equity. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] Figure 1 It is the overall block diagram of the system of the present invention.

[0121] Figure 2 It is the logic block diagram of the cloud data processing module of the present invention.

[0122] Figure 3 It is the logic block diagram of the central control module of the present invention.

[0123] Figure 4 It is the logic block diagram of the virtual interaction module of the present invention.

[0124] Figure 5 It is the logic block diagram of the hardware device module of the present invention.

[0125] Figure 6 It is the logic block diagram of the course analysis module of the present invention.

[0126] Figure 7 It is the logic block diagram of the assignment module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0127] Please refer to Figure 1-7 , the present invention provides an immersive teaching system and method based on virtual reality technology. The system aims to create an immersive learning environment through virtual reality technology, thereby improving the learning effects and experiences of students. The following will detail the specific embodiments of the present invention.

[0128] First of all, the immersive teaching system based on virtual reality technology of the present invention includes a data acquisition module 1, a cloud data processing module 2, a central control module 3, a virtual interaction module 4, and a hardware device module 5. These modules are communicatively connected to jointly form a complete teaching system.

[0129] The data acquisition module 1 is the input end of this system. Its main function is to obtain action data such as students' gestures and voices through various sensors, and form a real-time dynamic data stream of students' behaviors. Preferably, these sensors can include but are not limited to cameras, microphones, motion sensors, etc. For example, the camera can capture students' facial expressions and body movements, the microphone can record students' voices, and the motion sensor can track the movement trajectories of students in the virtual environment. The acquisition of these multi-modal data provides a rich information source for subsequent analysis.

[0130] The cloud data processing module 2 is communicatively connected to the data acquisition module 1 and is the core processing unit of this system. This module receives the students' behavior data uploaded by the data acquisition module 1 and conducts intelligent analysis on this data based on a pre-constructed personalized emotion recognition model. Through this analysis, the cloud data processing module 2 can generate the learning status and emotion analysis results of individual students.

[0131] In an embodiment of the present invention, the construction process of the personalized emotion recognition model is as follows: First, set teaching contents with different difficulty levels for different students in the virtual simulation teaching platform. Then, divide the students into different groups according to their mastery of the teaching contents and actual learning abilities. Next, match different groups of students with teaching contents of different difficulty levels, and analyze the acceptance and mastery of different students for different teaching contents. Finally, construct an adaptation network between the teaching contents and different individual students.

[0132] This process can be implemented through the following algorithm:

[0133] S = {s 1 , s 2 ,..., s n},

[0134] C = {c 1 , c 2 ,..., c m},

[0135] L = {l 1 , l 2 ,..., l k},

[0136] A ij = f(s i , c j , l p ),

[0137] G q = {s i | g(A ij ) > θ q},

[0138] Among them, S is the set of students, C is the set of teaching contents, and L is the set of difficulty levels. A ij represents the acceptance degree of student s i for content c j and is calculated by function f based on the student, content, and difficulty level. G q represents the q-th student group, which is divided by function g based on the value of A ij being greater than the threshold θ q .

[0139] The core idea of this algorithm is to group and match contents based on the acceptance degrees of students for different difficulty contents. For example, we can set θ 1 = 0.8, θ 2 = 0.6, θ 3 = 0.4, corresponding to the student groups of high, medium, and low difficulty levels respectively. Such a setting can ensure that students are assigned to groups suitable for their learning abilities, thereby improving the learning effect.

[0140] The central control module 3 is communicatively connected to the cloud data processing module 2 and is the decision-making center of this system. It receives the analysis results sent by the cloud data processing module 2, makes comprehensive judgments based on these results, and generates corresponding control instructions. These control instructions will determine the next teaching strategies and learning environment configurations of the system.

[0141] In practical applications, the central control module 3 may use a decision matrix to determine the most suitable teaching strategy. For example:

[0142]

[0143] Among them, d ij represents the teaching strategy to be adopted in learning state i and emotional state j. For example, d 11 may represent increasing the difficulty and entering the practical operation mode when the learning state is good and the emotion is positive, while d 33 may represent reducing the difficulty and entering the simulation mode when the learning state is poor and the emotion is negative.

[0144] The virtual interaction module 4 is communicatively connected to the central control module 3 and is responsible for executing the instructions of the central control module 3. According to the received control instructions, the virtual interaction module 4 can switch between the simulation mode and the practical operation mode. This ability to switch modes enables the system to provide the most suitable learning environment according to the real-time state of students.

[0145] The hardware device module 5 is communicatively connected to the virtual interaction module 4 and serves as the interface for students to directly interact with the system. In the simulation mode, the hardware device module 5 provides a virtual immersive teaching environment; in the practical operation mode, it provides a teaching environment for hands-on operation. This flexible environment switching can meet the needs of different learning stages and different learning contents.

[0146] Through the collaborative work of the above modules, the system of the present invention can achieve personalized and intelligent immersive teaching. The system can adjust the teaching strategy according to the real-time state of the students, provide the most suitable learning environment, and thus significantly improve the teaching effect.

[0147] In another embodiment of the present invention, the cloud data processing module 2 includes a personalized emotion recognition model construction unit 21 and an intelligent analysis unit 22. The personalized emotion recognition model construction unit 21 is responsible for constructing the above-mentioned personalized emotion recognition model, while the intelligent analysis unit 22 is responsible for using this model to analyze the behavior data of the students.

[0148] The working process of the personalized emotion recognition model construction unit 21 can be described in more detail as follows: First, it sets a series of teaching contents with increasing difficulty levels in the virtual simulation teaching platform. For example, for a programming course, it may start from a simple Hello World program and gradually increase to the implementation of complex algorithms. Then, it monitors the performance of each student on these contents, including indicators such as completion time, correct rate, and number of help requests. Based on these data, the unit 21 can calculate the performance scores of each student at different difficulty levels.

[0149] Preferably, the performance score can be calculated by the following formula:

[0150] P i =w 1 T i +w 2 A i +w 3 H i ,

[0151] Where P i is the performance score of the student at the i-th difficulty level, T i is the normalized score of the completion time, A i is the correct rate, H i is the normalized score of the number of help requests (the fewer the number of help requests, the higher the score). w 1 、w 2 、w 3is the weight coefficient, which can be adjusted according to specific circumstances. For example, it can be set to 0.3, 0.5, or 0.2. Next, the personalized emotion recognition model construction unit 21 will group the students according to these performance scores. For example, the following grouping criteria can be set: If the student's P i >0.8 at difficulty level i, then the student will be assigned to the group at difficulty level i + 1; if 0.6 ≤ P i ≤0.8, then the student will remain in the group at difficulty level i; if P i <0.6, then the student will be assigned to the group at difficulty level i - 1. This grouping method can ensure that each student studies at a difficulty level that is both challenging and not overly difficult.

[0152] The intelligent analysis unit 22 is responsible for using the constructed model to analyze the real-time behavior data of the students. It may use machine learning algorithms such as support vector machines (SVM) or deep neural networks to extract features from the students' behavior data and predict the students' learning status and emotions. For example, if it is detected that the student's actions become sluggish and the voice tone decreases, it may be judged that the student's emotional state has become negative, and thus the system is recommended to make corresponding adjustments.

[0153] In this way, the cloud data processing module 2 can provide a personalized learning experience for each student, greatly improving the teaching effect and efficiency. The central control module 3 of the present invention includes a judgment unit 31 and an instruction generation unit 32, and these two units work together to achieve the intelligent decision-making function of the system.

[0154] The judgment unit 31 is responsible for receiving the analysis results sent by the cloud data processing module 2 and judging the learning status and emotions of the students based on these results. In a preferred embodiment of the present invention, the judgment unit 31 adopts a multi-dimensional evaluation model, which comprehensively considers multiple factors such as learning efficiency, attention concentration, and emotional stability. For example, learning efficiency can be measured by the speed and accuracy of the student's task completion, attention concentration can be evaluated by the student's eye movement data and head posture, and emotional stability can be judged by voice emotion analysis and facial expression recognition.

[0155] Preferably, the judgment unit 31 can use the following formula to calculate the comprehensive state index of the student:

[0156] CSI = w 1 E + w 2 A + w 3 S,

[0157] Among them, CSI is the Comprehensive State Index, E is the learning efficiency score, A is the attention concentration score, and S is the emotional stability score. w 1 、w 2 、w 3 are weight coefficients and can be adjusted according to specific teaching requirements. In practice, these weights can be set as w 1 = 0.4, w 2 = 0.3, w 3 = 0.3 to balance the influence of various factors.

[0158] The instruction generation unit 32 is closely connected to the judgment unit 31 and generates corresponding control instructions according to the judgment result of the judgment unit 31. The system of the present invention adopts an adaptive teaching strategy and can dynamically adjust the teaching mode according to the real-time state of the student. Specifically, when the CSI is greater than a certain preset threshold (for example, 0.7), the system will consider that the student is in a good state and can enter a more challenging learning stage; when the CSI is lower than another threshold (for example, 0.4), the system will judge that the student may need more support and thus switch to a more friendly learning environment.

[0159] In another embodiment of the present invention, the virtual interaction module 4 includes a mode switching unit 41, a simulation environment generation unit 42, and a practical operation environment generation unit 43. This design enables the system to flexibly switch to different learning modes according to the state of the student, thereby providing an optimal learning experience.

[0160] The mode switching unit 41 is the core of the virtual interaction module 4. It is responsible for receiving the control instructions sent by the central control module 3 and switching between the simulation mode and the practical operation mode according to these instructions. This switching is seamless, and students can hardly feel an obvious interruption, thus maintaining the coherence and immersion of learning.

[0161] The simulation environment generation unit 42 comes into play when the system enters the simulation mode. It can generate a highly realistic virtual classroom environment, including the indoor environment, laboratory environment, and experimental equipment environment. In the preferred embodiment of the present invention, the simulation environment generation unit 42 adopts advanced 3D modeling technology and a physics engine and can simulate various complex experimental scenarios. For example, in a chemistry experiment course, students can safely conduct various dangerous experiments in the virtual environment, observe the process of chemical reactions, without worrying about the risks of actual operations.

[0162] The practical operation environment generation unit 43 is activated when the system enters the practical operation mode. Its main function is to convert the devices in the hardware device module 5 into an operable actual teaching environment. This conversion enables students to directly interact with physical objects, thereby obtaining a real operation experience. For example, in a mechanical engineering course, students can operate a real machine tool through virtual reality devices and experience various details in the actual machining process.

[0163] The system of the present invention further includes a course design module 6 and a course analysis module 7. The introduction of these two modules greatly improves the intelligence level and teaching effect of the system.

[0164] The course design module 6 maintains a communication connection with the central control module 3. It can receive the student learning status and emotion analysis results sent by the central control module 3 and dynamically adjust the course content based on these results. In an embodiment of the present invention, the course design module 6 adopts an adaptive learning algorithm, which can automatically adjust the course difficulty and content according to the real-time performance of students.

[0165] For example, if the system detects that a student encounters difficulties in a certain knowledge point, the course design module 6 will automatically insert some relevant basic knowledge review contents to help the student better understand the current learning content. On the contrary, if the student performs well, the system will appropriately increase the difficulty and provide more challenging learning tasks. This dynamic adjustment ensures that each student can learn at the most suitable difficulty level for themselves, thereby maximizing the learning effect.

[0166] The course analysis module 7 maintains communication connections with both the course design module 6 and the central control module 3. Its main function is to analyze the learning situation of students, evaluate the course effect, and calculate the user satisfaction. In a preferred embodiment of the present invention, the course analysis module 7 uses a multi-dimensional evaluation model, which not only considers the learning achievements of students but also includes multiple aspects such as the participation degree, knowledge point mastery situation, and learning strategy usage situation during the learning process.

[0167] For example, the course analysis module 7 may use the following formula to calculate the comprehensive learning effect index:

[0168] LEI = α 1 G + α 2 P + α 3 U + α 4 S,

[0169] where LEI is the learning effect index, G is the learning achievement, P is the participation score, U is the knowledge point understanding score, and S is the learning strategy usage score. α 1 、α 2 ,α 3 、α4 is a weight coefficient, which can be adjusted according to specific teaching objectives. In practical applications, these weights can be set as α 1 = 0.4, α 2 = 0.2, α 3 = 0.2, α 4 = 0.2, to balance the influence of various factors.

[0170] The course analysis module 7 will feedback the analysis results to the course design module 6 for further optimizing the course content. This closed-loop design ensures that the system can continuously self-optimize and provide more and more accurate and effective teaching content.

[0171] Through the collaborative work of the above modules, the system of the present invention can provide a highly personalized and intelligent learning experience for each student, significantly improving the teaching effect and learning efficiency. The course analysis module 7 of the present invention includes a learning emotion recognition unit 71, a learning content analysis unit 72, a course content analysis unit 73 and a homework analysis unit 74. These units work together to jointly form a comprehensive learning analysis system, which can evaluate the learning status of students from multiple dimensions.

[0172] The learning emotion recognition unit 71 is mainly responsible for recognizing and recording the facial expressions of students. In a preferred embodiment of the present invention, this unit adopts advanced computer vision technology and deep learning algorithms. For example, it may use a convolutional neural network (CNN) to analyze the facial features of students and identify different emotional states such as concentration, confusion, excitement, etc. Preferably, this unit performs emotion recognition at regular time intervals (such as every 5 seconds) and records the results to form an emotional change curve of students during the entire learning process.

[0173] The learning content analysis unit 72 focuses on recognizing and recording the actions of students to determine their learning progress. In a virtual reality environment, every action of a student can be accurately captured. The system of the present invention takes advantage of this to judge the learning progress of students by analyzing their operation sequences. For example, in a virtual chemistry experiment, the system can judge whether students are performing the experiment according to the correct steps by a series of actions of students such as picking up reagents, pouring them into a beaker, and adjusting the temperature.

[0174] The course content analysis unit 73 is mainly used for recognizing and recording the answering situations in students' homework exercises to determine students' understanding of the course. In an embodiment of the present invention, this unit not only focuses on whether the answers are correct, but also analyzes the problem-solving processes of students. For example, in a math problem, even if the final answer is wrong, the system can identify whether the problem-solving strategies used by students are correct, so as to more accurately evaluate students' understanding.

[0175] The operation analysis unit 74 is responsible for obtaining the action and emotion records of students on the virtual reality device and analyzing the completion of students' assignments. The uniqueness of this unit lies in its ability to comprehensively analyze by combining students' behavioral and emotional data. For example, if the system detects that a student has stayed on a certain question for too long and shows an anxious state of mind, then the system may judge that this question is too difficult for the student and additional tutoring is needed.

[0176] The hardware device module 5 of the present invention is the key interface for students to interact with the system. It includes a virtual reality helmet 51, a spatial locator 52, virtual reality gloves 53, and a somatosensory suit 54. These devices are in communication connection with the virtual interaction module 4 by wired or wireless means, jointly constructing a highly immersive learning environment.

[0177] The virtual reality helmet 51 is the main device for students to perceive the virtual teaching environment. In the preferred embodiment of the present invention, this helmet adopts a high-resolution display screen and an advanced optical system, capable of providing a wide viewing angle and high-definition visual experience. For example, the helmet may be equipped with an OLED display screen with a 4K resolution, a field of view angle of 110 degrees, and a refresh rate of 90Hz. These parameters can effectively reduce the sense of dizziness and provide a smooth visual experience.

[0178] The spatial locator 52 is used to accurately track the operation actions of students. The system of the present invention adopts high-precision optical tracking technology, capable of accurately positioning the position and posture of students at the millimeter level. This precise tracking enables the operations of students in the virtual environment to be very natural and smooth, greatly enhancing the immersion of learning.

[0179] The virtual reality gloves 53 are specifically used to capture the hand actions of students. In an embodiment of the present invention, this glove adopts flexible sensor technology, capable of accurately capturing the bending degree and position of each finger. In addition, the glove is also equipped with a tactile feedback system, which can simulate the touch of different objects, further enhancing the realism of the learning experience.

[0180] The somatosensory suit 54 is responsible for capturing the body actions of students. This suit is built-in with multiple inertial measurement units (IMUs), which can track the full-body actions of students in real time. In some learning scenarios that require full-body participation, such as physical education teaching or dance training, this full-body action capture technology can play an important role.

[0181] The system of the present invention also includes a multimedia data storage module 8 and a multimedia playback module 9. The design of these two modules greatly enriches the teaching content and presentation methods of the system.

[0182] The multimedia data storage module 8 is used to store various multimedia resources required by the system, including pictures, audio, video, and animations. In a preferred embodiment of the present invention, this module adopts a distributed storage technology, which can efficiently store and quickly call a large number of multimedia materials. For example, the system may use the Hadoop Distributed File System (HDFS) to store these multimedia resources. This technology not only provides high-throughput data access but also has good fault tolerance and scalability.

[0183] The multimedia playback module 9 maintains a communication connection with the multimedia data storage module 8 and the virtual interaction module 4. It can, according to the instructions of the virtual interaction module 4, retrieve the corresponding multimedia resources from the multimedia data storage module 8 and play these resources in the virtual teaching environment. In an embodiment of the present invention, the multimedia playback module 9 adopts an adaptive streaming media technology, which can automatically adjust the playback quality according to the network conditions and device performance to ensure a smooth playback experience.

[0184] Through this design, the system of the present invention can seamlessly integrate various multimedia teaching resources in the virtual reality environment, greatly enhancing the vividness and intuitiveness of teaching. For example, in a history course, the system can reproduce historical scenes in the virtual environment while playing relevant audio commentaries and video materials, creating an immersive learning experience for students.

[0185] The system of the present invention further includes a homework module 10, which maintains a communication connection with the central control module 3 and the virtual interaction module 4 and plays a crucial role throughout the learning process. The main function of the homework module 10 is to receive the homework setting instructions sent by the central control module 3, generate corresponding homework content in the virtual teaching environment, record the students' homework completion status, and feedback this status to the central control module 3.

[0186] In a preferred embodiment of the present invention, the homework module 10 includes a homework setting unit 101, an answer management unit 102, and a homework analysis unit 103. This structural design enables the homework module 10 to provide comprehensive homework management and analysis functions, further improving the teaching effect of the system.

[0187] The homework setting unit 101 is responsible for generating homework content suitable for the current learning status and progress of students according to the instructions of the central control module 3. In an embodiment of the present invention, the homework setting unit 101 adopts a dynamic difficulty adjustment algorithm. This algorithm will automatically adjust the difficulty level of the homework according to the students' recent performance. For example, if a student performs excellently in several consecutive homework assignments, the system may increase the homework difficulty; conversely, if a student encounters difficulties, the system will appropriately reduce the difficulty or provide more auxiliary information.

[0188] Preferably, the adjustment of the homework difficulty can be achieved through the following formula:

[0189] D new = D old + α(P - T),

[0190] where D new and D old represent the new and the original difficulty levels respectively, P represents the actual performance score of the student, T represents the target score (which can be set to 80 points for example), and α is an adjustment coefficient (which can be set to 0.1). This method can keep the homework difficulty at a level that is both challenging and not overly difficult. The answer management unit 102 is mainly responsible for storing and managing the standard answers of the homework questions. In the system of the present invention, the answer is not just a simple result, but also includes detailed problem-solving steps and ideas. This design enables the system to provide more accurate and targeted feedback. For example, when a student makes a mistake in a certain step, the system can accurately locate the error position and provide corresponding guidance. The homework analysis unit 103 is responsible for analyzing the student's homework completion situation. In an embodiment of the present invention, the homework analysis unit 103 not only focuses on the correct rate of the homework, but also analyzes multiple dimensions such as the student's problem-solving process, time allocation, and error types. For example, the system may use the following formula to calculate the comprehensive homework performance index:

[0191] PI = w 1 A + w 2 T + w 3 P + w 4 E,

[0192] where PI is the homework performance index, A is the accuracy rate, T is the time efficiency score, P is the problem-solving process score, and E is the error analysis score. w 1 、w 2 、w 3 、w 4 are weight coefficients, which can be adjusted according to specific teaching requirements. In practice, these weights can be set to w 1 = 0.4, w 2 = 0.2, w 3 = 0.2, w 4 = 0.2 to comprehensively evaluate the student's homework performance.

[0193] The present invention also provides an immersive teaching method based on virtual reality technology, and this method includes the following steps:

[0194] S1: Construct a personalized emotion recognition model, specifically including:

[0195] Set teaching contents with different difficulty levels for different students in the virtual simulation teaching platform;

[0196] Divide students into different groups according to their mastery of teaching contents and actual learning abilities;

[0197] Match teaching contents with different difficulty levels for the grouped students;

[0198] Analyze the acceptance and mastery of different teaching contents by different students;

[0199] Construct an adaptation network between teaching contents and different individual students; this process can be achieved through machine learning algorithms. For example, clustering algorithms can be used for student grouping, and collaborative filtering algorithms can be used for content matching.

[0200] S2: Obtain student behavior data, specifically including:

[0201] Real-time collect students' gesture information and speech information through multiple sensors;

[0202] Form a real-time dynamic behavior data stream of students in the virtual simulation environment;

[0203] S3: Analyze student behavior data, specifically including:

[0204] Input the student behavior data into the personalized emotion recognition model;

[0205] Generate an analysis result of the learning emotion state of individual students, including positive state and negative state; this step is to input the obtained student behavior data into the previously constructed personalized emotion recognition model to generate an analysis result of the learning emotion state of individual students. These results usually include two dimensions: positive state and negative state. In an embodiment of the present invention, the system may use the following formula to calculate the emotion state index of students:

[0206]

[0207] Among them, ESI is the Emotion State Index, PE is the positive emotion score, and NE is the negative emotion score. The value range of ESI is [-1, 1], and the larger the value, the more positive the emotion state.

[0208] Based on these analysis results, the system controls the virtual simulation teaching environment. Specifically, when the analysis results show that the student's emotional state is poor and the learning effect is poor, the system controls the hardware device module to enter the simulation mode, enabling the student to enter the immersive virtual teaching environment. On the contrary, when the analysis results show that the student's emotional state is good and the learning effect is good, the system controls the hardware device module to enter the practical operation mode, enabling the student to enter the actual teaching environment for operation.

[0209] S4: Control the virtual simulation teaching environment, specifically including:

[0210] According to the analysis results of the behavior data, send corresponding control instructions to the virtual interaction module;

[0211] When the analysis results show that the student's emotional state is poor and the learning effect is poor, control the hardware device module to enter the simulation mode, enabling the student to enter the immersive virtual teaching environment;

[0212] When the analysis results show that the student's emotional state is good and the learning effect is good, control the hardware device module to enter the practical operation mode, enabling the student to enter the actual teaching environment for operation;

[0213] S5: Dynamically adjust the course content, specifically including:

[0214] Based on the student's learning status and emotional analysis results, retrieve the corresponding basic knowledge content and professional knowledge content from the database;

[0215] Design personalized course content according to the student's learning situation;

[0216] S6: Analyze the learning effect, specifically including:

[0217] Record and analyze the student's learning behavior and homework completion situation in the virtual teaching environment;

[0218] Evaluate the course effect and student satisfaction;

[0219] S7: Optimize the teaching strategy, specifically including:

[0220] Based on the learning effect analysis results, adjust the course content and teaching methods;

[0221] Update the personalized emotion recognition model to improve the accuracy and adaptability of the model.

[0222] Through the above steps, the method of the present invention can provide a highly personalized and intelligent learning experience for each student, significantly improving the teaching effect and learning efficiency.

[0223] It should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An immersive teaching system based on virtual reality technology, characterized by: include: Data acquisition module, used to: Acquire students’ gesture and voice action data through a variety of sensors; Form a real-time dynamic student behavior data stream; The cloud data processing module is in communication with the data acquisition module and is used to: Receiving student behavior data uploaded by the data acquisition module; Based on a pre-built personalized emotion recognition model, intelligently analyze the student behavior data; Generate individual student learning status and emotion analysis results; The central control module is connected to the cloud data processing module for: Receiving the analysis result sent by the cloud data processing module; Based on the analysis results, make a comprehensive judgment; Generate corresponding control instructions; The virtual interaction module is connected to the central control module for: Receiving control instructions sent by the central control module; According to the control instruction, switch the simulation mode or the actual operation mode; The hardware device module is connected to the virtual interaction module for: In simulation mode, it provides a virtualized immersive teaching environment; In the practical operation mode, a practical teaching environment is provided.

2. The immersive teaching system based on virtual reality technology according to claim 1 is characterized in that: The cloud data processing module includes: Personalized emotion recognition model building unit for: Set teaching contents of different difficulty levels for different students in the virtual simulation teaching platform; Students are divided into different groups according to their mastery of teaching content and actual learning ability; Matching teaching contents of different difficulty levels to the student groups; Analyze different students' acceptance and mastery of different teaching contents; Build an adaptation network between teaching content and different individual students; The intelligent analysis unit is communicatively connected with the personalized emotion recognition model building unit, and is used to: Receiving student behavior data uploaded by the data acquisition module; Analyzing the student behavior data based on the personalized emotion recognition model; Generate individual student learning status and sentiment analysis results.

3. The immersive teaching system based on virtual reality technology according to claim 1 is characterized in that: The central control module comprises: A judgment unit, used for: Receiving the analysis result sent by the cloud data processing module; Based on the analysis results, determine the student's learning status and emotions; An instruction generating unit is communicatively connected with the judging unit and is used for: Generate corresponding control instructions according to the judgment result of the judgment unit; When the judgment result is that the student is in a low mood or in a poor learning state, an instruction to enter the simulation mode is generated; When the judgment result is that the student is emotionally agitated or in a high learning state, an instruction to enter the practical operation mode is generated.

4. The immersive teaching system based on virtual reality technology according to claim 1 is characterized in that: The virtual interaction module includes: Mode switching unit, used for: Receiving control instructions sent by the central control module; According to the control instruction, switch the simulation mode or the actual operation mode; A simulation environment generating unit is communicatively connected with the mode switching unit and is used for: In simulation mode, a virtual classroom environment is generated, including indoor environment, laboratory environment and experimental equipment environment; A practical operation environment generating unit is communicatively connected with the mode switching unit and is used for: In the practical operation mode, the devices in the hardware device module are converted into an operational actual teaching environment.

5. The immersive teaching system based on virtual reality technology according to claim 1 is characterized in that: The system further comprises: The course design module is in communication with the central control module and is used to: Receive the student learning status and emotion analysis results sent by the central control module; Dynamically adjust course content based on the analysis results; Retrieve basic knowledge content and professional knowledge content from the database; Design personalized course content based on students’ learning situation; The course analysis module is in communication with the course design module and the central control module and is used to: Receiving course content information sent by the course design module; Receive the student learning status and emotion analysis results sent by the central control module; Analyze students’ learning status, course effectiveness and user satisfaction; The analysis results are fed back to the course design module for further optimization of the course content.

6. The immersive teaching system based on virtual reality technology according to claim 5 is characterized in that: The course analysis module includes: Learning emotion recognition unit to recognize and record students’ facial expressions; A learning content analysis unit to identify and record students’ actions and determine their learning progress; Course content analysis unit, used to identify and record students' answers in homework exercises to determine students' understanding of the course; The homework analysis unit is used to obtain students' action and emotion records on virtual reality devices and analyze students' homework completion status.

7. The immersive teaching system based on virtual reality technology according to claim 1 is characterized in that: The hardware device module includes: Virtual reality helmets, used to display virtual teaching environments and content; A spatial locator to track students’ actions; Virtual reality gloves to capture students’ hand movements; Motion-sensing clothing, used to capture students’ body movements; The virtual reality helmet, space locator, virtual reality gloves and somatosensory clothing are connected to the virtual interaction module in a wired or wireless manner.

8. The immersive teaching system based on virtual reality technology according to claim 1 is characterized in that: The system further comprises: Multimedia data storage module, used to store multimedia resources required by the system, including pictures, audio, video and animation; The multimedia playback module is communicatively connected with the multimedia data storage module and the virtual interaction module, and is used to: Receiving a play instruction sent by the virtual interaction module; Retrieving corresponding multimedia resources from the multimedia data storage module; The multimedia resources are played in a virtual teaching environment.

9. The immersive teaching system based on virtual reality technology according to claim 1 is characterized in that: The system further comprises: The operation module is connected to the central control module and the virtual interaction module for: Receiving a job setting instruction sent by the central control module; Generate corresponding homework content in the virtual teaching environment; Record students' homework completion; Feedback the completion status of the operation to the central control module; Among them, the homework module includes a homework setting unit, an answer management unit and a homework analysis unit.

10. An immersive teaching method based on virtual reality technology, using the system according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1: Build a personalized emotion recognition model, including: Set teaching contents of different difficulty levels for different students in the virtual simulation teaching platform; Students are divided into different groups according to their mastery of teaching content and actual learning ability; Matching teaching contents of different difficulty levels to the student groups; Analyze different students' acceptance and mastery of different teaching contents; Build an adaptation network between teaching content and different individual students; S2: Obtain student behavior data, including: Collect students' gesture and voice information in real time through a variety of sensors; Form a real-time dynamic behavior data stream of students in a virtual simulation environment; S3: Analyze student behavior data, including: Inputting the student behavior data into the personalized emotion recognition model; Generate analysis results of individual students' learning emotional states, including positive and negative states; S4: Control the virtual simulation teaching environment, including: According to the analysis results of the behavior data, corresponding control instructions are issued to the virtual interaction module; When the analysis results show that the student's emotional state is not good and the learning effect is not good, the hardware device module is controlled to enter the simulation mode, so that the student enters an immersive virtual teaching environment; When the analysis results show that the students are in a good emotional state and have good learning effects, the hardware device module is controlled to enter the practical operation mode, allowing the students to enter the actual teaching environment for operation; S5: Dynamically adjust course content, including: Based on the students’ learning status and emotion analysis results, the corresponding basic knowledge content and professional knowledge content are retrieved from the database; Design personalized course content based on students’ learning situation; S6: Analyze learning effects, including: Record and analyze students’ learning behaviors and homework completion in a virtual teaching environment; Evaluate course effectiveness and student satisfaction; S7: Optimize teaching strategies, including: Adjust course content and teaching methods based on the results of learning effect analysis; update the personalized emotion recognition model to improve the accuracy and adaptability of the model.

Images