Education experience system based on virtual reality technology

Abstract

The invention discloses an education experience system based on a virtual reality technology, which relates to the technical field of virtual reality and comprises a data acquisition module, a virtual reality content creation module, an interaction module, a data analysis module, an evaluation module and a central control module. The data acquisition module acquires and processes teaching materials; the virtual reality content creation module constructs a teaching scene according to materials; the interaction module generates a plot by means of a specific architecture, and interacts with a user by combining action, voice and touch units; the data analysis module collects multi-dimensional learning data, carries out deep analysis and then outputs a result; the evaluation module evaluates the learning effect and the teaching suitability by means of the analysis result and the user performance; the central control module coordinates data interaction and instruction transmission of the related modules as a whole and assists education development together.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, in particular to an educational experience system based on virtual reality technology. Background Art

[0002] In the field of education, traditional teaching methods have long faced many difficulties and urgently need innovative breakthroughs. In the past, teaching relied on classroom lectures and book reading, which were single and static. It was difficult to present the content vividly to students with only text descriptions and two-dimensional pictures, which made it difficult for students to understand and comprehend it deeply. For example, when explaining complex knowledge points, flat materials cannot show their three-dimensional structure and scenes, making it difficult for students to grasp the essence, and their enthusiasm for participation and knowledge internalization were poor.

[0003] At present, teaching resources are not well integrated, materials are scattered, and the quality varies. They are not systematically sorted and labeled, which makes it inconvenient to use them efficiently to build high-quality courses. In addition, the teaching process roughly evaluates the students' learning status, lacks multi-dimensional and accurate insights, and is difficult to optimize teaching based on individual differences. The coordination of various teaching links depends on manual allocation, the module linkage is inefficient, and there are freezes and delays in data interaction and command transmission, which affects the continuity and fluency of teaching.

[0004] In the information age, students are faced with a large amount of information impact and the influence of diversified values. Teaching needs to pay more attention to guiding students to learn in an interesting way, find the right methods in learning, and improve learning efficiency. Traditional teaching has certain limitations in dealing with these challenges, and the application of virtual reality technology provides new ideas and methods to enhance the attractiveness, appeal and effectiveness of teaching.

[0005] In order to solve the above problems, the present invention proposes an educational experience system based on virtual reality technology. Summary of the invention

[0006] The purpose of the present invention is to provide an educational experience system based on virtual reality technology to solve the problems raised in the prior art.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] An educational experience system based on virtual reality technology includes a data acquisition module, a virtual reality content creation module, an interaction module, a data analysis module, an evaluation module and a central control module; the data acquisition module is used to collect text, pictures, audio and video materials for teaching, and to edit, annotate and classify and store the materials; the virtual reality content creation module uses three-dimensional modeling technology to digitally construct educational theme scenes based on the sorted materials; the interaction module generates teaching plots based on a finite state machine combined with a deep learning model architecture, and interacts with users in combination with motion capture, voice recognition and tactile feedback units; the data analysis module collects multi-dimensional learning data of users, conducts in-depth analysis with the help of big data and machine learning algorithms, and outputs the analysis results in the form of charts and reports; the evaluation module uses the data analysis results and combines the user's performance in the learning process to evaluate the user's learning effectiveness and the teaching adaptability of the system; the central control module is responsible for coordinating the data interaction and instruction transmission between the virtual reality content module and the interaction module.

[0009] The data acquisition module includes a material acquisition unit, a material editing unit and a material storage unit; the material acquisition unit adapts to specific screening rules of educational websites and digital libraries through web crawler technology to collect text, pictures, audio and video materials related to teaching; the material editing unit has built-in image editing algorithms, audio processing algorithms and video editing toolsets. For the acquired materials, it can preset knowledge tags and teaching scene requirements for different subjects, perform operations such as cropping redundant images, adjusting audio volume and clarity, and editing video clips, and supports manual or automatic labeling of key information of the materials, and the labeled information covers knowledge point categories and applicable teaching grades; the material storage unit uses a database management system to set and classify the material library based on classification logic, archive and store the materials, and the storage system supports the call of materials.

[0010] The virtual reality content creation module includes a 3D modeling unit and a scene rendering unit; the 3D modeling unit uses polygonal modeling and physics-based rendering technology to construct a 3D model for the theme scenes of different subjects in combination with the material data collected by the data acquisition module; the scene rendering unit combines the real-time global illumination algorithm to divide the core rendering area, the transition rendering area and the edge rendering area according to the user's perspective and attention focus; the core rendering area is the part where the user's current attention is most concentrated. Suppose the coordinates of the user's gaze point in the virtual reality scene are P (x, y, z), and a spherical space range is determined with the gaze point as the center of the sphere and the radius as r1. The range included in the spherical space is the core rendering area A1, and the formula is expressed as follows:

[0011]

[0012] Where (x′, y′, z′) represents the coordinates of any point in the scene space, and the value of the radius r1 can be dynamically adjusted according to the field of view of the head-mounted display device, the screen resolution, and the visual sensitivity of the human eye;

[0013] The transition rendering area is located in the zone where the core rendering area expands outwards, and the rendering accuracy gradually decreases. The annular space with a radius of r2 extending outwards from the boundary of the core rendering area is the transition rendering area A2, where r2>r1, and the formula is expressed as follows:

[0014]

[0015] Among them, r2 can be dynamically adjusted according to the associated device parameters and scene characteristics;

[0016] The edge rendering area covers the remaining vast part of the virtual reality scene except the core and transition rendering areas. Assuming the space set of the entire virtual reality scene is A, the formula of the edge rendering area A3 is expressed as follows:

[0017] A3=A-A1-A2

[0018]

[0019] By quantitatively defining three rendering areas based on mathematical formulas and cooperating with real-time global illumination algorithms, the rendering system can efficiently and accurately adjust the rendering strategies of each area as the user's perspective changes dynamically and the focus of attention shifts, and allocate rendering resources on demand according to different teaching scenarios, allowing students to immerse themselves in the learning world of virtual reality.

[0020] The interaction module includes an intelligent script writing unit, a motion capture unit, a voice recognition unit and a tactile feedback unit; the intelligent script writing unit is based on a finite state machine combined with a deep learning model architecture. The finite state machine predefines the basic interaction states and conversion logic of opening guidance, knowledge explanation, interactive questions, practical operations, and summary feedback. The deep learning model uses long short-term memory networks and convolutional neural networks as the core architecture, optimizes the collected teaching interaction samples for training, and provides interactive scenes in real time according to user needs; the motion capture unit has a built-in inertial measurement unit and an optical tracking sensor, and is used with a handheld interactive device to track the translation, rotation, and grasping movements of the user's hands in three-dimensional space; the voice recognition unit integrates a multi-microphone array, and performs voice sample training based on a deep neural network acoustic model, a teaching customized language model, and an acoustic model. The language model is preloaded with different languages ​​required for teaching, and recognizes voice commands for scene switching, information query, and role dialogue operations.

[0021] The tactile feedback unit uses a scene-based tactile algorithm core, combined with hardware and a tactile generation mechanism to achieve tactile effects; the tactile algorithm core includes tactile feedback intensity, tactile feedback frequency, and tactile feedback duration. An adjustable framework formula is constructed based on the tactile algorithm core to cope with diverse conditions by adapting key parameters;

[0022] For haptic feedback strength, adjust the formula as follows:

[0023]

[0024] Among them, k 1i represents the weight adjustment coefficient of the tactile intensity in the i-th scene, k 2ij is the weight adjustment coefficient of the tactile intensity after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of tactile intensity feedback in the i-th scene;

[0025] For tactile feedback frequency, the adjustment formula is as follows:

[0026]

[0027] Among them, K 3i is the weight adjustment coefficient of the tactile frequency in the i-th scene, k 4ij is the weight adjustment coefficient of the tactile frequency after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of tactile frequency feedback in the i-th scene;

[0028] For haptic duration, adjust the formula as follows:

[0029]

[0030] Among them, k 5i is the weight adjustment coefficient of the tactile duration in the i-th scene, k 6ij is the weight adjustment coefficient of the tactile duration after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of the tactile duration feedback in the i-th scene; {P i}(i=1,2...n) is a scene characteristic parameter set, representing the quantitative indicators of key features of different scenes, {D j}(j=1,2...m) is a set of device characteristic parameters, reflecting the hardware quantitative indicators of the tactile feedback device used;

[0031] According to the above adjustment formula, the three dimensions are integrated and the formula for the comprehensive effect T of tactile feedback is: T = k7 × H × f × t + k8

[0032] Among them, k7 and k8 are global adjustment coefficients, k7 is used to amplify and reduce the amplitude of the comprehensive effect, and k8 is used to compensate the basic threshold and calibrate the overall zero position;

[0033] The hardware and tactile generation mechanism includes the coordination of multiple actuators and physical principles to associate real touch. The coordination of multiple actuators relies on a variety of vibration motors and micro air pressure bags distributed on the wearable device to present tactile effects. In the virtual scene, the vibration motor rotates rapidly to produce high-frequency and small-amplitude vibrations, evoking a similar high-frequency jitter tactile sensation; the air pressure bag is gradually inflated to simulate the slow pressure process, reproducing the physical interaction effect corresponding to the virtual situation.

[0034] Using virtual reality (VR) or augmented reality (AR) technology to create different teaching scenarios and corresponding interactive modules, students can interact in the scenarios. Finally, the system will evaluate and guide students based on their performance, and counselors can further carry out targeted education based on this.

[0035] The data analysis module includes a data collection unit, an algorithm analysis unit and a visualization output unit; the data collection unit is associated with each interactive device of the system to collect user operation trajectory data, voice interaction data, gaze hotspot data and scene dwell time data, and the collection frequency is multiple samplings per second, without missing key information; the algorithm analysis unit relies on the big data analysis algorithm combined with machine learning classification and clustering technology, and uses the decision tree classification algorithm to distinguish the behavior data of different teaching links. The clustering algorithm clusters user groups according to user learning style and knowledge mastery, and evaluates the knowledge understanding degree deviation MSE based on the mean square error formula. The formula is:

[0036]

[0037] where y x represents the actual knowledge mastery value in the xth learning scenario, represents the predicted mastery value in the xth learning scenario, and n is the number of samples; the visualization output unit outputs the data analysis results in the form of a bar chart to compare the weak knowledge sections, a line chart to present the learning status that fluctuates over time, and a summary report to narrate data interpretation and teaching suggestions.

[0038] The evaluation module includes a knowledge mastery evaluation unit; the knowledge mastery evaluation unit is connected to the data analysis module, and uses the built-in knowledge graph to construct a submodule to sort out the weighted knowledge point architecture, and uses an intelligent algorithm to compare the student learning trajectory data and the graph, and conducts a comprehensive evaluation based on operation performance, voice query, and gaze hotspots; through multi-dimensional considerations, quantitative indicators are set to evaluate the knowledge mastery level. The calculation formula for the accuracy rate A of answering questions on theoretical knowledge points is as follows:

[0039] A=a / b×100%

[0040] Where a represents the number of questions answered correctly, and b represents the total number of questions. When the value of A exceeds the preset threshold, it is considered that the user has mastered the theoretical knowledge point. Otherwise, it is marked as a weak knowledge point and output by the data analysis module.

[0041] The calculation formula for the teaching experience task success rate S is as follows:

[0042] S=c / d×100%

[0043] Among them, c represents the number of tasks completed by students, and d represents the total number of teaching experience tasks; when the value of S reaches a pre-set threshold, it means that the user has a good grasp of the knowledge points associated with the corresponding teaching scenario, otherwise it means that the grasp is insufficient.

[0044] The central control module includes a load monitoring unit, a dynamic feedback algorithm unit and a resource allocation unit; the load monitoring unit counts the amount of data that each module interacts with the outside in a unit time, and then goes deep into each module to quantitatively evaluate the computing resources consumed by the computing task currently being executed;

[0045] The dynamic feedback algorithm unit integrates the data flow and computing load information obtained by the load monitoring unit, analyzes the proportion of each module's data flow and computing load in the entire system and its changing trend. The analysis process requires calculation of the load distribution coefficient, which is as follows:

[0046] Q y / ΣQ y =α y

[0047] L y / ΣL y =b y

[0048] where α y Indicates that the yth module calculates the load distribution coefficient corresponding to the data flow, b y Indicates that the yth module calculates the load distribution coefficient corresponding to the computing load, Q y represents the data flow of the yth module, L y represents the computational load of the yth module, ΣQ y Represents the total data flow of all modules in the system, ΣL y It represents the sum of the computing loads of all modules in the system. After calculation, the load distribution coefficient of each module can reflect its relative busyness and resource demand in the entire system.

[0049] The relative busyness and resource demand, for the data traffic part, when the data traffic Q of a module yThe total data flow in the system ΣQ y When the proportion is large, the α calculated according to the formula y will increase; for the computing load part, when the computing load L of a module y The total computing load of the system ΣL y When the value of b increases, y will be larger; when α y and b y When the operation result becomes larger, it means that the module is relatively busy and has greater resource requirements for system operation;

[0050] The resource allocation unit allocates resources to the modules that need resources through the central control module according to the load distribution coefficient calculated by the dynamic feedback algorithm unit.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. Improve learning experience and effect: Traditional teaching in the prior art is mostly static and flat knowledge transfer, and students lack intuitive feelings; the present invention uses virtual reality technology to build a virtual scene, allowing students to participate in learning in an immersive way; in the traditional teaching mode, students mostly passively accept knowledge and have low interest in learning. The present invention changes the relatively boring and overly theoretical mode of traditional teaching through interactive modules such as motion capture and voice recognition, and allows students to actively participate in learning in a more interesting and immersive way. With the help of this device, counselors can better stimulate students' interest in empty seats and enhance the effectiveness of teaching.

[0053] 2. Precision and personalization of teaching: The data analysis module collects multi-dimensional data such as operation, voice, and gaze at a high frequency every second, analyzes it with big data and machine learning algorithms, quantifies knowledge deviations with mean square error, and visualizes weak sections and learning status with bar charts and line charts, helping teachers to understand students' difficulties in understanding knowledge and individual learning style differences, laying a solid foundation for personalized teaching; the central control module load monitoring, dynamic feedback algorithm and resource allocation coordination, accurately allocate resources according to the real-time traffic and computing load of each module to ensure smooth operation of the system. For example, when the content creation module has high traffic, it obtains more bandwidth and computing power to ensure that complex scenes are rendered without jamming, matching the needs of each link of teaching, and optimizing the overall teaching rhythm and effect.

[0054] 3. Optimize teaching evaluation and feedback: The knowledge mastery assessment unit of the assessment module relies on the built-in knowledge graph to sort out the weights of knowledge points, compare students' learning trajectories, and comprehensively consider operations, voice, and gaze. It uses the dual indicators of theoretical answer accuracy and learning task success rate to locate weak points, helping teachers to strengthen teaching in a targeted manner and improve students' knowledge mastery. The emotional attitude assessment unit captures emotions through voice intonation, gaze, and tactile feedback, and uses the emotional tendency scale combined with the teaching scenario threshold to dynamically score and statistically trend, accurately judging students' interest and recognition in the learning content. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a schematic diagram of the organizational structure of an educational experience system based on virtual reality technology in the present invention;

[0056] Figure 2 The present invention is a system flow diagram of an educational experience system based on virtual reality technology. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution, an educational experience system based on virtual reality technology, including a data acquisition module, a virtual reality content creation module, an interaction module, a data analysis module, an evaluation module and a central control module; the data acquisition module is used to collect text, pictures, audio and video materials for teaching, and to edit, annotate and classify and store the materials; the virtual reality content creation module uses three-dimensional modeling technology to digitally construct educational theme scenes based on the sorted materials; the interaction module generates teaching plots based on a finite state machine combined with a deep learning model architecture, and interacts with users in combination with motion capture, voice recognition and tactile feedback units; the data analysis module collects multi-dimensional learning data of users, deeply analyzes it with the help of big data and machine learning algorithms, and outputs the analysis results in the form of charts and reports; the evaluation module uses the data analysis results and combines the user's performance in the learning process to evaluate the user's learning effectiveness and the teaching adaptability of the system; the central control module is responsible for coordinating the data interaction and instruction transmission between the virtual reality content module and the interaction module.

[0059] The data acquisition module includes a material acquisition unit, a material editing unit and a material storage unit; the material acquisition unit adapts to specific screening rules of educational websites and digital libraries through web crawler technology to collect text, pictures, audio and video materials related to teaching; the material editing unit has built-in image editing algorithms, audio processing algorithms and video editing toolsets. For the acquired materials, it can preset knowledge tags and teaching scene requirements for different subjects, perform operations such as cropping redundant images, adjusting audio volume and clarity, and editing video clips, and supports manual or automatic labeling of key information of the materials, and the labeled information covers knowledge point categories and applicable teaching grades; the material storage unit uses a database management system to set and classify the material library based on classification logic, archive and store the materials, and the storage system supports the call of materials.

[0060] The virtual reality content creation module includes a 3D modeling unit and a scene rendering unit; the 3D modeling unit uses polygonal modeling and physics-based rendering technology to construct a 3D model for the theme scenes of different subjects in combination with the material data collected by the data acquisition module; the scene rendering unit combines the real-time global illumination algorithm to divide the core rendering area, the transition rendering area and the edge rendering area according to the user's perspective and attention focus; the core rendering area is the part where the user's current attention is most concentrated. Suppose the coordinates of the user's gaze point in the virtual reality scene are P (x, y, z), and a spherical space range is determined with the gaze point as the center of the sphere and the radius as r1. The range included in the spherical space is the core rendering area A1, and the formula is expressed as follows:

[0061]

[0062] Where (x′, y′, z′) represents the coordinates of any point in the scene space, and the value of the radius r1 can be dynamically adjusted according to the field of view of the head-mounted display device, the screen resolution, and the visual sensitivity of the human eye;

[0063] The transition rendering area is located in the zone where the core rendering area expands outwards, and the rendering accuracy gradually decreases. The annular space with a radius of r2 extending outwards from the boundary of the core rendering area is the transition rendering area A2, where r2>r1, and the formula is expressed as follows:

[0064]

[0065] Among them, r2 can be dynamically adjusted according to the associated device parameters and scene characteristics;

[0066] The edge rendering area covers the remaining vast part of the virtual reality scene except the core and transition rendering areas. Assuming the space set of the entire virtual reality scene is A, the formula of the edge rendering area A3 is expressed as follows:

[0067] A3=A-A1-A2

[0068]

[0069] By quantitatively defining three rendering areas based on mathematical formulas and cooperating with real-time global illumination algorithms, the rendering system can efficiently and accurately adjust the rendering strategies of each area as the user's perspective changes dynamically and the focus of attention shifts, and allocate rendering resources on demand according to different teaching scenarios, allowing students to immerse themselves in the learning world of virtual reality.

[0070] The interaction module includes an intelligent script writing unit, a motion capture unit, a voice recognition unit and a tactile feedback unit; the intelligent script writing unit is based on a finite state machine combined with a deep learning model architecture. The finite state machine predefines the basic interaction states and conversion logic of opening guidance, knowledge explanation, interactive questions, practical operations, and summary feedback. The deep learning model uses long short-term memory networks and convolutional neural networks as the core architecture, optimizes the collected teaching interaction samples for training, and provides interactive scenes in real time according to user needs; the motion capture unit has a built-in inertial measurement unit and an optical tracking sensor, and is used with a handheld interactive device to track the translation, rotation, and grasping movements of the user's hands in three-dimensional space; the voice recognition unit integrates a multi-microphone array, and performs voice sample training based on a deep neural network acoustic model, a teaching customized language model, and an acoustic model. The language model is preloaded with different languages ​​required for teaching, and recognizes voice commands for scene switching, information query, and role dialogue operations.

[0071] The tactile feedback unit uses a scene-based tactile algorithm core, combined with hardware and a tactile generation mechanism to achieve tactile effects; the tactile algorithm core includes tactile feedback intensity, tactile feedback frequency, and tactile feedback duration. An adjustable framework formula is constructed based on the tactile algorithm core to cope with diverse conditions by adapting key parameters;

[0072] For haptic feedback strength, adjust the formula as follows:

[0073]

[0074] Among them, k 1i represents the weight adjustment coefficient of the tactile intensity in the i-th scene, k 2ij is the weight adjustment coefficient of the tactile intensity after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of tactile intensity feedback in the i-th scene;

[0075] For tactile feedback frequency, the adjustment formula is as follows:

[0076]

[0077] Among them, K3i is the weight adjustment coefficient of the tactile frequency in the i-th scene, k 4ij is the weight adjustment coefficient of the tactile frequency after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of tactile frequency feedback in the i-th scene;

[0078] For haptic duration, adjust the formula as follows:

[0079]

[0080] Among them, k 5i is the weight adjustment coefficient of the tactile duration in the i-th scene, k 6ij is the weight adjustment coefficient of the tactile duration after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of the tactile duration feedback in the i-th scene; {P i}(i=1,2...n) is a scene characteristic parameter set, representing the quantitative indicators of key features of different scenes, {D j}(j=1,2...m) is a set of device characteristic parameters, reflecting the hardware quantitative indicators of the tactile feedback device used;

[0081] According to the above adjustment formula, the three dimensions are integrated and the formula for the comprehensive effect T of tactile feedback is: T = k7 × H × f × t + k8

[0082] Among them, k7 and k8 are global adjustment coefficients, k7 is used to amplify and reduce the amplitude of the comprehensive effect, and k8 is used to compensate the basic threshold and calibrate the overall zero position;

[0083] The hardware and tactile generation mechanism includes the coordination of multiple actuators and physical principles to associate real touch. The coordination of multiple actuators relies on a variety of vibration motors and micro air pressure bags distributed on the wearable device to present tactile effects. In the virtual scene, the vibration motor rotates rapidly to produce high-frequency and small-amplitude vibrations, evoking a similar high-frequency jitter tactile sensation; the air pressure bag is gradually inflated to simulate the slow pressure process, reproducing the physical interaction effect corresponding to the virtual situation.

[0084] Using virtual reality (VR) or augmented reality (AR) technology to create different teaching scenarios and corresponding interactive modules, students can interact in the scenarios. Finally, the system will evaluate and guide students based on their performance, and counselors can further carry out targeted education based on this.

[0085] The data analysis module includes a data collection unit, an algorithm analysis unit and a visualization output unit; the data collection unit is associated with each interactive device of the system to collect user operation trajectory data, voice interaction data, gaze hotspot data and scene dwell time data, and the collection frequency is multiple samplings per second, without missing key information; the algorithm analysis unit relies on the big data analysis algorithm combined with machine learning classification and clustering technology, and uses the decision tree classification algorithm to distinguish the behavior data of different teaching links. The clustering algorithm clusters user groups according to user learning style and knowledge mastery, and evaluates the knowledge understanding degree deviation MSE based on the mean square error formula. The formula is:

[0086]

[0087] where y x represents the actual knowledge mastery value in the xth learning scenario, represents the predicted mastery value in the xth learning scenario, and n is the number of samples; the visualization output unit outputs the data analysis results in the form of a bar chart to compare the weak knowledge sections, a line chart to present the learning status that fluctuates over time, and a summary report to narrate data interpretation and teaching suggestions.

[0088] The evaluation module includes a knowledge mastery evaluation unit; the knowledge mastery evaluation unit is connected to the data analysis module, and uses the built-in knowledge graph to construct a submodule to sort out the weighted knowledge point architecture, and uses an intelligent algorithm to compare the student learning trajectory data and the graph, and conducts a comprehensive evaluation based on operation performance, voice query, and gaze hotspots; through multi-dimensional considerations, quantitative indicators are set to evaluate the knowledge mastery level. The calculation formula for the accuracy rate A of answering questions on theoretical knowledge points is as follows:

[0089] A=a / b×100%

[0090] Where a represents the number of questions answered correctly, and b represents the total number of questions. When the value of A exceeds the preset threshold, it is considered that the user has mastered the theoretical knowledge point. Otherwise, it is marked as a weak knowledge point and output by the data analysis module.

[0091] The calculation formula for the teaching experience task success rate S is as follows:

[0092] S=c / d×100%

[0093] Among them, c represents the number of tasks completed by students, and d represents the total number of teaching experience tasks; when the value of S reaches a pre-set threshold, it means that the user has a good grasp of the knowledge points associated with the corresponding teaching scenario, otherwise it means that the grasp is insufficient.

[0094] The central control module includes a load monitoring unit, a dynamic feedback algorithm unit and a resource allocation unit; the load monitoring unit counts the amount of data that each module interacts with the outside in a unit time, and then goes deep into each module to quantitatively evaluate the computing resources consumed by the computing task currently being executed;

[0095] The dynamic feedback algorithm unit integrates the data flow and computing load information obtained by the load monitoring unit, analyzes the proportion of each module's data flow and computing load in the entire system and its changing trend. The analysis process requires calculation of the load distribution coefficient, which is as follows:

[0096] Q y / ΣQ y =α y

[0097] L y / ΣL y =b y

[0098] where α y Indicates that the yth module calculates the load distribution coefficient corresponding to the data flow, b y Indicates that the yth module calculates the load distribution coefficient corresponding to the computing load, Q y represents the data flow of the yth module, L y represents the computational load of the yth module, ΣQ y Represents the total data flow of all modules in the system, ΣL y It represents the sum of the computing loads of all modules in the system. After calculation, the load distribution coefficient of each module can reflect its relative busyness and resource demand in the entire system.

[0099] The relative busyness and resource demand, for the data traffic part, when the data traffic Q of a module y The total data flow in the system ΣQ y When the proportion is large, the α calculated according to the formula y will increase; for the computing load part, when the computing load L of a module y The total computing load of the system ΣL y When the value of b increases, y will be larger; when α y and b y When the operation result becomes larger, it means that the module is relatively busy and has greater resource requirements for system operation;

[0100] The resource allocation unit allocates resources to the modules that need resources through the central control module according to the load distribution coefficient calculated by the dynamic feedback algorithm unit.

[0101] 1. Material and data preparation:

[0102] In the material collection unit of the data collection module, the access rules and screening conditions of the authoritative education website and the local teaching material scanning and input parameters are pre-configured to start the material collection process and collect text, pictures, audio and video materials covering different teaching subjects.

[0103] After the material collection is completed, the material editing unit is used to process the material according to the preset knowledge tags and common teaching scene requirements. Blurred, repeated and redundant images in the video material are cut out, the volume of the audio material is adjusted to a uniform and comfortable listening range and the clarity is optimized, and video clips that match the length of the knowledge point explanation are edited. At the same time, key information such as "Peach Blossom Spring—Applicable to the second grade of junior high school" is manually annotated. Some simple annotations are completed with the help of the automatic annotation function to build a material library.

[0104] 2. Virtual content creation and rendering:

[0105] The 3D modeling unit of the virtual reality content creation module retrieves Peach Blossom Spring-related materials from the material library, uses polygonal modeling to shape the physical model, and combines it with physically-based rendering technology to restore the light and shadow effects of the time. For example, the winding forest path is covered with fallen petals, and every step is taken on softness and fragrance; the sun shines through the sparse treetops, casting mottled light and shadows.

[0106] The scene rendering unit is based on the parameters of the head-mounted display device, assuming a field of view of 110°, a screen resolution of 3840×2160 and the characteristics of human visual sensitivity, and sets the radius r1 of the core rendering area to a certain value. After testing, it is set to 3 meters, focusing on the environment around the user's gaze point, and the radius r2 of the transition rendering area is 5 meters. When the user enters the virtual scene experience, based on the user's real-time perspective and attention focus, such as focusing on fertile fields, beautiful ponds, and roads and traffic, the rendering system defines the area according to the formula for precise rendering, displays the core area with high image quality, moderately reduces the accuracy of the transition area but ensures visual continuity, and maintains the integrity of the basic scene in the edge area, allowing the user to be immersed in the scene and feel the beauty of the paradise.

[0107] 3. Interactive operation and experience:

[0108] The intelligent script writing unit is based on the opening guidance state preset by the finite state machine. The narration introduces the background of the fisherman to start the experience. When entering the knowledge explanation state, the deep learning model is used to analyze the user's previous interaction, such as what sentence describes the scenery in Taohuayuan, and gives graphic prompts in time. When the user triggers an interactive question, questions related to the details of Taoyuan pop up for the user to think about and answer. The user can use the voice recognition unit to say "What year is it today?" to query. After the unit recognizes the command, the answer pop-up window is displayed in the scene; or through the handheld interactive device motion capture unit, the practical operation of the fisherman visiting the villagers is simulated, and finally the knowledge points are reviewed in the summary feedback state.

[0109] The tactile feedback unit sets the adjustment coefficient based on the scene characteristics and device characteristics through the tactile algorithm core calculation. It is assumed that k7=1.2 enhances the overall effect amplitude and k8=0.1 calibrates the zero position.

[0110] 4. Data analysis and evaluation feedback:

[0111] During the entire user experience, the data collection unit collects operation trajectories, voice interactions, gaze hotspots, and scene dwell time data multiple times per second and transmits it to the algorithm analysis unit.

[0112] The algorithm analysis unit uses decision trees to distinguish between teaching behaviors such as opening and explanation, cluster analysis to find the learning style of user groups, and uses the mean square error formula to evaluate knowledge understanding deviation. Assuming that the actual mastery of the knowledge point value of Peach Blossom Spring is 80%, the predicted mastery value The MSE value is 60%, and the deviation can be quantified to locate the weak points. The visual output unit uses a bar chart to show the weak comparison of the "Peach Blossom Spring Scenery Details" section, and a line chart reflects the ups and downs of the user status during the experience, and generates a report for teachers' reference.

[0113] The knowledge mastery assessment unit of the assessment module uses the built-in knowledge graph to conduct a comprehensive assessment based on operation, voice, and gaze, and determines whether the user has mastered the knowledge based on the accuracy of the user's answers to knowledge points and scenario questions, and identifies knowledge weaknesses; the emotion attitude assessment unit determines positive emotions from passionate voice, focused gaze, and positive tactile seeking. If there are signs of negativity such as wandering eyes and resistant tactile, dynamic scoring and statistical trends are performed to help teachers adjust the teaching rhythm and atmosphere.

[0114] 5. System performance optimization and control:

[0115] The load monitoring unit monitors the data flow Q of each module in real time i , such as the virtual reality content creation module pushing scene data traffic peak and computing load L to the head-mounted device i ,For example, when the data analysis module processes massive amounts of data, the CPU is highly occupied, and the data is updated every second and fed back to the dynamic feedback algorithm unit.

[0116] The dynamic feedback algorithm unit integrates and analyzes the proportion trend of each module and calculates the load distribution coefficient α i , if the data flow Q of the virtual reality content creation module at a certain moment i The total flow rate in the system ΣQ i The proportion is 40%, corresponding to α i The resource allocation unit allocates more network bandwidth to it accordingly, assuming that it is increased from the regular 100Mbps to 200Mbps, and computing resources, assuming that an additional 2 CPU cores are allocated, to ensure that the system runs stably and efficiently when multiple users are experiencing it at the same time.

[0117] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An educational experience system based on virtual reality technology, characterized by: It includes a data acquisition module, a virtual reality content creation module, an interaction module, a data analysis module, an evaluation module and a central control module; the data acquisition module is used to collect text, pictures, audio and video materials for teaching, and to edit, annotate and classify and store the materials; the virtual reality content creation module uses 3D modeling technology to digitally construct educational theme scenes based on the sorted materials; the interaction module generates teaching plots based on a finite state machine combined with a deep learning model architecture, and interacts with users in combination with motion capture, voice recognition and tactile feedback units; The data analysis module collects multi-dimensional learning data of users, conducts in-depth analysis with the help of big data and machine learning algorithms, and outputs the analysis results in the form of charts and reports; the evaluation module uses the data analysis results and combines the user's performance in the learning process to evaluate the user's learning outcomes and the teaching adaptability of the system; the central control module is responsible for coordinating the data interaction and command transmission between the virtual reality content modules and the interactive modules.

2. The educational experience system based on virtual reality technology according to claim 1, characterized in that: The data acquisition module includes a material acquisition unit, a material editing unit and a material storage unit; the material acquisition unit adapts to specific screening rules of educational websites and digital libraries through web crawler technology to collect text, pictures, audio and video materials related to teaching; the material editing unit has built-in image editing algorithms, audio processing algorithms and video editing toolsets. For the acquired materials, it can preset knowledge tags and teaching scene requirements for different subjects, perform operations such as cropping redundant images, adjusting audio volume and clarity, and editing video clips, and supports manual or automatic labeling of key information of the materials, and the labeled information covers knowledge point categories and applicable teaching grades; the material storage unit uses a database management system to set and classify the material library based on classification logic, archive and store the materials, and the storage system supports the call of materials.

3. The educational experience system based on virtual reality technology according to claim 1, characterized in that: The virtual reality content creation module includes a three-dimensional modeling unit and a scene rendering unit; the three-dimensional modeling unit uses polygonal modeling and physically based rendering technology to construct a three-dimensional model for the theme scenes of different subjects in combination with the material data collected by the data collection module; The scene rendering unit combines the real-time global illumination algorithm and divides the core rendering area, the transition rendering area and the edge rendering area according to the user's perspective and attention focus; the core rendering area is the part where the user's current attention is most concentrated. Assuming the coordinates of the user's gaze point in the virtual reality scene are P (x, y, z), a spherical space range is determined with the gaze point as the center of the sphere and a radius of r1. The range included in the spherical space is the core rendering area A1, and the formula is expressed as follows: Where (x′, y′, z′) represents the coordinates of any point in the scene space, and the value of the radius r1 can be dynamically adjusted according to the field of view of the head-mounted display device, the screen resolution, and the visual sensitivity of the human eye; The transition rendering area is located in the zone where the core rendering area expands outwards, and the rendering accuracy gradually decreases. The annular space with a radius of r2 extending outwards from the boundary of the core rendering area is the transition rendering area A2, where r2>r1, and the formula is expressed as follows: Among them, r2 can be dynamically adjusted according to the associated device parameters and scene characteristics; The edge rendering area covers the remaining vast part of the virtual reality scene except the core and transition rendering areas. Assuming the space set of the entire virtual reality scene is A, the formula of the edge rendering area A3 is expressed as follows: A3=A-A1-A2 4. The educational experience system based on virtual reality technology according to claim 1, characterized in that: The interaction module includes an intelligent script writing unit, a motion capture unit, a voice recognition unit and a tactile feedback unit; the intelligent script writing unit is based on a finite state machine combined with a deep learning model architecture. The finite state machine predefines the basic interaction states and conversion logic of opening guidance, knowledge explanation, interactive questions, practical operations, and summary feedback. The deep learning model uses long short-term memory networks and convolutional neural networks as the core architecture, optimizes the collected teaching interaction samples for training, and provides interactive scenes in real time according to user needs; the motion capture unit has a built-in inertial measurement unit and an optical tracking sensor, and is used with a handheld interactive device to track the translation, rotation, and grasping movements of the user's hands in three-dimensional space; the voice recognition unit integrates a multi-microphone array, and performs voice sample training based on a deep neural network acoustic model, a teaching customized language model, and an acoustic model. The language model is preloaded with different languages ​​required for teaching, and recognizes voice commands for scene switching, information query, and role dialogue operations.

5. The educational experience system based on virtual reality technology according to claim 4, characterized in that: The tactile feedback unit uses a scene-based tactile algorithm core, combined with hardware and a tactile generation mechanism to achieve tactile effects; the tactile algorithm core includes tactile feedback intensity, tactile feedback frequency, and tactile feedback duration. An adjustable framework formula is constructed based on the tactile algorithm core to cope with diverse conditions by adapting key parameters; For haptic feedback strength, adjust the formula as follows: Among them, k 1i represents the weight adjustment coefficient of the tactile intensity in the i-th scene, k 2ij is the weight adjustment coefficient of the tactile intensity after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of tactile intensity feedback in the i-th scene; For tactile feedback frequency, the adjustment formula is as follows: Among them, K 3i is the weight adjustment coefficient of the tactile frequency in the i-th scene, k 4ij is the weight adjustment coefficient of the tactile frequency after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of tactile frequency feedback in the i-th scene; For haptic duration, adjust the formula as follows: Among them, k 5i is the weight adjustment coefficient of the tactile duration in the i-th scene, k 6ij is the weight adjustment coefficient of the tactile duration after the interaction between the i-th scene and the j-th device, P i is the scene characteristic of the tactile duration feedback in the i-th scene; {P i }(i=1,2...n) is a scene characteristic parameter set, representing the quantitative indicators of key features of different scenes, {D j }(j=1,2...m) is a set of device characteristic parameters, reflecting the hardware quantitative indicators of the tactile feedback device used; According to the above adjustment formula, the three dimensions are integrated and the formula for the comprehensive effect T of tactile feedback is: T = k7 × H × f × t + k8 Among them, k7 and k8 are global adjustment coefficients, k7 is used to amplify and reduce the amplitude of the comprehensive effect, and k8 is used to compensate the basic threshold and calibrate the overall zero position; The hardware and tactile generation mechanism includes the coordination of multiple actuators and physical principles to associate real touch. The coordination of multiple actuators relies on a variety of vibration motors and micro air pressure bags distributed on the wearable device to present tactile effects. In the virtual scene, the vibration motor rotates rapidly to produce high-frequency and small-amplitude vibrations, evoking a similar high-frequency jitter tactile sensation; the air pressure bag is gradually inflated to simulate the slow pressure process, reproducing the physical interaction effect corresponding to the virtual situation.

6. The educational experience system based on virtual reality technology according to claim 1, characterized in that: The data analysis module includes a data collection unit, an algorithm analysis unit and a visual output unit; the data collection unit is associated with each interactive device of the system to collect user operation trajectory data, voice interaction data, gaze hotspot data and scene dwell time data, and the collection frequency is multiple samplings per second to avoid missing key information; The algorithm analysis unit relies on the big data analysis algorithm combined with machine learning classification and clustering technology, and uses the decision tree classification algorithm to distinguish the behavioral data of different teaching links. The clustering algorithm clusters user groups according to user learning style and knowledge mastery, and evaluates the knowledge understanding degree deviation MSE based on the mean square error formula. The formula is: where y x represents the actual knowledge mastery value in the xth learning scenario, represents the predicted mastery value in the xth learning scenario, and n is the number of samples; the visualization output unit outputs the data analysis results in the form of a bar chart to compare the weak knowledge sections, a line chart to present the learning status that fluctuates over time, and a summary report to narrate data interpretation and teaching suggestions.

7. The educational experience system based on virtual reality technology according to claim 1, characterized in that: The evaluation module includes a knowledge mastery evaluation unit; the knowledge mastery evaluation unit is connected to the data analysis module, and uses the built-in knowledge graph to construct a submodule to sort out the weighted knowledge point architecture, and uses an intelligent algorithm to compare the student learning trajectory data and the graph, and conducts a comprehensive evaluation based on operation performance, voice query, and gaze hotspots; through multi-dimensional considerations, quantitative indicators are set to evaluate the knowledge mastery level. The calculation formula for the accuracy rate A of answering questions on theoretical knowledge points is as follows: A=a / b×100% Where a represents the number of questions answered correctly, and b represents the total number of questions. When the value of A exceeds the preset threshold, it is considered that the user has mastered the theoretical knowledge point. Otherwise, it is marked as a weak knowledge point and output by the data analysis module. The calculation formula for the success rate S of the teaching experience task is as follows: S=c / d×100% Among them, c represents the number of tasks completed by students, and d represents the total number of teaching experience tasks; when the value of S reaches a pre-set threshold, it means that the user has a good grasp of the knowledge points associated with the corresponding teaching scenario, otherwise it means that the grasp is insufficient.

8. The educational experience system based on virtual reality technology according to claim 1, characterized in that: The central control module includes a load monitoring unit, a dynamic feedback algorithm unit and a resource allocation unit; the load monitoring unit counts the amount of data that each module interacts with the outside in a unit time, and then goes deep into each module to quantitatively evaluate the computing resources consumed by the computing task currently being executed; The dynamic feedback algorithm unit integrates the data flow and computing load information obtained by the load monitoring unit, analyzes the proportion of each module's data flow and computing load in the entire system and its changing trend. The analysis process requires calculation of the load distribution coefficient, which is as follows: Q y / SQ y =a y L y / ΣL y =b y where α y Indicates that the yth module calculates the load distribution coefficient corresponding to the data flow, b y Indicates that the yth module calculates the load distribution coefficient corresponding to the computing load, Q y represents the data flow of the yth module, L y represents the computational load of the yth module, ΣQ y Represents the total data flow of all modules in the system, ΣL y It represents the sum of the computing loads of all modules in the system. After calculation, the load distribution coefficient of each module can reflect its relative busyness and resource demand in the entire system. The resource allocation unit allocates resources to the modules that need resources through the central control module according to the load distribution coefficient calculated by the dynamic feedback algorithm unit.

Images