Online network teaching practice method based on virtual technology

Abstract

The invention relates to the technical field of online network teaching, and discloses an online network teaching practice method based on a virtual technology, and the method comprises the steps: obtaining virtual scene data, and carrying out the integration and building of a dynamic knowledge graph and a multi-modal teaching resource library; learning behavior features are extracted according to the cognitive features of the learner, knowledge difficulty levels are divided, and immersive learning is guided; constructing a cognitive evaluation model, optimizing a learning path through hierarchical reinforcement learning, and generating a self-adaptive teaching instruction; collecting interaction track data to generate an operation thermodynamic diagram, and evaluating the skill mastering degree; complexity parameters of the virtual scene are dynamically adjusted, a virtual-real linkage experiment simulation environment is constructed, and skill training closed-loop feedback is achieved. The invention further relates to specific implementation modes of resource integration, cognitive feature extraction, learning path optimization and the like. According to the method, the personalized and immersive experience and skill training effect of online teaching are improved, and the method is suitable for the field of online network teaching.

Description

Technical Field

[0001] The present invention relates to the technical field of online network teaching, and specifically to an online network teaching practice method based on virtual technology. Background Art

[0002] With the rapid development of Internet technology, online network teaching has become an important trend in the education field. However, traditional online teaching often faces problems such as single teaching forms, insufficient learner participation, and difficulty in realizing personalized teaching. Especially in practical skill training, it cannot provide learners with a real and immersive learning environment, resulting in learners' insufficient understanding and mastery of knowledge and poor skill training effects. In the traditional teaching mode, teaching resources usually mainly consist of static texts, pictures or videos, lacking dynamic interactivity and being difficult to stimulate learners' learning interest and initiative. For complex knowledge systems, learners are difficult to intuitively understand the relationships between knowledge points, resulting in fragmented knowledge and being unfavorable for constructing a complete knowledge structure. At the same time, teachers cannot accurately understand learners' learning status and cognitive characteristics in real time, making it difficult to conduct targeted teaching guidance according to learners' individual differences and unable to achieve personalized teaching.

[0003] In terms of skill training, many professional fields such as experimental science and engineering technology require learners to master relevant skills through actual operations. However, traditional online teaching cannot provide a real experimental environment and operation platform. Learners can only learn by watching demonstration videos or reading experimental instruction books, lacking actual operation experience and being difficult to master practical operation skills and the ability to solve practical problems. In addition, safety issues during the experiment also limit the development of some high-risk and high-cost experiments.

[0004] The development of virtual technology provides new ideas and methods for solving the above problems. Virtual technology can construct a realistic virtual scene, providing learners with an immersive learning environment, enabling learners to conduct independent exploration and interactive operations in the virtual environment, and enhancing the interest and participation of learning. By integrating and analyzing virtual scene data, a dynamic knowledge graph can be established to help learners better understand the relationships between knowledge points and construct a complete knowledge structure. At the same time, by using virtual technology to monitor and analyze learners' learning behaviors in real time, the cognitive characteristics and learning status of learners can be accurately obtained, providing a basis for personalized teaching.

[0005] Currently, there are still some technical problems in the practice of online network teaching based on virtual technology. For example, how to effectively integrate and manage virtual scene data, establish a dynamic knowledge graph, and achieve the efficient utilization of multi-modal teaching resources; how to accurately divide the knowledge difficulty levels according to the cognitive characteristics and learning behaviors of learners, and realize personalized learning path planning; how to construct an effective cognitive evaluation model and interactive feedback mechanism, real-time evaluate the learning effects and skill mastery levels of learners, and dynamically adjust teaching strategies and the complexity of virtual scenes according to the evaluation results; how to realize the linkage between the virtual experimental environment and real experimental instruments, construct a virtual-real integrated experimental simulation environment, and ensure the authenticity and safety of experimental operations, etc.

[0006] Therefore, there is an urgent need for an online network teaching practice method based on virtual technology, which can make full use of the advantages of virtual technology, solve the problems existing in traditional online teaching, improve the quality and effect of online teaching, meet the needs of learners for personalized and immersive learning, especially achieve breakthroughs in practical skill training, and provide technical support for the innovative development of the education field. Summary of the Invention

[0007] The purpose of the present invention is to provide an online network teaching practice method based on virtual technology to solve the problems raised in the above background technology.

[0008] To achieve the above purpose, the present invention provides the following technical solution: An online network teaching practice method based on virtual technology, the method includes: Obtain virtual scene data corresponding to teaching content, including three-dimensional model data and interaction behavior data; integrate the virtual scene data and establish a dynamic knowledge graph to construct a multi-modal teaching resource library; Extract learning behavior characteristics according to the cognitive characteristics of learners, use the learning behavior characteristics to divide the knowledge difficulty levels, load the corresponding-level virtual teaching scenes for immersive learning guidance, and dynamically correct the learning behavior characteristics; Construct a cognitive evaluation model based on an interactive feedback mechanism, use the corrected learning behavior characteristics as the model input, optimize the learning path through hierarchical reinforcement learning, and generate adaptive teaching instructions according to the optimization results in combination with the virtual scene; Obtain the interaction trajectory data of learners through multi-dimensional data collection of the virtual scene, extract the operation heat map according to the interaction trajectory data, and generate the skill mastery evaluation result in combination with the operation heat map; Dynamically adjust the complexity parameters of the virtual scene according to the skill mastery evaluation result, use the complexity parameters to drive the change of the physical attributes of virtual objects, construct a virtual-real linked experimental simulation environment, and realize the closed-loop feedback of skill training.

[0009] Preferably, the virtual scene data is integrated and a dynamic knowledge graph is established to construct a multimodal teaching resource library, specifically: The semantic parsing engine is used to extract entity relationships from 3D model data and establish a knowledge point association network. The interactive behavior data is converted into event sequence labels and semantically aligned with the knowledge point association network. According to the teaching objectives, the knowledge granularity threshold is set, and the knowledge point association network is modularly cut to generate combinable teaching units. The teaching units are rendered by the virtual reality engine to build a multimodal resource library that includes visual, auditory, and tactile feedback. Establish a resource library version management mechanism, dynamically update the weight coefficients of teaching units based on teaching feedback data, and achieve continuous optimization of the resource library.

[0010] Preferably, learning behavior features are extracted based on the learner's cognitive features, specifically: Collect learners' eye tracking data, operation timing data and spatial displacement data in virtual scenes; perform feature dimension reduction on the original data to extract gaze hotspot distribution, operation frequency distribution and time-space correlation features; Construct a learner cognitive portrait model, input gaze hotspot distribution, operation frequency distribution and spatiotemporal correlation features into the model, and generate potential cognitive feature vectors through a generative adversarial network; Match the knowledge difficulty level according to the potential cognitive feature vector, set the interaction complexity parameters of the virtual scene, and realize personalized learning path planning.

[0011] Preferably, the learning path is optimized by hierarchical reinforcement learning, specifically: A three-layer reinforcement learning framework is established, which includes state space, action space and reward function. The state space includes learning progress, knowledge mastery and interaction effectiveness indicators; the action space defines the operation set of scene switching, object interaction and feedback intensity; Design a dynamic reward function based on course objectives, weighted calculation of short-term operational feedback and long-term knowledge mastery; train the policy gradient through a deep Q network to generate the optimal teaching action sequence; Course constraints are introduced to construct a constrained reinforcement learning model, and the feasibility of the teaching action sequence is verified to ensure that the teaching path conforms to cognitive laws.

[0012] Preferably, a virtual-real linkage experimental simulation environment is constructed, specifically: Establish a virtual experimental object model driven by a physical engine, configure the object's kinematic parameters and dynamic constraints; set the experimental failure safety boundary, and trigger the automatic protection mechanism when the virtual operation exceeds the boundary; Develop a multi-channel interaction adapter to map the operation signals of real experimental instruments into interaction instructions for virtual scenarios; transmit the mechanical response data of virtual experiments in real time through a haptic feedback device; Establish an experimental data mapping rule library to perform conversion calculations between virtual experiment parameters and real experiment standards, and generate a verifiable experimental conclusion report.

[0013] Preferably, combine the operation heat map to generate a skill mastery evaluation result, specifically: Conduct spatial clustering analysis on the operation heat map to identify high-frequency operation areas and low-frequency operation blind spots; calculate operation accuracy indicators, including trajectory deviation, timing accuracy, and force control; Construct a skill evaluation matrix, perform matrix operations on the high-frequency operation area ratio, operation accuracy indicators, and knowledge graph node coverage rate, and output a multi-dimensional skill evaluation vector; Generate a radar map according to the skill evaluation vector, mark the core skill advantage area and the weak area to be improved, and guide the setting of subsequent training focuses.

[0014] Preferably, dynamically adjust the complexity parameters of the virtual scenario, specifically: Establish a scene complexity evaluation model to comprehensively calculate the number of virtual objects, interaction response delay, and environmental rendering load; set a complexity adjustment gradient, and dynamically adjust the rendering resolution and physical simulation accuracy according to hardware performance indicators; Develop an adaptive loading system. When detecting network bandwidth fluctuations, prioritize ensuring the loading of resources for core teaching content and implement a progressive loading strategy for non-critical scene elements; Construct a resource preloading prediction model to predict the next operation requirements based on the learner's behavior pattern and pre-cache relevant scene resources in advance.

[0015] Preferably, perform resource integration on the virtual scenario data and establish a dynamic knowledge graph, specifically: Adopt an ontology modeling language to construct a knowledge representation framework, define the inheritance relationship and combination rules between knowledge points; implement an incremental update mechanism for the knowledge graph, and fuse new teaching cases through an incremental learning algorithm; Develop a cross-platform resource adaptation middleware to convert heterogeneous teaching resources into standardized description files; establish a resource index database to support fast resource retrieval based on semantics.

[0016] Preferably, generate potential cognitive feature vectors through a generative adversarial network, specifically: Construct a generative adversarial network framework. The generator network receives operation timing data to generate potential features, and the discriminator network distinguishes real features from generated features; design a course perception loss function to constrain the matching degree between the generated features and the teaching syllabus; Implement the feature space alignment algorithm, map the generated latent features to a predefined cognitive ability evaluation space, and output an interpretable cognitive feature distribution map.

[0017] Preferably, construct a constrained reinforcement learning model, specifically: Define three types of soft constraint conditions: curriculum progress constraint, knowledge coherence constraint, and cognitive load constraint; design a constraint relaxation factor to dynamically adjust the constraint intensity according to the real-time state of the learner; Develop a constraint satisfaction solver, use the Monte Carlo tree search algorithm to generate a feasible solution set that meets the constraint conditions, and screen the optimal teaching action sequence through the dominance pruning strategy.

[0018] Compared with the prior art, the beneficial effects of the present invention are: In terms of personalized learning path planning, by collecting the eye movement tracking data, operation timing data, and spatial displacement data of learners in the virtual scene, constructing a cognitive portrait model after dimensionality reduction processing of features, and using the generative adversarial network to generate latent cognitive feature vectors, it can accurately match the knowledge difficulty level and dynamically set the interaction complexity parameters of the virtual scene. This analysis based on the individual cognitive characteristics of learners changes the traditional "one-size-fits-all" teaching mode, adapts the teaching content to the ability level of learners, and effectively improves learning efficiency. For example, for learners with stronger cognitive abilities, the system can automatically load virtual teaching scenarios at a high difficulty level and provide more challenging learning content; while for learners with weak foundations, it starts from a low difficulty level and gradually increases the learning difficulty to ensure that each learner can make progress on a learning path suitable for themselves.

[0019] In terms of knowledge system construction and teaching resource management, through the semantic parsing engine, extract entity relationships from 3D model data, establish a knowledge point association network, and convert the interaction behavior data into event sequence labels to achieve semantic alignment. Combine the knowledge granularity threshold for modular cutting to generate combinable teaching units, and construct a dynamic knowledge graph and a multimodal teaching resource library. This process not only realizes the structured integration of teaching resources, but also endows the resources with multimodal feedback such as vision, hearing, and touch through the virtual reality engine, making abstract knowledge more intuitive and three-dimensional. At the same time, the resource library version management mechanism and incremental update mechanism can dynamically optimize the weight coefficients of teaching units according to teaching feedback data, integrate new teaching cases, and ensure that the teaching resources always maintain timeliness and scientificity to meet the ever-changing teaching needs. For example, in the teaching of mechanics in physics, through the multimodal resource library, learners can intuitively observe the force analysis process of objects and feel the action effects of different forces through tactile feedback, deepening their understanding of mechanics knowledge.

[0020] In terms of learning effect evaluation and teaching strategy optimization, the cognitive evaluation model built based on the interactive feedback mechanism, combined with the hierarchical reinforcement learning and constrained reinforcement learning models, can dynamically optimize the learning path. The three-layer reinforcement learning framework of state space, action space, and reward function incorporates indicators such as learning progress, knowledge mastery, and interaction effectiveness into the evaluation system, generates the optimal teaching action sequence through deep Q-network training, and introduces course constraint conditions to ensure that the teaching path conforms to the cognitive law. This evaluation and optimization mechanism makes the teaching process no longer a one-way knowledge transfer, but continuously adjusts the teaching strategy according to the real-time feedback of learners, realizing the dynamization and intelligentization of the teaching process. For example, when the system detects that the learner has a low mastery of a certain knowledge point, it will automatically adjust the teaching actions, increase the explanation duration and practice intensity of this knowledge point until the learner meets the mastery requirements.

[0021] In terms of skill training and experimental simulation, the constructed virtual-real linkage experimental simulation environment has significant advantages. The virtual experimental object model driven by the physics engine is configured with kinematic parameters and dynamic constraints. Combined with the multi-channel interaction adapter and tactile feedback device, it maps the operation signals of real experimental instruments into interaction instructions in the virtual scene and transmits the mechanical response data in real time, enabling learners to obtain a near-real experimental operation experience in the virtual environment. At the same time, the experimental data mapping rule library converts the virtual experimental parameters and real experimental standards to generate a verifiable experimental conclusion report, ensuring the scientificity and reliability of the experimental results. This virtual-real linkage method not only solves the problem of carrying out high-risk and high-cost experiments in traditional experimental teaching but also provides learners with the opportunity to repeat experiments multiple times, facilitating learners to deeply master experimental skills and principles. For example, in chemistry experimental teaching, learners can safely carry out flammable and explosive experimental operations in the virtual environment and master the correct experimental steps and operation skills through repeated practice. In terms of learning behavior analysis and skill assessment, by performing spatial clustering analysis on the operation heat map and calculating the operation accuracy index, constructing a skill assessment matrix and generating a radar chart, it can evaluate the learner's skill mastery degree in multiple dimensions and accurately mark the core skill advantage areas and weak areas to be improved. This provides clear feedback information for teachers and learners, facilitating targeted adjustment of training focuses and improving the effectiveness of skill training. For example, in mechanical operation skill training, the system analyzes the operation heat map and finds that the learner has problems such as low-frequency blind spots and insufficient operation accuracy in the installation operation of a certain component. Teachers can arrange special training accordingly to help learners make up for the shortcoming.

[0022] In addition, in terms of system performance optimization, the scenario complexity evaluation model and the adaptive loading system can dynamically adjust the rendering resolution, physical simulation accuracy, and resource loading strategy according to the hardware performance metrics and network bandwidth fluctuations, ensuring the stable operation of the system in different device and network environments and providing learners with a smooth learning experience. The resource preloading prediction model caches relevant scenario resources in advance based on the learner behavior patterns, further improving the system's response speed and resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is the working principle diagram of the online network teaching practice method based on virtual technology according to the present invention; Figure 2 is the design diagram for learning behavior feature extraction; Figure 3 is the design diagram for generating the skill mastery evaluation result; Figure 4 is the design diagram for the constrained reinforcement learning model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0025] Please refer to Figures 1-4 , for the online network teaching practice method based on virtual technology involved in the present invention, the specific implementation steps are as follows: Obtain virtual scene data including 3D model data and interaction behavior data, integrate the virtual scene data and establish a dynamic knowledge graph, and construct a multi-modal teaching resource library.

[0026] Extract learning behavior features according to the cognitive characteristics of learners, use the learning behavior features to divide the knowledge difficulty levels, load the corresponding level of virtual teaching scenes for immersive learning guidance, and dynamically correct the learning behavior features.

[0027] Build a cognitive evaluation model based on the interactive feedback mechanism, use the corrected learning behavior features as the model input, optimize the learning path through hierarchical reinforcement learning, and generate adaptive teaching instructions according to the optimization results in combination with the virtual scene.

[0028] Obtain the interaction trajectory data of learners through multi-dimensional data collection of the virtual scene, extract the operation heat map according to the interaction trajectory data, and generate the skill mastery evaluation result in combination with the operation heat map.

[0029] Dynamically adjust the complexity parameters of the virtual scene according to the evaluation results of skill mastery, use the complexity parameters to drive the changes in the physical properties of virtual objects, construct a virtual-real interactive experimental simulation environment, and achieve closed-loop feedback for skill training.

[0030] The following further elaborates on the technical solutions of the present invention in conjunction with specific embodiments.

[0031] Embodiment 1

[0032] The implementation methods of integrating virtual scene data resources and establishing a dynamic knowledge graph, and constructing a multimodal teaching resource library specifically include the parsing and association network construction of virtual scene data, the semantic alignment of interaction behavior data, the modular cutting and multimodal rendering of teaching units, the establishment of a resource library version management mechanism, and the extended construction of the knowledge graph and cross-platform resource adaptation. Each link cooperates with each other to achieve the systematic integration and dynamic optimization of teaching resources.

[0033] First, for the processing of three-dimensional model data, entity relationship extraction is performed through a semantic parsing engine. The semantic parsing engine can adopt named entity recognition (NER) technology and relationship extraction algorithms in the field of natural language processing (NLP), such as the BiLSTM-CRF model or graph neural network (GNN) model based on deep learning, to perform semantic parsing on information such as geometric entities, material properties, and spatial positions contained in the three-dimensional model, identify knowledge point entities in the model (such as "gear transmission structure", "chemical reaction vessel", etc.), and extract the association relationships between entities (such as "composition relationship", "causal relationship", "hierarchical relationship", etc.). For example, in the teaching scenario of mechanical principles, the semantic parsing engine can extract entities such as "number of teeth of the gear", "module", and "transmission ratio" from the three-dimensional model of the gear, and establish association relationships such as "the number of teeth affects the transmission ratio" and "the module determines the gear size", and then construct a knowledge point association network. This network is stored in the form of a graph structure, with nodes being knowledge point entities and edges being semantic relationships between entities, forming a knowledge network architecture containing knowledge nodes and relationship links.

[0034] Secondly, for the processing of interaction behavior data, it is necessary to convert it into event sequence tags and perform semantic alignment with the knowledge point association network. Interaction behavior data includes operation records such as clicks, drags, rotations, and zooms of learners in the virtual scene, as well as metadata such as timestamps and spatial coordinates when the operations occur. Through the event modeling method, the original interaction data is mapped into standardized event sequence tags, such as "SELECT_OBJECT(object ID, timestamp)", "ROTATE_OBJECT(object ID, rotation angle, timestamp)", etc. These tags are matched with the nodes in the knowledge point association network through the semantic alignment algorithm. For example, the "SELECT_OBJECT(gear model)" event can be associated with the "gear transmission structure" node in the knowledge point association network, indicating the learner's attention to this knowledge point. The semantic alignment process can be achieved by establishing an interaction behavior-knowledge point mapping table, which defines the knowledge point entities corresponding to different interaction actions and the association strength threshold, ensuring that the event sequence tags can accurately reflect the learner's knowledge interaction trajectory.

[0035] After completing data parsing and association, it is necessary to set a knowledge granularity threshold according to the teaching objectives and perform modular cutting on the knowledge point association network. The knowledge granularity threshold is used to measure the abstraction level and complexity of knowledge points. For example, in the primary teaching objective, a coarser granularity threshold can be set to combine multiple basic knowledge points into one teaching unit (such as combining "gear structure", "belt drive", and "chain drive" into the "basic mechanical drive" unit); while in the advanced teaching objective, a finer granularity threshold is set to split the knowledge points into smaller modules (such as splitting "gear drive" into sub-modules such as "spur gear drive", "helical gear drive", and "bevel gear drive"). The modular cutting is achieved through graph segmentation algorithms, such as the graph segmentation method based on spectral clustering. On the premise of ensuring the tightness of knowledge point association, the knowledge point association network is divided into several semantically complete and relatively independent teaching units. Each teaching unit contains a set of knowledge point nodes with strong association relationships and corresponding three-dimensional model data and interaction behavior data, forming a reusable teaching resource module.

[0036] Next, use a virtual reality engine to render the teaching unit scenes and build a multi-modal resource library. Virtual reality engines (such as Unity and Unreal Engine) provide functions such as three-dimensional scene rendering, physical simulation, and interactive logic development, and can render the three-dimensional model data in the teaching unit into a realistic virtual scene. In terms of visual feedback, the engine supports rendering technologies such as material texture mapping, lighting simulation, and particle effects. For example, it can simulate the color change of solutions in a chemical experiment and the light and shadow effects in mechanical motion. In terms of auditory feedback, environmental sound effects (such as background noise in a laboratory), operation prompt sounds (such as button click sounds), and voice explanations of knowledge points can be integrated through an audio engine. In terms of tactile feedback, combined with peripherals such as force feedback gloves, the interactive operations in the virtual scene (such as pressing a button and grasping an object) are converted into tactile stimuli. For example, it can simulate the tactile differences when grasping objects of different materials. The multi-modal resource library manages various teaching resources through a unified data interface, supports calling and combining according to the teaching unit dimension, and provides diverse sensory experiences for immersive teaching.

[0037] To achieve the continuous optimization of the resource library, a resource library version management mechanism needs to be established. This mechanism uses a version control tool (such as Git) to manage the versions of teaching units, recording information such as the creation time, modified content, and version number of each teaching unit. When receiving teaching feedback data (such as learner operation logs, knowledge point mastery statistics, and teacher evaluation opinions), the system dynamically adjusts the weight coefficients of knowledge points in the teaching unit according to a preset weight update algorithm. For example, if the feedback data shows that the learning error rate of the knowledge point "gear transmission ratio calculation" in a teaching unit is relatively high, then increase the weight coefficient of this knowledge point in the unit, improving its appearance frequency and explanation depth in the teaching scenario. The adjustment of the weight coefficient is achieved through the update of the edge weights of the knowledge graph, and at the same time, it triggers an incremental update of the resource library, synchronizing the optimized teaching unit version to the multi-modal resource library to ensure that the teaching resources are dynamically matched with the needs of learners.

[0038] In terms of the extended construction of the knowledge graph, ontology modeling languages (such as OWL and RDF) are used to build the knowledge representation framework. Ontology modeling languages provide formal definitions of classes, properties, and relationships, which can clarify the inheritance relationships between knowledge points (such as "spur gear" is a subclass of "gear") and combination rules (such as "gear drive system" is composed of "gear", "shaft", and "bearing"). The incremental update mechanism of the knowledge graph is realized through incremental learning algorithms. When new teaching cases (such as new mechanical structures and cutting-edge chemical experiments) are introduced, the algorithm automatically identifies new knowledge points and their associated relationships in the cases and integrates them into the existing knowledge graph, avoiding the cost of reconstructing the entire graph. For example, when introducing the teaching case of "epicyclic gear train", the incremental learning algorithm can identify new entities such as "planet carrier", "sun gear", and "planet gear", and establish their associated relationships with the "gear drive" node, realizing the dynamic expansion of the knowledge graph.

[0039] Regarding the compatibility issue of heterogeneous teaching resources, a cross-platform resource adaptation middleware is developed. This middleware supports converting teaching resources in different formats (such as DWG models of AutoCAD, OBJ models of Maya, and knowledge point documents in text format) into standardized description files and storing and transmitting them in a unified data format (such as JSON-LD). The standardized description files contain metadata of the resources (such as file type, creator, version number), semantic descriptions (such as knowledge point name, subject, difficulty level), and data content, ensuring that virtual teaching systems on different platforms (such as Windows, Android, and Web) can parse and load the resources. At the same time, a resource index database is established, and technologies such as inverted index and semantic index are used to support fast semantic-based resource retrieval. For example, when a learner searches for "gear drive efficiency", the system can match the knowledge point nodes related to "transmission efficiency" in the knowledge graph through semantic indexing and return a list of teaching units containing this knowledge point, improving the accuracy and efficiency of resource retrieval.

[0040] At the data interaction and storage level, the resource integration process of virtual scene data is realized through a distributed data processing architecture. 3D model data and interaction behavior data are stored in a distributed file system (such as HDFS) or an object storage service (such as Amazon S3) and are uniformly managed and scheduled through a data middle platform. Components such as semantic parsing engines and virtual reality engines interact with the data middle platform through API interfaces to realize data reading, processing, and writing. The distributed architecture supports high-concurrency data processing requirements, ensuring the stability and real-time nature of the resource integration and knowledge graph construction processes in large-scale online teaching scenarios.

[0041] In addition, the dynamic nature of the knowledge graph is also reflected in its support for teaching strategies. The edge weights in the knowledge point association network can reflect the knowledge mastery of the learner group in real time. For example, when multiple learners make mistakes in the knowledge point of "gear module calculation", the system automatically increases the connection weights between this knowledge point and related knowledge points such as "gear size design" and "transmission ratio calculation", prompting the teacher to strengthen the associated explanations between knowledge points in teaching. This adjustment of teaching strategies based on the knowledge graph can realize the transformation from one-way indoctrination centered on teachers to personalized teaching centered on learners.

[0042] During the construction of the multi-modal resource library, attention should be paid to the coordination of different modal feedbacks. For example, when explaining the knowledge point of "circuit connection", visual feedback presents the three-dimensional structure and connection method of circuit components, auditory feedback synchronously explains the circuit principle and operation steps, and tactile feedback simulates the resistance feeling when plugging and unplugging wires. The cooperation of these three modalities can enhance the memory effect and operation proficiency of learners. The synchronous control of multi-modal feedback is achieved through the time axis mechanism, and each modal element is triggered in the virtual scene according to the preset time sequence to ensure the consistency and fluency of information transmission.

[0043] Embodiment 2

[0044] The implementation method of extracting learning behavior characteristics according to the cognitive characteristics of learners specifically includes the collection and preprocessing of multi-dimensional learning behavior data, feature extraction and dimensionality reduction based on deep learning, the construction of a cognitive portrait model and the generation of latent features, as well as knowledge difficulty level matching and personalized learning path planning based on cognitive feature vectors. Each link realizes the accurate modeling of the learner's cognitive state and the dynamic generation of adaptive teaching strategies through a data-driven approach.

[0045] In the stage of collecting multi-dimensional learning behavior data, the system deploys a data collection architecture integrating multiple sensors. Eye movement tracking data is collected by an infrared eye tracker. This device emits near-infrared light to irradiate the eyeball, captures the corneal reflection light through a camera, calculates the relative position of the pupil center and the corneal reflection point, and then determines the fixation point coordinates. The sampling frequency of the eye tracker is set to 60Hz - 120Hz, which can meet the fixation point detection requirements in most teaching scenarios. Operation timing data is obtained by listening to user interaction events in the virtual scene, including mouse clicks, keyboard inputs, gesture operations, etc. Each operation event records information such as the timestamp, operation type, and operation object ID. Spatial displacement data is collected by an action capture system, using an inertial measurement unit (IMU) or optical motion capture technology to obtain the position coordinates and attitude data of the learner in the virtual space, with a sampling frequency of 30Hz - 60Hz to ensure continuous recording of the action trajectory.

[0046] After the acquisition of raw data, preprocessing is required to improve data quality. For eye-tracking data, noise filtering is first performed, and median filtering or Kalman filtering algorithms are used to remove instantaneous noises such as blinks and saccades. Then, fixation point identification is carried out, and continuous eye movement trajectories are divided into fixation points and saccades through the I-VT (Velocity-Threshold Identification) algorithm or I-DT (Dispersion-Threshold Identification) algorithm to generate a fixation point sequence. The preprocessing of operation timing data includes event deduplication, outlier detection and correction. For example, when detecting repeated click events within a short period, the first operation is retained and subsequent repeated operations are filtered. For operation parameters outside the reasonable range (such as too large a rotation angle), interpolation is used for correction. The preprocessing of spatial displacement data includes coordinate system unification, data smoothing and missing value filling. The coordinate data collected by different sensors are converted into a unified virtual scene coordinate system, the Savitzky-Golay filter is used for data smoothing, and linear interpolation or Kalman filtering is used to fill in the missing data points caused by occlusion.

[0047] In the feature extraction stage, a multi-modal feature fusion strategy is adopted. The fixation hotspot distribution features are extracted from the preprocessed eye movement data, and indicators such as the proportion of fixation time and fixation point density in each area of the virtual scene are calculated. Saccade features are extracted, including parameters such as saccade amplitude, saccade direction, and saccade speed. The pupil diameter change features are extracted to reflect the change of the learner's cognitive load. The operation frequency distribution features are extracted from the operation timing data, and the occurrence frequencies of different operation types (such as selection, movement, rotation) are counted. The operation sequence pattern features are extracted, and sequence mining algorithms (such as the PrefixSpan algorithm) are used to discover frequently occurring operation sequence patterns, such as the operation sequence of "select gear → adjust parameters → observe movement". The motion trajectory features are extracted from the spatial displacement data, and indicators such as trajectory length, motion speed, and acceleration are calculated. The spatial exploration features are extracted, including exploration area coverage rate, residence time distribution, etc.

[0048] To reduce the feature dimension and retain key information, feature dimension reduction techniques are adopted. For high-dimensional original feature vectors, first, feature standardization is performed to scale each eigenvalue to the same numerical range (such as the interval [0,1]) to eliminate the influence of dimensionality. Then, the principal component analysis (PCA) algorithm is applied for linear dimensionality reduction. The eigenvalues and eigenvectors of the feature covariance matrix are calculated, and the top k principal components with the largest variance contribution are selected as the new feature vectors to retain the main variability of the original data. For non-linearly separable feature data, non-linear dimensionality reduction algorithms such as t-SNE (t-Distributed Stochastic Neighbor Embedding) or UMAP (Uniform Manifold Approximation and Projection) are used to map high-dimensional features to a low-dimensional space while retaining the local and global structural relationships between data points. The dimension of the reduced feature vector is usually set to 1 / 3 - 1 / 2 of the original dimension to reduce the computational complexity of the subsequent model while ensuring the feature expression ability.

[0049] The cognitive portrait model is constructed using a generative adversarial network (GAN) architecture, including a generator network and a discriminator network. The generator network receives operation time series data as input and generates potential cognitive feature vectors through a multi-layer fully connected neural network and a long short-term memory network (LSTM). The LSTM layer is used to capture the temporal dependencies in the operation sequence, such as the thinking steps and operation order of learners when solving problems. The discriminator network receives real cognitive feature vectors (from expert annotations or historical data) and the feature vectors generated by the generator, and judges the authenticity of the input vectors through a binary classifier. The generator and the discriminator are continuously optimized through adversarial training. The generator learns to generate more realistic potential cognitive feature vectors, and the discriminator learns to improve its ability to distinguish between real and generated features. Improved algorithms such as WassersteinGAN (WGAN) or WGAN-GP (Gradient Penalty) are used during the training process to solve problems such as unstable training and mode collapse in traditional GANs.

[0050] To constrain the matching degree between the generated potential features and the teaching objectives, a curriculum-aware loss function is designed. This loss function consists of three parts: reconstruction loss, curriculum constraint loss, and adversarial loss. The reconstruction loss measures the accuracy of reconstructing the original operation data from the generated potential feature vectors, calculated using the mean squared error (MSE); the curriculum constraint loss evaluates the matching degree between the potential feature vectors and the syllabus, achieved by calculating the cosine similarity between the feature vectors and predefined curriculum knowledge vectors; the adversarial loss is calculated based on the output of the discriminator to guide the generator to generate potential features that are more in line with the real distribution. The three parts of the loss are weighted and summed to obtain the final loss function, and the weight coefficients are dynamically adjusted according to the importance of the teaching tasks.

[0051] After generating the potential cognitive feature vectors, they need to be mapped to an interpretable cognitive ability evaluation space. Implement a feature space alignment algorithm to map the generated potential feature vectors to predefined cognitive ability metrics (such as attention concentration, spatial imagination, logical reasoning ability, etc.). This process is achieved by training a regression model, using the expert-annotated cognitive ability scores as labels and the potential feature vectors as inputs to train a multi-layer perceptron (MLP) regression model. After the model training is completed, any potential feature vector can be converted into a cognitive ability evaluation score, and a feature distribution map containing multi-dimensional cognitive ability metrics is output. The map is presented in the form of a radar chart or a heat map, intuitively reflecting the strengths and weaknesses of the learner in different cognitive ability dimensions.

[0052] Based on the generated cognitive feature vectors, perform knowledge difficulty level matching and personalized learning path planning. First, establish a knowledge difficulty level system, dividing the teaching content into four levels: the basic level, the advanced level, the improvement level, and the innovation level. Each level sets clear knowledge goals and skill requirements. For example, the knowledge goal of the basic level is to master basic concepts and operation skills, the advanced level requires the ability to apply knowledge to solve routine problems, the improvement level involves the analysis and comprehensive application of complex problems, and the innovation level focuses on the transfer and creative application of knowledge. Each level corresponds to different virtual scenario interaction complexity parameters, including the number of scenario elements, interaction constraint conditions, feedback prompt levels, etc.

[0053] The matching process is achieved by calculating the similarity between the cognitive feature vectors and the feature templates of each difficulty level. Each difficulty level predefines a feature template, representing the cognitive ability distribution that learners at that level should possess. Use cosine similarity or Euclidean distance to calculate the similarity between the cognitive feature vectors and each template, and select the level with the highest similarity as the initial matching result. If the similarity value is lower than the set threshold (such as 0.6), it is determined that the learner's current cognitive ability does not meet the requirements of the lowest difficulty level, and the basic tutoring module is triggered.

[0054] Personalized learning path planning is generated based on the matched knowledge difficulty level, combined with the knowledge point association relationships in the knowledge graph. Starting from the starting knowledge point of the current level, plan the learning path according to the prerequisite relationships and association strengths between knowledge points. The path planning algorithm uses the A* search algorithm or the Dijkstra algorithm, with the knowledge point mastery degree as the heuristic function, and preferentially selects knowledge points with strong associations and low mastery degrees as the subsequent learning goals. During the learning process, the learner's cognitive feature vectors and knowledge point mastery degrees are updated in real time, and the learning path is dynamically adjusted. For example, when the learner shows strong comprehension ability in a certain knowledge point, the system automatically skips the basic explanation part of that knowledge point and directly enters the advanced practice; conversely, if difficulties are encountered in a certain knowledge point, relevant auxiliary resources and practice questions are increased.

[0055] To support real-time dynamic learning path optimization, the system adopts an online learning mechanism. Every time a teaching unit or practice task is completed, new learning behavior data is collected and the cognitive feature vector is updated. Through incremental learning algorithms, the system can quickly adapt to changes in the learner's cognitive state without retraining the entire model. At the same time, a learning path caching mechanism is established to pre-compute and cache some possible learning path branches. When the path needs to be adjusted, candidate paths are directly retrieved from the cache, reducing real-time computational overhead and ensuring the smoothness of the learning process.

[0056] At the technical implementation level, the training and inference of the cognitive portrait model are deployed on a distributed computing platform. Deep learning frameworks such as TensorFlow or PyTorch are used to build the model, and a GPU cluster is utilized to accelerate the training process. After the model training is completed, it is exported in TensorFlowLite or ONNX format and deployed to edge computing devices (such as VR headsets or smart terminals) for local inference, reducing network latency. The collection and preprocessing of multi-sensor data are achieved through a microservices architecture. Each sensor data processing module runs as an independent microservice, and data transmission and synchronization are carried out through a message queue (such as Kafka), ensuring the scalability and fault tolerance of the system.

[0057] The construction of the cognitive portrait model also takes into account the individual differences of learners. For learners of different ages, genders, and educational backgrounds, the model introduces personalized parameters for adjustment. For example, adolescent learners have a shorter attention span, so the model appropriately reduces the weight of the time dimension when calculating cognitive load; learners with a science and engineering background may perform better in spatial imagination, so the model gives higher attention to this dimension when generating potential features. These personalized parameters are automatically generated based on user registration information and initial test data, and also support manual adjustment by teachers according to teaching experience.

[0058] During the process of collecting learning behavior data, data privacy protection is emphasized. All collected data is anonymized, and information that can identify personal identity (such as name, student ID) is deleted. Encryption technology is used for data storage, and the TLS protocol is used for encryption during transmission and the AES algorithm is used for encryption during storage. Learners can view and manage their own learning data at any time and have the right to choose to delete records for a specific period or reject certain types of data collection. Embodiment

[0059] The implementation method of learning path optimization based on hierarchical reinforcement learning specifically includes the construction of a three-layer reinforcement learning framework, the design of a dynamic reward function, the training of a deep Q-network and the generation of an optimal action sequence, and the introduction of a constrained reinforcement learning model and the screening of feasible solutions. Each link realizes the intelligent planning of the teaching path and the optimal decision-making under constraints through mathematical modeling and algorithm optimization.

[0060] First, a three - layer reinforcement learning framework including a state space, an action space, and a reward function is established. The state space consists of a triple, that is , where represents the learning progress index, quantified as the ratio of the number of completed teaching units to the total number of units; represents the knowledge mastery index, calculated through the correct rate of knowledge point tests; represents the interaction effectiveness index, defined as the ratio of the number of effective interactions (such as correct operation steps) to the total number of interactions. The action space is defined as the operation set , where represents the scenario switching action, including loading a new teaching scenario or returning to the review scenario; represents the object interaction action, including selecting virtual objects, adjusting parameters, triggering experiments, etc.; represents the feedback intensity action, including enhancing prompts (such as highlighting key steps), weakening prompts (such as hiding auxiliary lines), or switching feedback modalities (such as from visual prompts to auditory prompts).

[0061] The dynamic reward function is designed as a weighted combination of short - term operation feedback and long - term knowledge mastery, and the formula is:

[0062] where is the weight coefficient , used to adjust the balance between short - term and long - term goals is the short - term reward, calculated based on the correctness of a single interaction. A correct operation is given a positive reward value, and an incorrect operation is given a negative reward value; is the long - term reward, calculated according to the change rate of knowledge mastery , that is , where is the difference in mastery within adjacent time intervals, is the length of the time interval. The dynamic reward function dynamically adjusts the reward weights by monitoring the learner's state in real - time. For example, it increases the value during the knowledge consolidation stage to strengthen correct operations, and decreases the value during the comprehensive application stage to focus on the long - term mastery effect.

[0063] The Deep Q - Network (DQN) is used to train the policy gradient and generate the optimal teaching action sequence. The network architecture adopts a Convolutional Neural Network (CNN) or a Fully - Connected Neural Network (FCN). The input is the vector representation of the state space , and the output is the action space The Q-values (expected cumulative rewards) for each action. During the training process, the experience replay mechanism is adopted, and the transition samples of state-action-reward-next state are stored in the experience pool, and small batches of samples are randomly sampled for training to alleviate the problems of data correlation and non-stationary distribution. The network parameters are updated through the temporal difference (TD) algorithm, and the objective function is:

[0064] where, is the current network parameter, is the target network parameter (updated regularly to stabilize the training), is the discount factor , which is used to balance immediate rewards and future rewards, represents uniform sampling from the experience pool . After the training is completed, the network outputs the Q-values of each action according to the current state, and selects the action with the largest Q-value as the optimal action to form an action sequence .

[0065] To ensure that the teaching path conforms to the cognitive law, course constraint conditions are introduced to construct a constrained reinforcement learning model. Three types of soft constraint conditions are defined: Course progress constraint: It is required that the order of teaching units in the learning path should not be earlier than its pre-requisite unit. For example, the "Circuit Analysis" unit needs to be unlocked after the "Circuit Basics" unit is completed; Knowledge coherence constraint: The knowledge correlation degree between adjacent teaching units needs to be higher than a threshold (measured by the weight of the edge in the knowledge graph, for example), to avoid cognitive breaks caused by jumping learning; Cognitive load constraint: The sum of the number of knowledge points and the interaction complexity per unit time should not exceed the cognitive load threshold of the learner. The load level is evaluated by real-time monitoring of the pupil diameter change and operation frequency in the eye movement data.

[0066] The constrained reinforcement learning model dynamically adjusts the constraint intensity by designing a constraint relaxation factor . When the learner's state is stable (such as the knowledge mastery degree is higher than 80% and the interaction effectiveness is higher than 70%), increase to relax the constraint and allow a more flexible learning path; when the learner has learning obstacles (such as three consecutive wrong operations), decrease to strictly constrain and ensure the robustness of the path.

[0067] The constraint satisfaction solution adopts the Monte Carlo tree search (MCTS) algorithm, and the process is as follows: Selection stage: Starting from the root node (current state), select child nodes according to the upper confidence bound (UCB) formula, and the formula is:

[0068] Among them, is the average reward in the state under the action, is the number of visits to the state, is the state under which the action is executed, and is the exploration coefficient used to balance exploration and exploitation; Expansion stage: Generate all executable actions (actions that meet the current constraint conditions) at the selected leaf node and create corresponding child nodes;

[0069] Select the optimal teaching action sequence through the advantage pruning strategy, retain the paths with cumulative rewards higher than the average reward and meeting all constraint conditions, and eliminate the paths that violate the constraints or have too low rewards. The finally output action sequence needs to meet both the optimization objective of the dynamic reward function and the course constraint conditions to ensure that the teaching path is both efficient and in line with the cognitive law.

[0070] In terms of technical implementation, the hierarchical reinforcement learning framework and the constraint solving module are integrated through a microservices architecture. The real-time data of the state space and the action space are transmitted through a message queue. The training and inference of the deep Q-network are deployed on a GPU server, and the Monte Carlo tree search is accelerated through parallel computing. The constraint relaxation factor and the weight coefficient can be manually adjusted by the teacher according to the course characteristics or automatically optimized through a meta-learning algorithm.

[0071] During the entire implementation process, the quantitative relationships between states, actions, and rewards are clarified through mathematical modeling, and the optimal decision-making under complex constraints is realized by using deep learning and search algorithms, avoiding the influence of subjective experience on the teaching path planning, and providing a data-driven intelligent learning path optimization solution for online network teaching.

[0072] Example 4: The implementation method of constructing a virtual-real interactive experimental simulation environment, taking the chemical experiment teaching scenario as an example, specifically includes links such as virtual experimental object modeling, safety boundary setting, multi-channel interaction adaptation, tactile feedback implementation, and experimental data mapping. Each link is driven by a physics engine and coordinated by software and hardware to achieve two-way linkage between virtual experiments and real operations.

[0073] In the aspect of virtual experimental object modeling, taking the "acid-base neutralization titration" experiment as an example, a physical engine (such as NewtonGameDynamics) is used to construct models of experimental apparatuses such as burettes, conical flasks, and beakers. Kinematic parameters are configured for the burette, including the angular range of piston rotation (0° - 360°) and the liquid dripping rate (which can be dynamically adjusted by the piston rotation speed); dynamic constraints are configured for the conical flask, such as the static friction coefficient when placed on the experimental bench and the rolling friction coefficient when tilted. The physical properties of virtual liquids (such as sodium hydroxide solution, hydrochloric acid solution) are defined through fluid simulation algorithms, including density, viscosity, and color change threshold (pH value triggering color change). Through the collision detection and dynamic calculation of the physical engine, real simulations of effects such as liquid volume change and color mixing during the titration process are achieved.

[0074] The setting of the experimental failure safety boundary is for possible dangerous operation scenarios. For example, when a learner accidentally pours acid directly into the alkali storage tank in a virtual experiment, the preset safety boundary in the system detects that the operation exceeds the danger threshold (such as the mixing ratio exceeding 1:1), and immediately triggers an automatic protection mechanism: pausing the experimental process, popping up a prompt box warning "Dangerous operation, may cause a violent reaction", and at the same time locking the interaction permissions of relevant apparatuses to prevent further incorrect operations. The safety boundary parameters are set based on the safety standards of real experiments, such as the upper limit of chemical mixing ratio, the upper limit of heating temperature, etc., and real-time monitoring of virtual operations is achieved through the constraint conditions of the physical engine.

[0075] The development of a multi-channel interaction adapter needs to be compatible with the signal mapping between real experimental apparatuses and virtual scenarios. Taking a hardware burette as an example, the piston rotation angle is collected by a potentiometer sensor and converted into a voltage signal of 0 - 5V; after receiving this signal, the interaction adapter maps the voltage value to the piston rotation angle (0° - 360°) of the virtual burette through an analog-to-digital conversion module (ADC), and then drives the liquid dripping rate of the virtual liquid. The liquid level scale of the real burette is collected through camera image recognition technology, and computer vision algorithms (such as YOLO object detection) are used to identify the scale value and transmit it to the virtual scenario to synchronously update the liquid level display of the virtual burette. For non-electronic apparatuses such as pH test papers, virtual-real mapping is achieved through QR code labels: after a learner uses a pH test paper to detect a solution in a real experiment and scans the QR code on the test paper, the system automatically synchronizes the detection result (pH value) to the virtual scenario and triggers the corresponding color change effect.

[0076] The real-time response of the tactile feedback device is achieved through force feedback gloves. In the virtual experiment, when the learner operates the piston to rotate, the physical engine calculates the torque required for rotation (based on parameters such as piston friction and liquid pressure), and transmits the torque value to the force feedback glove through the USB interface; the motor built into the glove applies reverse resistance according to the torque value, so that the learner can feel a resistance similar to the real operation. In the "pouring liquid" operation, the weight of the virtual liquid is converted into the grip feedback of the glove through dynamic calculation: when the weight of the liquid in the virtual beaker increases, the glove automatically tightens to simulate the feeling of holding weight. The faster the liquid is poured, the greater the grip feedback strength. The delay of tactile feedback is controlled within 20ms to ensure the synchronization of operation and feedback.

[0077] The establishment of the experimental data mapping rule base needs to cover the conversion logic between virtual parameters and real standards. Taking "concentration calculation" as an example, the parameters such as titration volume (V1) and standard solution concentration (C1) recorded in the virtual experiment are mapped through the formula in the rule base. Automatically calculate the concentration of the solution to be tested (C2), where is the volume of the solution to be tested (preset fixed value). The mapping rule library also contains standardized descriptions of experimental phenomena, such as the correspondence between pH value and color (red for pH<3, orange for 3≤pH≤6, and blue for pH>7), to ensure that the phenomenon output of the virtual experiment meets the real chemical standards. After the experiment, the system generates an experimental conclusion report containing operation steps, parameter records, and phenomenon descriptions based on the virtual operation records and data mapping results, which can be downloaded by learners or submitted to the teacher for review.

[0078] At the system integration level, the realization of virtual-reality linkage depends on real-time data communication protocols. The virtual scene runs on the PC or VR headset, and establishes a long connection with the hardware interactive devices (such as burettes and force feedback gloves) through the WebSocket protocol to achieve millisecond-level transmission of operation signals and feedback data. The hardware devices are driven by microcontrollers such as Arduino development boards or Raspberry Pi, and custom firmware is written to achieve sensor data acquisition and actuator control. For example, the potentiometer sensor of the burette is connected to the analog input pin of the Arduino Uno development board. The development board communicates with the PC through a USB to serial port module and sends real-time angle data to the virtual scene.

[0079] In terms of compatibility design, the multi-channel interaction adapter supports a variety of input and output interfaces, including USB, Bluetooth, Wi-Fi, etc., and can adapt to experimental instruments and peripherals of different brands. For example, for a wireless Bluetooth force feedback glove, data transmission is achieved through the Bluetooth protocol stack; for a Wi-Fi-based intelligent pH meter, detection data is synchronized through the HTTP API interface. Cross-platform adaptation is achieved through a unified device abstraction layer, which defines standard device operation interfaces, shields the underlying differences of different hardware, and ensures that the virtual experiment system can be flexibly connected to a variety of real instruments.

[0080] Security and reliability mechanisms run through the entire implementation process. In the virtual experiment object modeling stage, physical constraint isolation is carried out for dangerous scenarios such as high temperature and high pressure to ensure that virtual operations do not trigger simulation effects beyond the safety boundary; at the data transmission level, the TLS encryption protocol is used to prevent operation instructions and experimental data from being stolen or tampered with; at the hardware device level, an emergency stop button is set to cut off the power of all devices in case of emergencies to ensure the safety of learners. In addition, the system is built with a fault diagnosis module that continuously monitors the hardware connection status and virtual scene operation logs. When a communication interruption or data anomaly is detected, it automatically switches to an alternative connection channel or triggers an error recovery process.

[0081] Taking the experiment of "preparing a solution with a certain molar concentration" as an example, in the real environment, the learner uses a balance to weigh the mass of the drug, and the balance transmits the weighing data (such as 5.85 g of sodium chloride) to the virtual scene through the serial port; the virtual scene automatically calculates the amount of substance (0.1 mol) according to the preset molar mass (58.5 g / mol), and simulates subsequent steps such as dissolution, transfer, and volume fixation. In the volume fixation operation, the learner uses a real dropper to add distilled water, and the camera captures the liquid level height in real time and synchronizes it to the virtual scale line. When the liquid level approaches the scale line, the virtual scene triggers a prompt of "slowing down the dropping speed" to guide the learner to perform delicate operations. After the experiment is completed, the report generated by the system includes content such as the actual weighing error and the analysis of the volume fixation error, helping the learner understand the impact of operation details on the experimental results.

[0082] The virtual-real interactive experimental simulation environment not only supports single-user operations but can also be extended to multi-person collaboration scenarios. For example, in the "organic synthesis experiment", multiple learners are respectively responsible for different experimental links (such as feeding, temperature control, and product separation), and each operates through real instruments. The virtual scene synchronizes the operation data of all people in real time to form a collaborative experimental process. The system assigns different operation permissions through a permission management mechanism. For example, the main operator can adjust the reaction temperature, and the collaborator can only add materials, ensuring the orderliness of multi-person collaboration.

[0083] Example 5: The implementation method of combining operation heatmaps to generate skill proficiency assessment results and dynamically adjusting the complexity parameters of virtual scenarios, taking the virtual teaching scenario of mechanical assembly as an example, specifically includes spatial clustering analysis of operation heatmaps, construction of skill assessment matrices, generation of radar maps, as well as adjustment of scenario complexity parameters, adaptive loading, and resource preloading. The following will be detailed through specific processes and examples: I. Analysis of Operation Heatmaps and Skill Assessment In the virtual experiment of mechanical assembly, learners need to complete the task of "assembling a gear reducer". The system obtains their interaction trajectory data through multi-dimensional data collection, including mouse click positions, dragging paths, and keyboard operation frequencies. Spatial clustering is performed on the interaction data through the Gaussian kernel density estimation algorithm to generate an operation heatmap. The heatmap shows the operation density of each region in the virtual scenario with a color gradient (e.g., red represents high-frequency regions, and blue represents low-frequency regions). For example, in the reducer assembly interface, the heatmap may show that the "gear shaft installation area" is a red high-frequency area, while the "bearing lubrication step prompt button" is a blue low-frequency blind area, indicating that learners may have overlooked the operation guidance for the lubrication link.

[0084] Next, calculate the operation accuracy indicators: Trajectory deviation: Measures the deviation of the part movement path from the standard path during the assembly process. For example, when a learner installs a gear, if the mouse dragging trajectory deviates from the preset axial alignment path, the system calculates the deviation angle and distance through the path fitting algorithm to generate a deviation value.

[0085] Timing accuracy: Evaluates the rationality of the operation step sequence. The standard assembly process requires "install the bearing first and then insert the gear shaft". If a learner reverses the order, the system records the number of timing errors and calculates the accuracy score.

[0086] Force control degree: For operations that require pressure feedback (such as tightening bolts), judge whether the operation force is within the standard range (such as 10 - 15 N·m) through the force data collected by the force feedback glove. Exceeding the range is regarded as a force control deviation.

[0087] When constructing the skill assessment matrix, perform matrix operations on the proportion of high-frequency operation areas (such as the proportion of the operation time of learners in key assembly areas to the total time), the standardized values of operation accuracy indicators (trajectory deviation, timing accuracy, force control degree), and the coverage rate of knowledge graph nodes (such as the coverage of knowledge points such as "gear meshing principle" and "bearing positioning method" involved in the assembly process). For example, if the proportion of high-frequency operation areas of a learner reaches 70%, but the trajectory deviation is high (85 points), the timing accuracy is low (60 points), and the coverage rate of knowledge graph nodes is only 50%, the skill assessment vector output after matrix operations may show that "the operation proficiency is medium, but the knowledge application and step standardization are insufficient".

[0088] Generate a radar chart based on the skill assessment vector, using the core skill dimensions (such as spatial positioning, step logic, knowledge application, force control) as the coordinate axes and marking the scores for each dimension. In the example, the learner's radar chart may show that the "spatial positioning" dimension is prominent (80 points), but the "step logic" (65 points) and "knowledge application" (60 points) dimensions are relatively low, forming an obvious weak area, which guides the subsequent training to focus on strengthening the logical understanding of the assembly process and the associated application of knowledge points.

[0089] II. Dynamic Adjustment of Virtual Scenario Complexity Based on the skill assessment results, the system dynamically adjusts the complexity parameters of the virtual scenario. Taking the "gear reducer assembly" scenario as an example: Number of virtual objects: When a beginner learner first encounters assembly, the scenario only shows the core components (gear shafts, bearings, housings); as the skills improve, secondary components such as screws and seals are gradually added to increase the scenario complexity.

[0090] Interaction response delay: In the initial stage, the system provides real-time magnetic alignment assistance for part dragging operations (response delay < 100ms); in the advanced stage, the assistance is cancelled and the operation delay is increased (200 - 300ms) to simulate real physical inertia.

[0091] Environmental rendering load: Dynamically adjust the model accuracy according to the hardware performance (such as the computing power of the graphics card). Low - configuration devices display a simplified version of the gear model (with the number of triangular faces less than 1000), and high - configuration devices load a high - precision model (with the number of triangular faces more than 10000). At the same time, rendering technologies such as screen - space ambient occlusion (SSAO) are used to enhance the realism.

[0092] The complexity adjustment gradient is hierarchically controlled through preset difficulty levels (beginner, intermediate, advanced). For example, there are more interaction constraint conditions at the beginner level (such as parts can only move along a specific axis), the intermediate level reduces the constraints and increases the assembly tolerance requirements (such as the bearing installation error needs to be < 0.1mm), and the advanced level introduces fault - troubleshooting tasks (such as simulating abnormal noises caused by gear wear and requiring the learner to diagnose the cause by themselves). The system automatically switches levels according to the learner's skill assessment results. For example, if the learner reaches the intermediate level tasks twice in a row, they will be promoted to the advanced level; if they do not meet the standard, they will be returned to the beginner level for review.

[0093] III. Adaptive Loading and Resource Pre - loading When network bandwidth fluctuations are detected (such as switching from a wired network to Wi-Fi), the adaptive loading system prioritizes the loading of resources for core teaching content. For example, in an assembly scenario, it preferentially loads the models and physical engine data of key components such as gear shafts and housings, and implements a progressive loading strategy for non-critical elements such as tool racks and background decorations: first display low-precision model placeholders, and then update them to high-precision models after the network recovers. Progressive loading is achieved through chunk compression technology, which splits complex models into independent data chunks such as geometry, materials, and animations, and transmits them in order of priority.

[0094] The resource preloading prediction model is built based on the learner's behavior patterns. By analyzing historical operation data, common operation sequences are identified (such as "view assembly drawings → select parts → adjust perspective → start assembly"), and resources that may be needed next are cached in advance. For example, when a learner views the drawings before assembly multiple times, the system predicts that their next operation may require accessing the drawing file, so the drawing data is cached in the local memory in advance when entering the scene. The prediction model is trained using a long short-term memory network (LSTM), with the input being features such as the current operation type, dwell time, and historical path, and the output being the probability distribution of resource requirements for the next operation.

[0095] Specific examples: Initial stage: Learner A is performing the reduction gear assembly for the first time. The operation heat map shows that they frequently click the "assembly step hint" button, but the operations in the bearing installation area are sparse. The skill assessment matrix shows that the knowledge graph coverage rate is only 40%, and the "step logic" and "knowledge application" dimensions in the radar graph are weak. The system determines that they are at a primary level and dynamically reduces the scene complexity: hides secondary components, enhances step hints, and extends the interaction response delay to provide more adjustment time.

[0096] Advanced stage: After three trainings, the operation heat map of Learner A covers the key assembly areas, the trajectory deviation rate drops from 45% to 15%, and the timing accuracy increases from 55% to 80%. The system detects that their skill assessment vector meets the standards and promotes them to the intermediate level, adds components such as screws and seals, cancels the magnetic alignment assistance, and introduces assembly tolerance requirements (such as the gear meshing clearance needs to be controlled within 0.05 - 0.1 mm).

[0097] Advanced stage: After Learner A completes the intermediate tasks, the system triggers a fault simulation scenario: when the virtual reduction gear is running, abnormal noises occur. The operation heat map shows that they quickly locate the gear wear area and correctly replace the components. The skill assessment shows that the knowledge graph coverage rate reaches 90%, and all dimensions in the radar graph are balanced. The system further increases the complexity, requiring the completion of reverse disassembly and improvement design without drawing guidance, and at the same time enables high-precision rendering and real-time physical simulation to enhance the real operation experience.

[0098] The generation and analysis of the operation heat map are realized through WebGL rendering technology, supporting real-time drawing and interactive query on the browser side. The adjustment of complexity parameters and the resource loading logic are implemented through a scripting engine (such as C# in Unity or Blueprint in Unreal), decoupled from the virtual scene rendering thread to avoid performance impact. The adaptive loading system is compatible with the HTTP / 2 protocol and cache control headers (Cache-Control), supporting CDN content delivery network to accelerate resource transmission.

[0099] Throughout the implementation process, the system obtains the learner's behavior trajectory through non-invasive data collection, avoiding interference with the normal operation process. The skill assessment results are only used for personalized teaching path adjustment and do not involve horizontal comparison among learners, ensuring the constructiveness and motivation of the assessment. Through dynamic complexity adjustment and intelligent resource management, this solution adapts to different hardware performances and network environments while ensuring teaching effects, improving the universality and stability of virtual teaching.

[0100] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0101] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An online network teaching practice method based on virtual technology, characterized in that, It includes the following steps: Obtain the virtual scene data corresponding to the teaching content, including 3D model data and interaction behavior data; integrate the virtual scene data and establish a dynamic knowledge graph, and construct a multi-modal teaching resource library; Extract the learning behavior characteristics according to the cognitive characteristics of the learners, use the learning behavior characteristics to divide the knowledge difficulty levels, load the virtual teaching scenes at the corresponding levels for immersive learning guidance, and dynamically correct the learning behavior characteristics; Construct a cognitive evaluation model based on the interactive feedback mechanism, use the corrected learning behavior characteristics as the model input, optimize the learning path through hierarchical reinforcement learning, and generate adaptive teaching instructions according to the optimization results combined with the virtual scene; Obtain the interaction trajectory data of the learners through multi-dimensional data collection of the virtual scene, extract the operation heat map according to the interaction trajectory data, and generate the skill mastery evaluation result in combination with the operation heat map; Dynamically adjust the complexity parameters of the virtual scene according to the skill mastery evaluation result, use the complexity parameters to drive the change of the physical attributes of the virtual objects, construct a virtual-real linked experimental simulation environment, and realize the closed-loop feedback of skill training.

2. The online network teaching practice method based on virtual technology according to claim 1, wherein Integrate the virtual scene data and establish a dynamic knowledge graph, and construct a multi-modal teaching resource library. Specifically: Extract the entity relationship of the 3D model data through the semantic parsing engine and establish a knowledge point association network; convert the interaction behavior data into event sequence tags and perform semantic alignment with the knowledge point association network; Set the knowledge granularity threshold according to the teaching goal, modularly cut the knowledge point association network to generate combinable teaching units; render the teaching units through the virtual reality engine to construct a multi-modal resource library including visual, auditory, and tactile feedback; Establish a resource library version management mechanism, dynamically update the weight coefficients of the teaching units according to the teaching feedback data, and realize the continuous optimization of the resource library.

3. The online network teaching practice method based on virtual technology according to claim 1, characterized in that Extract the learning behavior characteristics according to the cognitive characteristics of the learners. Specifically: Collect the eye movement tracking data, operation timing data, and spatial displacement data of the learners in the virtual scene; perform feature dimensionality reduction processing on the original data, and extract the fixation hotspot distribution, operation frequency distribution, and spatio-temporal correlation characteristics; Construct a learner cognitive portrait model, input the fixation hotspot distribution, operation frequency distribution, and spatio-temporal correlation characteristics into the model, and generate a potential cognitive feature vector through the generative adversarial network; Match the knowledge difficulty level according to the potential cognitive feature vector, set the interaction complexity parameters of the virtual scene, and realize personalized learning path planning.

4. The online network teaching practice method based on virtual technology according to claim 1, characterized in that Optimize the learning path through hierarchical reinforcement learning. Specifically: Establish a three-layer reinforcement learning framework including a state space, an action space, and a reward function. The state space includes learning progress, knowledge mastery, and interaction effectiveness indicators; the action space defines an operation set for scene switching, object interaction, and feedback intensity; Design a dynamic reward function based on the curriculum goal, perform weighted calculation on the short-term operation feedback and the long-term knowledge mastery; train the policy gradient through the deep Q network to generate the optimal teaching action sequence; Introduce curriculum constraint conditions to construct a constraint reinforcement learning model, verify the feasibility of the teaching action sequence, and ensure that the teaching path conforms to the cognitive law.

5. The online network teaching practice method based on virtual technology according to claim 1, characterized in that Construct a virtual-real interactive experimental simulation environment, specifically as follows: Establish a virtual experimental object model driven by a physics engine, configure the kinematic parameters and dynamic constraints of the object; set the safety boundary for experimental failure, and trigger an automatic protection mechanism when virtual operations exceed the boundary; Develop a multi-channel interaction adapter to map the operation signals of real experimental instruments into interaction instructions for the virtual scene; transmit the mechanical response data of the virtual experiment in real time through a tactile feedback device; Establish an experimental data mapping rule library, perform conversion calculations on virtual experimental parameters and real experimental standards, and generate a verifiable experimental conclusion report.

6. The online network teaching practice method based on virtual technology according to claim 1, characterized in that, Generate a skill mastery evaluation result in combination with the operation heat map, specifically as follows: Conduct spatial clustering analysis on the operation heat map to identify high-frequency operation areas and low-frequency operation blind spots; calculate operation accuracy indicators, including trajectory deviation, timing accuracy, and force control; Construct a skill evaluation matrix, perform matrix operations on the proportion of high-frequency operation areas, operation accuracy indicators, and the coverage rate of knowledge graph nodes, and output a multi-dimensional skill evaluation vector; Generate a radar chart based on the skill evaluation vector, mark the core skill advantage areas and weak areas to be improved, and guide the setting of subsequent training focuses.

7. The online network teaching practice method based on virtual technology according to claim 1, characterized in that, Dynamically adjust the complexity parameters of the virtual scene, specifically as follows: Establish a scene complexity evaluation model, comprehensively calculate the number of virtual objects, interaction response delay, and environmental rendering load; set the complexity adjustment gradient, and dynamically adjust the rendering resolution and physical simulation accuracy according to hardware performance indicators; Develop an adaptive loading system. When detecting network bandwidth fluctuations, prioritize ensuring the resource loading of core teaching content, and implement a progressive loading strategy for non-critical scene elements; Construct a resource preloading prediction model, predict the next operation requirements according to the learner's behavior pattern, and cache relevant scene resources in advance.

8. The online network teaching practice method based on virtual technology according to claim 2, wherein Integrate the virtual scene data and establish a dynamic knowledge graph, specifically as follows: Use an ontology modeling language to construct a knowledge representation framework, define the inheritance relationship and combination rules between knowledge points; implement an incremental update mechanism for the knowledge graph, and fuse new teaching cases through incremental learning algorithms; Develop a cross-platform resource adaptation middleware to convert heterogeneous teaching resources into standardized description files; Establish a resource index database to support fast resource retrieval based on semantics.

9. The online network teaching practice method based on virtual technology according to claim 3, characterized in that Generate potential cognitive feature vectors through a generative adversarial network, specifically as follows: Construct a generative adversarial network framework. The generator network receives operation timing data to generate potential features, and the discriminator network distinguishes real features from generated features; Design a curriculum-aware loss function to constrain the matching degree between generated features and the teaching syllabus; Implement a feature space alignment algorithm to map the generated potential features to a predefined cognitive ability evaluation space, and output an interpretable cognitive feature distribution map.

10. The online network teaching practice method based on virtual technology according to claim 4, characterized in that, Construct a constrained reinforcement learning model, specifically as follows: Define three types of soft constraint conditions: curriculum progress constraint, knowledge coherence constraint, and cognitive load constraint; design a constraint relaxation factor to dynamically adjust the constraint intensity according to the learner's real-time state; Develop a constraint satisfaction solver, use the Monte Carlo tree search algorithm to generate a feasible solution set that meets the constraint conditions, and screen the optimal teaching action sequence through a dominance pruning strategy.

Images