VR Teaching Method, Device, Electronic Device, Storage Medium and Program Product

Abstract

The present invention provides a VR teaching method, device, electronic device, storage medium and program product. The method includes collecting learning process data of the learner when it is detected that the learner is learning through a VR learning course; performing an analysis of the learning situation category on the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance. By obtaining the learning process data of the learner, the present invention analyzes the learning process of the learner, so as to realize accurate and personalized learning assessment and teaching guidance for the learner, thereby improving the learning experience of the learner and finally improving the intelligent level of VR teaching.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent teaching, and in particular to a VR teaching method, device, electronic device, storage medium and program product. Background Art

[0002] Intelligent teaching is an important research field in educational technology. Based on the basic laws of educational science, it uses artificial intelligence technology to improve the teaching level of human teachers, thereby helping each learner acquire the required knowledge and improve weak skills.

[0003] A virtual reality system (VR) is a computer simulation system that can create and experience virtual worlds. Using VR technology, a simulated environment is generated on a computer and a display device, enabling users to immerse themselves in the environment, and the content displayed can break through the limitations of time and space. Due to its multi-sensory stimulation and intuitive and vivid immersive experience, VR technology is widely used in the field of intelligent teaching to enhance the learning experience and learning interest of learners through an immersive and gamified approach.

[0004] Currently, the VR teaching system uses VR devices as carriers and videos as media to create and present a vivid and realistic learning environment. However, the VR learning courses of this VR teaching system are fixed and the same for different learners, resulting in most learners being unable to adapt to the VR learning courses, thus reducing the learning experience of learners. Therefore, the existing VR teaching has the problem of low intelligent level. Summary of the Invention

[0005] The present invention provides a VR teaching method, device, electronic device, storage medium and program product to solve the defect of low intelligent level in existing VR teaching and achieve intelligent VR teaching.

[0006] The present invention provides a VR teaching method, including:

[0007] When it is detected that a learner is learning through a VR learning course, collecting learning process data of the learner;

[0008] Performing analysis of the learning situation category on the learning process data to obtain the learning situation information of the learner, where the learning situation information is used for intelligent guidance.

[0009] According to the VR teaching method provided by the present invention, the collecting of the learning process data of the learner includes:

[0010] Collecting the eye movement information and the first interaction information of the learner through an eye movement device; and / or,

[0011] The second interaction information of the learner is collected through a control device, where the control device is a device used by the learner to control the VR learning course.

[0012] According to a VR teaching method provided by the present invention, collecting the learner's eye movement information and first interaction information through an eye movement device includes:

[0013] The learner's eye movement information, first interaction information and course test scores of the VR learning course are collected through a head-mounted VR eye movement device.

[0014] According to a VR teaching method provided by the present invention, the learning process data is subjected to a learning situation category analysis to obtain the learning situation information of the learner, including:

[0015] The learning process data is input into the trained learning situation prediction model to perform learning situation category prediction, and the learning situation information of the learner output by the learning situation prediction model is obtained.

[0016] According to a VR teaching method provided by the present invention, the learning situation prediction model includes a learning behavior prediction model, a cognitive style prediction model and a digital portrait prediction model, and the learning process data is input into the trained learning situation prediction model to perform learning situation category prediction to obtain the learner's learning situation information, including:

[0017] Inputting the learning process data into the learning behavior prediction model, performing learning behavior category prediction, and obtaining the learner's learning behavior output by the learning behavior prediction model;

[0018] Inputting the learning process data into the cognitive style prediction model to perform cognitive style category prediction to obtain the learner's cognitive style;

[0019] The learning process data is input into the digital portrait prediction model to perform digital portrait category prediction, and obtain the digital portrait of the learner output by the digital portrait prediction model.

[0020] According to a VR teaching method provided by the present invention, after performing learning situation category analysis on the learning process data to obtain the learning situation information of the learner, the method further includes:

[0021] Based on the learning information, updating the learning information of the learner;

[0022] Based on the learning information, the VR learning course is updated so that the learner can learn through the updated VR learning course.

[0023] The present invention also provides a VR teaching device, comprising:

[0024] A data acquisition device, configured to acquire the learning process data of a learner when it is detected that the learner is learning through a VR learning course;

[0025] A learning situation analysis device, configured to perform a learning situation category analysis on the learning process data to obtain the learning situation information of the learner, where the learning situation information is used for intelligent learning guidance.

[0026] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of any one of the above-mentioned VR teaching methods are implemented.

[0027] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned VR teaching methods are implemented.

[0028] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of any one of the above-mentioned VR teaching methods are implemented.

[0029] The VR teaching method, device, electronic device, storage medium, and program product provided by the present invention acquire the learning process data of a learner when it is detected that the learner is learning through a VR learning course, and then perform a learning situation category analysis on the learning process data to obtain the learning situation information of the learner for use in intelligent learning guidance. In the above manner, the present invention analyzes the learning process of the learner by obtaining the learning process data of the learner, so as to perform accurate personalized learning assessment and teaching guidance on the learner, thereby improving the learning experience of the learner and ultimately improving the intelligent level of VR teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1 One of the flowcharts of the VR teaching method provided by the present invention;

[0032] Figure 2 Another flowchart of the VR teaching method provided by the present invention;

[0033] Figure 3 A schematic diagram of the VR teaching device provided by the present invention;

[0034] Figure 4 Illustrates a schematic diagram of the physical structure of an electronic device. Specific implementation manners

[0035] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0036] Figure 1 One of the flowcharts of the VR teaching method provided by the present invention is as Figure 1 shown. The VR teaching method provided by the present invention includes:

[0037] Step 100, when it is detected that a learner is learning through a VR learning course, collect the learning process data of the learner;

[0038] In this embodiment, the VR teaching method can be applied to a VR teaching system, which may include a VR teaching module, a learner data collection module, and a learner data analysis module. Further, the VR teaching system may also include a learner data storage module.

[0039] The VR teaching module is used to provide a VR learning course for learners to learn. Specifically, the VR teaching module is used to combine the personal basic information and learning information of the learner to provide interactive learning materials, and then output a learner learning situation report. The VR teaching module may include the learner's personal basic information, the learner's learning information, the VR learning course, etc.

[0040] The learner's personal basic information includes name, grade, age, historical scores of relevant tests of the VR learning course, etc. The learner's learning information includes learning behaviors, cognitive styles, and digital portraits, etc. The VR learning course includes the development and interaction of teaching content (i.e., knowledge points), the test of the mastery degree of knowledge points (i.e., course test), and the learner learning situation report, etc.

[0041] The learner learning situation report includes the current learning information of the learner, the scores of this course test, and suggestions on teaching methods and strategies. It is used for teachers and learners to view.

[0042] Among them, the VR learning course is used for learners to study, and the VR learning course includes VR videos. The VR learning course can be a course in any subject or field. For example, a VR learning course in geography, a VR learning course in biology, a VR learning course in kinematics, etc., which are not limited here. In addition, the VR learning course can be provided by a VR teaching module, and the VR teaching module can be a VR device.

[0043] In one embodiment, when it is detected that a learner studies a VR learning course through a VR teaching module, the learning process data of the learner is collected. Specifically, when it is detected that a learner studies a VR learning course through a VR device, the learning process data of the learner is collected.

[0044] In one embodiment, through a learner data acquisition module, the learning process data of the learner is collected. The learner data acquisition module is used to collect the learning process data of the learner, and it can include an eye movement device, a control device, a locator, a bracket, etc.

[0045] The eye movement device is used to collect the eye movement information of the learner and the interaction information of the course. The eye movement device can be a head-mounted eye movement device, which is convenient for the learner to wear and is convenient for accurately obtaining the eye movement information and interaction information of the learner. The eye movement device can be combined with a VR device to obtain a head-mounted VR eye movement device, and the head-mounted VR eye movement device is used to collect the eye movement information of the learner, the interaction information of the course, and the test scores of the learner in the course.

[0046] It should be noted that the eye movement device stores eye movement tracking technology. Through the eye movement tracking technology, detailed information such as the fixation points and eye movement of the learner when performing a certain task can be accurately obtained, and it can be used to analyze indicators such as the psychological state, learning style, and cognitive process of the learner's learning process.

[0047] The control device is used to collect the interaction information between the learner and the course. The control device can include devices such as a control handle, a mouse, and a keyboard.

[0048] The locator is used to ensure that various types of information can be collected by the VR device and the eye movement device in the field.

[0049] The bracket is used to fix the locator to stably obtain various types of information collected by the VR device and the eye movement device.

[0050] Among them, the learning process data is the relevant data generated during the learner's learning process, which can characterize the learner's cognitive reasoning process and cognitive logic, that is, the learner's actions are determined through the learning process data. The learning process data can include eye movement information, interaction information, and the course test scores of the VR learning course.

[0051] Eye movement information includes: blinking, gazing, closing eyes, and fixation point coordinates, etc. Interaction information includes: operations on the VR device, operations on the eye movement device, operations on the control handle, operations within the VR learning course, etc. Course test scores are obtained through tests within the VR learning course.

[0052] Further, after the above step 100, the VR teaching method further includes:

[0053] Storing the learning process data in a local database; or, storing the learning process data in a cloud database. That is, the learner data storage module may include a local database or a cloud database. Among them, the local database is used to store the collected learning process data locally. For example, data can be stored through a hard disk or flash memory; the cloud database is used to store the collected learning process data in the cloud database, thereby supporting large-scale online education.

[0054] Step 200, performing a learning situation category analysis on the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance.

[0055] In this embodiment, the learning situation information is used to characterize the cognitive reasoning ability and learning characteristics of the learner, etc. The learning situation information may include learning behaviors, cognitive styles, and digital portraits, etc.

[0056] In one embodiment, through the learner data analysis module, a learning situation category analysis is performed on the learning process data to obtain the learning situation information of the learner. The learner data analysis module may include a learning behavior unit, a cognitive style unit, and a digital portrait unit.

[0057] The learning behavior unit is used to analyze the behavior of the learner during learning based on the learning process data of the learner. The behavior of the learner during learning includes: concentration, fatigue, daze, boredom, excitement, confusion, boredom, etc. Further, the learning behavior unit may be a prediction model of machine learning, that is, the learning behavior of the learner is analyzed through machine learning algorithms. For example, a learning behavior category analysis is performed on the eye movement information of the learner to obtain that the learning behavior of the learner is excitement or boredom, etc.

[0058] A cognitive style unit is used to infer the cognitive style of a learner by analyzing the cognitive process data of the learner, that is, to understand how the learner analyzes and solves problems. Among them, the cognitive process data of the learner includes: abnormal behavior data and problem-solving data in the learning process data. The information that needs to be recorded for the abnormal behavior data includes: the timestamp and duration of the abnormal behavior, the interaction operations during the period, the restoration of the scene, etc.; the information that needs to be recorded for the problem-solving data includes: the components and operation sequence of the interaction operation, the key information for success / failure, all the interaction data before the problem is successfully solved, the timestamp when the problem is solved, the duration used to solve the problem, etc.

[0059] Furthermore, the cognitive style unit can be a prediction model of machine learning, that is, the cognitive style of the learner is analyzed through machine learning algorithms.

[0060] The cognitive style of a learner is the habitual behavior pattern shown by the learner in the cognitive process, which includes the information processing method, thinking style, problem-solving style, etc.

[0061] The information processing method includes the simultaneous processing of information and the successive processing of information. It should be noted that learners who process information simultaneously are good at using divergent thinking to comprehensively think about problems from multiple perspectives and can connect each component with the whole of things when solving problems; learners who process information successively tend to take a step-by-step, one-link-after-another approach, with an obvious chronological order in time.

[0062] The thinking style includes the characteristics of analysis and synthesis, divergence and concentration, and the width and narrowness of classification. It should be noted that analysis means that the learner decomposes the recognized concept or problem in the mind to understand, while synthesis means that the learner grasps things as a whole, with lower profundity and accuracy of thinking and higher intuitiveness and ambiguity; those who are good at divergent thinking are more enthusiastic and impulsive, and will think about problems along different directions and angles, reorganize information or interact with each other, while convergent thinking means that the learner is colder and more cautious, and prefers to think in one direction based on known information and familiar rules; learners with a wide classification will use vague criteria to classify new information into categories with too broad an extension, while learners with a narrow classification will use precise criteria to identify new information.

[0063] The problem-solving style includes reflection and impulsiveness, which can reflect the speed and accuracy of the cognitive process. It should be noted that reflective learners will spend sufficient time considering, weighing various problem-solving methods, and then select an optimal solution that meets multiple conditions. Although the reaction speed is slow, the quality of problem-solving is high; impulsive learners often make decisions in an intuitive way, with a fast reaction speed but prone to errors.

[0064] A digital portrait unit for obtaining a learner's digital portrait through the analysis of the learner's learning process data, which may include a knowledge graph of the knowledge mastered by the learner and personal learning characteristics.

[0065] Further, the digital portrait unit may be a prediction model of machine learning, that is, analyzing the learner's digital portrait through machine learning algorithms.

[0066] The knowledge graph includes the breadth, depth, and difficulty of knowledge point mastery, as well as the connection between previous and subsequent knowledge points.

[0067] Personal learning characteristics, and the indicators to be examined are the learner's learning efficiency and learning interest. It should be noted that the less time it takes to solve a problem, the higher the degree of knowledge point mastery, or the more problems solved, the higher the learning efficiency; the proportion of the learner's normal behavior during the learning process reflects the learner's learning interest.

[0068] Further, after the above step 200, the VR teaching method further includes:

[0069] Based on the learning situation information, determining reminder information and / or guiding operations for the learner to learn based on the reminder information and / or guiding operations. For example, when the VR teaching system identifies abnormal behavior data or cognitive characteristics in the learning behavior unit and cognitive style unit, corresponding reminders and guides can be provided in real time.

[0070] In addition, various data of the learner can be transmitted through one or more of a wired network, 4G network, 5G network, GPRS network, and WIFI network.

[0071] In a specific embodiment, before the above step 100, the VR teaching further includes:

[0072] Initializing the VR learning course and obtaining the learner's personal basic information; if the learner learns the VR learning course again, updating the learner's learning information based on the previous learning situation information. It should be noted that when first entering the VR teaching system, it is necessary to input the learner's personal basic information and initialize the learning information. If not first entering the VR teaching system, the learning information is updated based on the previous learning situation information.

[0073] In a specific embodiment, after the above step 200, the VR teaching method further includes:

[0074] Generating a learning situation report based on the learning situation information. The learning situation report includes learning situation information, the scores of this course test, tutoring suggestions, etc.

[0075] According to the VR teaching method of the embodiments of the present invention, when it is detected that a learner is learning through a VR learning course, the learning process data of the learner is collected. Then, the learning situation category analysis is performed on the learning process data to obtain the learning situation information of the learner, so as to use the learning situation information for intelligent learning guidance. In the above manner, the embodiments of the present invention obtain the learning process data of the learner to analyze the learning process of the learner, so as to achieve precise personalized learning evaluation and teaching guidance for the learner, thereby improving the learning experience of the learner and finally improving the intelligent level of VR teaching.

[0076] Further, based on the above first embodiment, a second embodiment of the VR teaching method of the present invention is proposed. In this embodiment, in the above step 100, collecting the learning process data of the learner includes:

[0077] Step 110, collecting the eye movement information and the first interaction information of the learner through an eye movement device;

[0078] In this embodiment, the eye movement device is used to collect the eye movement information of the learner and the first interaction information of the course. The eye movement information includes: blinking, gazing, closing eyes, and fixation point coordinates, etc. The first interaction information includes: operations on the eye movement device and operations within the VR learning course, etc. Specifically, the first interaction information can be obtained through the analysis of the eye movement information.

[0079] It should be noted that the eye movement device stores an eye movement tracking technology. Through the eye movement tracking technology, detailed information such as the fixation point and eye movement of the learner when performing a certain task can be accurately obtained, which can be used to analyze indicators such as the psychological state, learning style, and cognitive process of the learner's learning process.

[0080] In one embodiment, the above step 110 includes:

[0081] Collecting the eye movement information, the first interaction information of the learner, and the course test score of the VR learning course through a VR eye movement device. The VR eye movement device is a device that combines a VR device and an eye movement device, and this VR eye movement device has the functions of both a VR device and an eye movement device at the same time.

[0082] The course test score is obtained through a test within the VR learning course. Specifically, the course test score determined by the VR device in the VR eye movement device can be obtained, or the course test score can be determined by integrating the eye movement information.

[0083] In another embodiment, the above step 110 includes:

[0084] Step 111, collecting the eye movement information, the first interaction information of the learner, and the course test score of the VR learning course through a head-mounted VR eye movement device.

[0085] Among them, compared with the above VR eye movement device, the head-mounted VR eye movement device has a head-mounted function, which is convenient for learners to wear and is convenient for accurately obtaining the eye movement information and interaction information of learners.

[0086] Step 120: Collect the second interaction information of the learner through a control device, where the control device is a device for the learner to control the VR learning course.

[0087] In this embodiment, the control device is used to collect the second interaction information between the learner and the course. The control device may include devices such as a control handle, a mouse, and a keyboard.

[0088] Among them, the interaction information includes: operations on the VR device, operations on the eye movement device, operations on the control handle, operations within the VR learning course, etc. Further, the operations on the VR device and the operations on the eye movement device may be operations on the VR eye movement device.

[0089] Further, after the above step 120, the VR teaching method further includes:

[0090] Perform aggregation processing on the first interaction information and the second interaction information to obtain aggregated interaction information. Among them, the aggregated interaction information includes the first interaction information and the second interaction information, which is used to characterize the interaction characteristics between the learner and the VR learning course.

[0091] In this embodiment, the eye movement device is used to collect the eye movement information and interaction information of the learner, so as to analyze indicators such as the psychological state, learning style, and cognitive process of the learner's learning process based on the eye movement information and interaction information, thereby accurately obtaining the learning situation information of the learner, and further improving the accuracy of the VR teaching method.

[0092] Further, based on the above first embodiment, a third embodiment of the VR teaching method of the present invention is proposed. In this embodiment, the above step 200 includes:

[0093] Step 210: Input the learning process data into the trained learning situation prediction model for learning situation category prediction to obtain the learning situation information of the learner output by the learning situation prediction model.

[0094] In this embodiment, the learning situation prediction model is a machine learning model. Specifically, the learning situation prediction model is a model obtained by iteratively training the model to be trained based on the learning process training data. The learning situation information of the learner is the output of the learning situation prediction model.

[0095] In a specific embodiment, the learning situation prediction model includes a feature extractor and a classifier. Specifically, based on the feature extractor in the trained learning situation prediction model, the learning process feature information in the learning process data is extracted. Then, based on the learning process feature information and the classifier in the learning situation prediction model, the learning process feature information is classified and predicted to obtain a classification prediction result, that is, the learning situation information of the learner.

[0096] Among them, the specific execution process of the classifier is to obtain a classification probability vector, and then determine the learning situation information corresponding to the maximum classification probability value in the classification probability vector.

[0097] In one embodiment, the learning process data includes eye movement information, interaction information, and the course test scores of VR learning courses. Correspondingly, the feature extractor of the learning situation prediction model includes a first feature extractor, a second feature extractor, and a third feature extractor.

[0098] Specifically, based on the first feature extractor in the trained learning situation prediction model, the eye movement feature information in the eye movement information is extracted. Based on the second feature extractor in the trained learning situation prediction model, the interaction feature information in the interaction information is extracted. Based on the third feature extractor in the trained learning situation prediction model, the score feature information in the course test scores is extracted. Then, based on the eye movement feature information, the interaction feature information, the score feature information, and the classifier in the learning situation prediction model, the eye movement feature information, the interaction feature information, and the score feature information are classified and predicted to obtain a classification prediction result, that is, the learning situation information of the learner.

[0099] In some embodiments, the learning situation prediction model is an encoder-decoder neural network model. The learning situation prediction model includes an encoder, a decoder, and a classifier. The above step 210 includes:

[0100] Based on the encoder, feature extraction is respectively performed on the eye movement vector corresponding to the eye movement information, the interaction vector corresponding to the interaction information, and the score vector corresponding to the course test scores to obtain an eye movement feature vector, an interaction feature vector, and a score feature vector. And the eye movement feature vector, the interaction feature vector, and the score feature vector are aggregated to obtain an aggregated feature vector. Based on the decoder, the aggregated feature vector is decoded to obtain a decoded vector. Based on the classifier, the decoded vector is classified and predicted to obtain a classification prediction result (i.e., the learning situation information).

[0101] It should be noted that the encoder can be composed of a recurrent neural network, which can be an LSTM (long short-term memory) neural network, or the encoder can be composed of a deep convolutional neural network, etc. Correspondingly, the decoder can be composed of a recurrent neural network, which can be an LSTM (long short-term memory) neural network, or the decoder can be composed of a deep convolutional neural network, etc. The classifier can be composed of a fully connected layer.

[0102] Of course, other neural network models can be adopted for the learning situation prediction model, for example, a convolutional neural network model, a recurrent neural network model, etc.

[0103] In one embodiment, the learning situation prediction model includes a learning behavior prediction model, a cognitive style prediction model, and a digital portrait prediction model. The learning situation information includes learning behavior, cognitive style, and digital portrait. The above step 210 includes:

[0104] Step 211, input the learning process data into the learning behavior prediction model to perform learning behavior category prediction, and obtain the learning behavior of the learner output by the learning behavior prediction model;

[0105] In this embodiment, the learning behavior prediction model is a machine learning model. Specifically, the learning behavior prediction model is a model obtained by iteratively training the model to be trained based on the learning behavior training data. The learning behavior of the learner is the output of the learning behavior prediction model.

[0106] In a specific embodiment, the learning behavior prediction model includes a feature extractor and a classifier. Specifically, based on the feature extractor in the trained learning behavior prediction model, the learning process feature information in the learning process data is extracted. Then, based on the learning process feature information and the classifier in the learning behavior prediction model, the learning process feature information is classified and predicted to obtain a classification prediction result, that is, the learning behavior of the learner.

[0107] Among them, the specific execution process of the classifier is to obtain a classification probability vector, and then determine the learning behavior corresponding to the maximum classification probability value in the classification probability vector.

[0108] In one embodiment, the learning process data includes eye movement information, interaction information, and the course test scores of VR learning courses. Correspondingly, the feature extractor of the learning behavior prediction model includes a first feature extractor, a second feature extractor, and a third feature extractor.

[0109] Specifically, based on the first feature extractor in the trained learning behavior prediction model, eye movement feature information in the eye movement information is extracted. Based on the second feature extractor in the trained learning behavior prediction model, interaction feature information in the interaction information is extracted. Based on the third feature extractor in the trained learning behavior prediction model, performance feature information in the course test scores is extracted. Then, based on the eye movement feature information, interaction feature information, performance feature information, and the classifier in the learning behavior prediction model, the eye movement feature information, interaction feature information, and performance feature information are classified and predicted to obtain a classification prediction result, that is, the learning behavior of the learner.

[0110] To train the learning behavior prediction model, before the above step 211, this VR teaching method further includes:

[0111] Obtain learning process training data, and perform label annotation for learning behaviors on the learning process training data to obtain learning behavior label data; obtain a model to be trained, and select training sample data from the learning process training data and the learning behavior label data; based on the training sample data, perform iterative training on the model to be trained to obtain the learning behavior prediction model.

[0112] Step 212, input the learning process data into the cognitive style prediction model to perform cognitive style category prediction to obtain the cognitive style of the learner;

[0113] In this embodiment, the cognitive style prediction model is a machine learning model. Specifically, the cognitive style prediction model is a model obtained by performing iterative training on a model to be trained based on cognitive style training data. The cognitive style of the learner is the output of the cognitive style prediction model.

[0114] In a specific embodiment, the cognitive style prediction model includes a feature extractor and a classifier. Specifically, based on the feature extractor in the trained cognitive style prediction model, learning process feature information in the learning process data is extracted. Then, based on the learning process feature information and the classifier in the cognitive style prediction model, the learning process feature information is classified and predicted to obtain a classification prediction result, that is, the cognitive style of the learner.

[0115] Among them, the specific execution process of the classifier is to obtain a classification probability vector, and then determine the cognitive style corresponding to the maximum classification probability value in the classification probability vector.

[0116] In an embodiment, the learning process data includes eye movement information, interaction information, and the course test scores of the VR learning course. Correspondingly, the feature extractor of the cognitive style prediction model includes a first feature extractor, a second feature extractor, and a third feature extractor.

[0117] Specifically, based on the first feature extractor in the trained cognitive style prediction model, the eye movement feature information in the eye movement information is extracted. Based on the second feature extractor in the trained cognitive style prediction model, the interaction feature information in the interaction information is extracted. Based on the third feature extractor in the trained cognitive style prediction model, the performance feature information in the course test scores of the VR learning course is extracted. Then, based on the eye movement feature information, the interaction feature information, the performance feature information, and the classifier in the cognitive style prediction model, the eye movement feature information, the interaction feature information, and the performance feature information are classified and predicted to obtain a classification prediction result, that is, the cognitive style of the learner.

[0118] To train the cognitive style prediction model, before the above step 212, the VR teaching method further includes:

[0119] Obtain learning process training data, and perform label annotation for the cognitive style on the learning process training data to obtain cognitive style label data; obtain a model to be trained, and select training sample data from the learning process training data and the cognitive style label data; based on the training sample data, perform iterative training on the model to be trained to obtain the cognitive style prediction model.

[0120] Step 213, input the learning process data into the digital portrait prediction model for digital portrait category prediction, and obtain the digital portrait of the learner output by the digital portrait prediction model.

[0121] In this embodiment, the digital portrait prediction model is a machine learning model. Specifically, the digital portrait prediction model is a model obtained by performing iterative training on a model to be trained based on digital portrait training data. The digital portrait of the learner is the output of the digital portrait prediction model.

[0122] In a specific embodiment, the digital portrait prediction model includes a feature extractor and a classifier. Specifically, based on the feature extractor in the trained digital portrait prediction model, the learning process feature information in the learning process data is extracted. Then, based on the learning process feature information and the classifier in the digital portrait prediction model, the learning process feature information is classified and predicted to obtain a classification prediction result, that is, the digital portrait of the learner.

[0123] Among them, the specific execution process of the classifier is to obtain a classification probability vector, and then determine the digital portrait corresponding to the maximum classification probability value in the classification probability vector.

[0124] In an embodiment, the learning process data includes eye movement information, interaction information, and the course test scores of the VR learning course. Correspondingly, the feature extractor of the digital portrait prediction model includes a first feature extractor, a second feature extractor, and a third feature extractor.

[0125] Specifically, based on the first feature extractor in the trained digital portrait prediction model, extract the eye movement feature information in the eye movement information. Based on the second feature extractor in the trained digital portrait prediction model, extract the interaction feature information in the interaction information. Based on the third feature extractor in the trained digital portrait prediction model, extract the performance feature information in the course test scores. Then, based on the eye movement feature information, interaction feature information, performance feature information, and the classifier in the digital portrait prediction model, classify and predict the eye movement feature information, interaction feature information, and performance feature information to obtain the classification prediction result, that is, the digital portrait of the learner.

[0126] To train the digital portrait prediction model, before the above step 213, this VR teaching method further includes:

[0127] Obtain learning process training data, and perform label annotation on the learning process training data for the digital portrait to obtain digital portrait label data; obtain the model to be trained, and select training sample data from the learning process training data and the digital portrait label data; based on the training sample data, perform iterative training on the model to be trained to obtain the digital portrait prediction model.

[0128] To train the learning situation prediction model, in one embodiment, before the above step 210, this VR teaching method further includes:

[0129] Obtain learning process training data, and perform label annotation on the learning process training data for the learning situation information to obtain learning situation information label data; obtain the model to be trained, and select training sample data from the learning process training data and the learning situation information label data; based on the training sample data, perform iterative training on the model to be trained to obtain the learning situation prediction model.

[0130] Specifically, extract each learning process representation value in the learning process training data, and then based on each learning process representation value, match the corresponding learning situation information for the learning process training data, and then obtain the learning situation information label data.

[0131] Among them, the learning process training data includes at least one learning process.

[0132] In this embodiment, the training sample data includes at least one training sample, and one training sample includes one learning process from the learning process training data and one learning situation information label from the learning situation information label data.

[0133] Furthermore, divide the training sample data into a training set and a test set. For example, divide the training sample data into a training set and a test set according to a certain ratio. Among them, the training set is used to train the model, and the test set is used to test the model.

[0134] In this embodiment, training samples are selected from the training sample data, and the learning process and learning situation information corresponding to the training samples are input into the model to be trained. Model prediction is performed to obtain the model output labels. Then, the difference between the model output labels and the learning situation information labels corresponding to the training samples is calculated to obtain the model loss. Further, based on the model loss, the model to be trained is updated until the number of iterations of the model to be trained reaches the preset number of iterations or the corresponding loss function (objective function) reaches the preset value.

[0135] It should be noted that the appropriate number of iterations can be continuously adjusted in combination with the training effect. In addition, through gradient descent, the optimal weight value that minimizes the objective function can be found, and the weight value can be learned autonomously through training, and then the model to be trained is updated.

[0136] In addition, it should also be noted that training is performed using the training set to make the objective function as small as possible, and the test set is used to evaluate and verify the model after each round of training until the weights of the model are exported after the model converges, and then the final learning situation prediction model is obtained.

[0137] In this embodiment, based on the established and trained learning situation prediction model, the learning process data is automatically analyzed to obtain the learning situation information of the learner, so as to perform more accurate and personalized teaching based on the learning situation information obtained through intelligent analysis, thereby further improving the accuracy of VR teaching and ultimately further improving the intelligent level of VR teaching.

[0138] Furthermore, based on the above embodiments, a fourth embodiment of the VR teaching method of the present invention is proposed. Figure 2 It is the second flowchart of the VR teaching method provided by the present invention. As Figure 2 shown, in this embodiment, after the above step 200, the VR teaching method further includes:

[0139] Step 300, updating the learning information of the learner based on the learning situation information;

[0140] In this embodiment, the learning information of the learner includes learning behaviors, cognitive styles, digital portraits, etc. If the learning situation information also includes learning behaviors, cognitive styles, and digital portraits, then the learning behaviors, cognitive styles, and digital portraits in the learning situation information are used to replace the learning behaviors, cognitive styles, and digital portraits in the original learning information to update the learning information of the learner in real time.

[0141] Specifically, if it is detected that the learner needs to learn the VR learning course again, the learning information of the learner is updated based on the learning situation information. In addition, if it is detected that the learner does not need to learn the VR learning course again, the VR teaching system is exited, and the learning information of the learner may not be updated.

[0142] In one embodiment, step 300 above includes:

[0143] Send the learning situation information to the VR teaching module; update the learning information of the learner in the VR teaching module based on the learning situation information. It should be noted that updating the learning information of the learner in the VR teaching module can enable the VR teaching module to conduct teaching based on the latest learning information of the learner.

[0144] Step 400, update the VR learning course based on the learning information for the learner to learn through the updated VR learning course.

[0145] Specifically, generate the learning assistance of the learner based on the learning information; update the VR learning course based on the learning assistance for the learner to learn through the updated VR learning course, that is, learn the VR learning course again according to the learning assistance.

[0146] In one embodiment, update the VR learning course in the VR teaching module based on the learning information for the learner to learn through the updated VR learning course.

[0147] In this embodiment, the learning information of the learner is updated based on the learning situation information, and then the VR learning course is updated, so that when the learner learns the VR learning course again, personalized settings can be made according to his previous learning situation, thus meeting the personalized needs of the learner. Compared with the unchanged VR learning course, the intelligence level of VR teaching is further improved in this embodiment.

[0148] The VR teaching device provided by the present invention is described below. The VR teaching device described below can be mutually corresponding and referred to the VR teaching method described above.

[0149] Figure 3 It is a schematic diagram of the VR teaching device provided by the present invention, as Figure 3 shown, the VR teaching device provided by the present invention includes:

[0150] A data acquisition device 310, configured to collect the learning process data of the learner when it is detected that the learner learns through the VR learning course;

[0151] The learning situation analysis device 320 is used to analyze the learning situation categories of the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance.

[0152] Figure 4 An entity structure diagram of an electronic device is illustrated, as Figure 4 shown. The electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logical instructions in the memory 430 to execute the VR teaching method, and the method includes: when it is detected that a learner is learning through a VR learning course, collecting the learning process data of the learner; performing a learning situation category analysis on the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance.

[0153] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0154] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the VR teaching method provided by the above-mentioned various methods. The method includes: when it is detected that a learner is learning through a VR learning course, collecting the learning process data of the learner; performing a learning situation category analysis on the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance.

[0155] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the VR teaching method provided by the above-mentioned various methods. The method includes: when it is detected that a learner is learning through a VR learning course, collecting the learning process data of the learner; performing an analysis of the learning situation category on the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0157] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A VR teaching method, characterized in that Including: When it is detected that the learner is learning through a VR learning course, collect the learning process data of the learner; The learning process data includes eye movement information, interaction information, and the course test scores of the VR learning course; Perform an analysis of the learning situation category on the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance; The performing an analysis of the learning situation category on the learning process data to obtain the learning situation information of the learner includes: Input the learning process data into the trained learning situation prediction model for learning situation category prediction, and obtain the learning situation information of the learner output by the learning situation prediction model; The learning situation prediction model includes a learning behavior prediction model, a cognitive style prediction model, and a digital portrait prediction model. The inputting the learning process data into the trained learning situation prediction model for learning situation category prediction and obtaining the learning situation information of the learner output by the learning situation prediction model includes: Input the learning process data into the learning behavior prediction model for learning behavior category prediction, and obtain the learning behavior of the learner output by the learning behavior prediction model; Input the learning process data into the cognitive style prediction model for cognitive style category prediction to obtain the cognitive style of the learner; Input the learning process data into the digital portrait prediction model for digital portrait category prediction, and obtain the digital portrait of the learner output by the digital portrait prediction model; The inputting the learning process data into the learning behavior prediction model for learning behavior category prediction and obtaining the learning behavior of the learner output by the learning behavior prediction model includes: Based on the first feature extractor in the learning behavior prediction model, extract the eye movement feature information in the eye movement information; Based on the second feature extractor in the learning behavior prediction model, extract the interaction feature information in the interaction information; Based on the third feature extractor in the learning behavior prediction model, extract the score feature information in the course test scores; According to the classifier in the learning behavior prediction model, perform learning behavior category prediction on the eye movement feature information, the interaction feature information, and the score feature information, and obtain the learning behavior of the learner output by the classifier.

2. The VR teaching method according to claim 1, wherein, The collecting the learning process data of the learner includes: Through an eye movement device, collect the eye movement information and the first interaction information of the learner; and / or, Through a control device, collect the second interaction information of the learner, and the control device is a device for the learner to control the VR learning course.

3. The VR teaching method according to claim 2, wherein The through an eye movement device, collecting the eye movement information and the first interaction information of the learner includes: Through a head-mounted VR eye movement device, collect the eye movement information, the first interaction information, and the course test scores of the VR learning course.

4. The VR teaching method according to any one of claims 1 to 3, characterized in that After the performing an analysis of the learning situation category on the learning process data to obtain the learning situation information of the learner, it further includes: Based on the learning situation information, update the learning information of the learner; Update the VR learning course based on the learning information for the learner to learn through the updated VR learning course.

5. A VR teaching device, characterized in that, Including: A data acquisition device for collecting the learning process data of the learner when it is detected that the learner is learning through the VR learning course; The learning process data includes eye movement information, interaction information, and the course test scores of the VR learning course; A learning situation analysis device for performing a learning situation category analysis on the learning process data to obtain the learning situation information of the learner, and the learning situation information is used for intelligent learning guidance; The performing a learning situation category analysis on the learning process data to obtain the learning situation information of the learner includes: Inputting the learning process data into a trained learning situation prediction model for learning situation category prediction, and obtaining the learning situation information of the learner output by the learning situation prediction model; The learning situation prediction model includes a learning behavior prediction model, a cognitive style prediction model, and a digital portrait prediction model. The inputting the learning process data into a trained learning situation prediction model for learning situation category prediction, and obtaining the learning situation information of the learner output by the learning situation prediction model includes: Inputting the learning process data into the learning behavior prediction model for learning behavior category prediction, and obtaining the learning behavior of the learner output by the learning behavior prediction model; Inputting the learning process data into the cognitive style prediction model for cognitive style category prediction, and obtaining the cognitive style of the learner; Inputting the learning process data into the digital portrait prediction model for digital portrait category prediction, and obtaining the digital portrait of the learner output by the digital portrait prediction model; The inputting the learning process data into the learning behavior prediction model for learning behavior category prediction, and obtaining the learning behavior of the learner output by the learning behavior prediction model includes: Extracting the eye movement feature information in the eye movement information based on the first feature extractor in the learning behavior prediction model; Extracting the interaction feature information in the interaction information based on the second feature extractor in the learning behavior prediction model; Extracting the score feature information in the course test scores based on the third feature extractor in the learning behavior prediction model; Performing learning behavior category prediction on the eye movement feature information, the interaction feature information, and the score feature information according to the classifier in the learning behavior prediction model, and obtaining the learning behavior of the learner output by the classifier.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein When the processor executes the program, it implements the steps of the VR teaching method according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the VR teaching method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the VR teaching method according to any one of claims 1 to 4.

Images