Learning imaging method and device based on VR interaction, electronic equipment and medium

Abstract

The invention discloses a learning imaging method and device based on VR interaction, electronic equipment and a medium, and the method comprises the steps: obtaining multi-modal emotional physiological data of a target learner in a current VR scene, and carrying out the extraction of an emotional physiological feature vector according to the multi-modal emotional physiological data; comparing the emotional physiological feature vector with a pre-constructed personalized emotional physiological baseline model of the target learner, and judging whether the learning state of the target learner is normal or not; and when the learning state of the target learner is abnormal, obtaining target learning content which is highly related to the current learning content and is interested by the target learner, and generating a target VR scene according to the target learning content. The method improves the learning efficiency and learning experience of the learner, and can be widely applied to the technical field of virtual reality.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a method, device, electronic equipment and medium for learning imaging based on VR interaction. Background Art

[0002] In the prior art, the following technical problems exist in the process of learning through VR interaction:

[0003] First, learners’ emotional changes and physiological data can reflect their interest and involvement in the learning content, but the collection and analysis of these data are difficult. There are many types of emotional and physiological signals, which vary greatly between learners, and there is a lot of noise in the data, resulting in uneven data quality. In addition, the correlation between emotional and physiological data and learning interest and involvement is difficult to accurately model, which is prone to bias and misjudgment.

[0004] Secondly, even if we can accurately judge the learner's interest and level of engagement, it is still a big challenge to adjust the presentation method, style and learning path in real time. Different learners have different preferences for presentation methods and styles, and excessive adjustments may be counterproductive. The optimization of the learning path needs to take into account the logical order and knowledge system of the learning content. Adjustment based solely on the learner's interests may lead to fragmentation of learning content, affecting the systematicness and integrity of knowledge.

[0005] Finally, the learning process in VR scenarios is highly immersive and interactive. The evaluation of learning experience cannot be limited to interest and involvement, but also needs to consider factors such as the smoothness of VR interactive operations, the timeliness of scene responses, and overall immersion. The performance limitations of VR devices and network transmission may also affect the learning experience, making optimization more complicated.

[0006] The above problems need to be solved urgently.

[0007] Terminology explanation:

[0008] Knowledge graph: A technology that uses graph structures to represent knowledge and information, which can help people better understand and use knowledge in various fields. It consists of nodes (entities) and edges (relationships) and is used to describe various concepts, entities and the relationships between them in the real world. It represents knowledge in a structured way, allowing computers to better understand and process human knowledge.

[0009] Natural language interaction system: A computer system that can understand human natural language and respond in natural language. The natural language interaction system aims to enable communication between people and computers in the natural language used by humans in daily life, without the need for users to learn specific programming languages ​​or instruction formats. It can understand the natural language expressions in the form of text or voice input by users and generate corresponding natural language responses to meet the user's needs for information query, task execution or communication interaction. Summary of the invention

[0010] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0011] To this end, an object of an embodiment of the present invention is to provide a learning imaging method based on VR interaction, which analyzes the learner's interest and involvement in the learning content based on emotional changes and physiological data during the learning process, and adjusts the subsequent learning content as well as the presentation method, style and learning path in real time, thereby improving the learner's learning efficiency and learning experience.

[0012] Another object of an embodiment of the present invention is to provide a learning imaging device based on VR interaction.

[0013] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0014] On the one hand, an embodiment of the present invention provides a method for learning imaging based on VR interaction, comprising the following steps:

[0015] Acquire multimodal emotional physiological data of a target learner in a current VR scene, and extract an emotional physiological feature vector according to the multimodal emotional physiological data;

[0016] Comparing the emotional physiological feature vector with a pre-constructed personalized emotional physiological baseline model of the target learner to determine whether the learning state of the target learner is normal;

[0017] When the learning state of the target learner is abnormal, target learning content that is highly relevant to the current learning content and that the target learner is interested in is obtained, and a target VR scene is generated according to the target learning content.

[0018] Further, in one embodiment of the present invention, the multimodal emotional physiological data includes facial expression data, eye movement data, heart rate data and brain wave data, and the emotional physiological feature vector is extracted according to the multimodal emotional physiological data, which specifically includes:

[0019] Performing emotion recognition on the target learner according to the facial expression data to obtain emotional state characteristics;

[0020] Determining sight line change data of the target learner according to the eye movement data, and determining a gaze point heat map according to the sight line change data, and then extracting attention distribution features according to the gaze point heat map;

[0021] Performing heart rate variability analysis on the target learner according to the heart rate data to obtain stress level characteristics;

[0022] Performing frequency domain analysis on the brain wave data to obtain energy distribution information of each frequency band, and determining cognitive load characteristics according to the energy distribution information;

[0023] The emotional state feature, the attention distribution feature, the stress level feature, and the cognitive load feature are normalized and fused to obtain the emotional physiological feature vector.

[0024] Furthermore, in one embodiment of the present invention, the personalized emotional physiological baseline model is constructed by the following steps:

[0025] Acquire personal attribute information of a plurality of target learners, and acquire a target emotional physiological feature vector of each target learner in a normal learning state;

[0026] Performing cluster analysis on the target learners according to the personal attribute information and the target emotional physiological feature vector to obtain a plurality of user groups;

[0027] Performing numerical statistics on the target emotional physiological characteristic vectors of the target learners of the same user group in each dimension, determining the emotional physiological characteristic baseline value of the corresponding dimension according to the mean value of each dimension, and determining the emotional physiological characteristic fluctuation range of the corresponding dimension according to the standard deviation of each dimension;

[0028] Determining an emotional physiological baseline model of the user group according to the emotional physiological characteristic baseline value and the emotional physiological characteristic fluctuation range;

[0029] The emotional physiological baseline model is adjusted according to the personal attribute information of each target learner in the user group to obtain a personalized emotional physiological baseline model for each target learner.

[0030] Further, in one embodiment of the present invention, comparing the emotional physiological feature vector with a pre-constructed personalized emotional physiological baseline model of the target learner to determine whether the learning state of the target learner is normal specifically includes:

[0031] Determining target emotional physiological characteristic values ​​of the target learner in each dimension according to the emotional physiological characteristic vector;

[0032] Determine the target emotional physiological characteristic baseline value and target emotional physiological characteristic fluctuation range of the target learner in each dimension according to the personalized emotional physiological baseline model;

[0033] Determine the emotional physiological characteristic deviation value of the corresponding dimension according to the difference between the target emotional physiological characteristic value and the target emotional physiological characteristic baseline value of the corresponding dimension, and judge whether the emotional physiological characteristic deviation value is within the emotional physiological characteristic fluctuation range of the corresponding dimension;

[0034] When the deviation values ​​of the emotional physiological characteristics of each dimension are all within the corresponding fluctuation range of the emotional physiological characteristics, it is determined that the learning state of the target learner is normal;

[0035] When the deviation value of the emotion-physiological characteristic of at least one dimension is not within the corresponding fluctuation range of the emotion-physiological characteristic, it is determined that the learning state of the target learner is abnormal.

[0036] Further, in one embodiment of the present invention, the step of acquiring target learning content that is highly relevant to the current learning content and that the target learner is interested in, and generating a target VR scene according to the target learning content, specifically includes:

[0037] Determine, according to a preset learning content knowledge graph, candidate learning content whose relevance to the current learning content is greater than or equal to a preset first threshold;

[0038] Determining the target learning content that the target learner is interested in from the candidate learning content by using a collaborative filtering algorithm;

[0039] Determining the cognitive load level of the target learner according to the emotional physiological feature vector, and determining the cognitive load state of the target learner according to the cognitive load level and a preset cognitive load threshold;

[0040] The learning task setting, plot development setting and interaction mode setting are performed according to the target learning content, and the information presentation mode and information presentation style are adjusted according to the cognitive load state to obtain the target VR scene.

[0041] Furthermore, in one embodiment of the present invention, the learning imaging method further comprises the following steps:

[0042] Acquiring hand movement data and eye movement data of the target learner;

[0043] Performing gesture recognition on the target learner according to the hand motion data to obtain operation intention information;

[0044] Determining the focus information of the target learner according to the eye movement data;

[0045] The VR interaction mode is optimized in real time according to the operation intention information, the focus information and the personal preference / habit information of the target learner.

[0046] Furthermore, in one embodiment of the present invention, the learning imaging method further comprises the following steps:

[0047] Determining the target learner's emotional engagement and cognitive load level according to the emotional physiological feature vector;

[0048] Determine a plurality of knowledge points according to the target learning content, and calculate the knowledge point association between the knowledge points;

[0049] Acquiring the interactive behavior data of the target learner, and determining the interactive behavior pattern of the target learner according to the interactive behavior data;

[0050] According to the emotional engagement, the cognitive load level, the knowledge point relevance and the interactive behavior pattern, the learning path of each of the knowledge points is optimized through a reinforcement learning algorithm.

[0051] On the other hand, an embodiment of the present invention provides a learning imaging device based on VR interaction, comprising:

[0052] An emotion physiological feature extraction module is used to obtain multimodal emotion physiological data of the target learner in the current VR scene, and extract an emotion physiological feature vector based on the multimodal emotion physiological data;

[0053] An emotion-physiology feature comparison module, used to compare the emotion-physiology feature vector with a pre-built personalized emotion-physiology baseline model of the target learner to determine whether the learning state of the target learner is normal;

[0054] The target VR scene generation module is used to obtain target learning content that is highly relevant to the current learning content and that the target learner is interested in when the learning state of the target learner is abnormal, and generate a target VR scene based on the target learning content.

[0055] On the other hand, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the program is executed by the processor, the learning imaging method based on VR interaction as described above is realized.

[0056] On the other hand, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the VR interaction-based learning imaging method as described above.

[0057] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0058] The embodiment of the present invention obtains the multimodal emotional physiological data of the target learner in the current VR scene, extracts the emotional physiological feature vector based on the multimodal emotional physiological data, compares the emotional physiological feature vector with the pre-constructed personalized emotional physiological baseline model of the target learner, and determines whether the learning state of the target learner is normal. When the learning state of the target learner is abnormal, the target learning content that is highly relevant to the current learning content and of interest to the target learner is obtained, and a target VR scene is generated based on the target learning content. Based on the emotional changes and physiological data during the learning process, the embodiment of the present invention analyzes the learner's interest and involvement in the learning content, adjusts the subsequent learning content as well as the presentation method, style and learning path in real time, and improves the learner's learning efficiency and learning experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 A flowchart of a method for learning imaging based on VR interaction provided by an embodiment of the present invention;

[0061] Figure 2 A step flow chart of step S101 provided in an embodiment of the present invention;

[0062] Figure 3 A flowchart of steps for constructing a personalized emotional physiological baseline model provided by an embodiment of the present invention;

[0063] Figure 4 A step flow chart of step S102 provided in an embodiment of the present invention;

[0064] Figure 5 A step flow chart of step S103 provided in an embodiment of the present invention;

[0065] Figure 6 Another step flow chart of the VR interactive learning imaging method provided in an embodiment of the present invention;

[0066] Figure 7 Another step flow chart of the VR interactive learning imaging method provided in an embodiment of the present invention;

[0067] Figure 8 A schematic diagram of the structure of a VR interactive learning imaging device provided in an embodiment of the present invention;

[0068] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0069] Fig.10 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. It should be noted that, although the functional module division is performed in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order from the module division in the system schematic diagram or the flow chart. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0071] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of the first or the second, it is only for the purpose of distinguishing the technical features, and it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by technicians in the technical field of this application. The terms used in this document are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0072] The VR-interactive learning imaging method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the VR-interactive learning imaging method, etc., but is not limited to the above forms.

[0073] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0074] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0075] like Figure 1 FIG. 1 is a flowchart of a method for learning imaging based on VR interaction provided by an embodiment of the present invention, referring to FIG. Figure 1 The embodiment of the present invention provides a method for learning imaging based on VR interaction, which specifically includes the following steps:

[0076] S101, obtaining multimodal emotional physiological data of a target learner in a current VR scene, and extracting an emotional physiological feature vector based on the multimodal emotional physiological data.

[0077] Specifically, first, on the VR platform, pre-edit the learning scene and learning content to generate an initialized learning scene, for example, obtain the preset learning objectives and syllabus, determine the theme and element composition of the VR learning scene according to the learning content, and generate an initialized virtual learning environment. Enter the learning scene, and in the VR learning process, collect learners' multimodal emotional physiological signal data such as facial expressions, eye movements, heart rate and EEG during the VR learning process. Through data cleaning and feature extraction, remove noise and redundancy in the signal, improve data quality, and obtain standardized emotional physiological feature vectors.

[0078] like Figure 2 FIG. 1 is a flowchart of step S101 provided in an embodiment of the present invention, referring to FIG. Figure 2 As an optional implementation, the multimodal emotional physiological data includes facial expression data, eye movement data, heart rate data and brain wave data. The emotional physiological feature vector is extracted based on the multimodal emotional physiological data, which specifically includes:

[0079] S1011, performing emotion recognition on the target learner according to the facial expression data to obtain the emotional state characteristics;

[0080] S1012, determining the sight line change data of the target learner according to the eye movement data, and determining the gaze point heat map according to the sight line change data, and then extracting the attention distribution feature according to the gaze point heat map;

[0081] S1013, performing heart rate variability analysis on the target learner according to the heart rate data to obtain stress level characteristics;

[0082] S1014, performing frequency domain analysis on the brain wave data to obtain energy distribution information of each frequency band, and determining cognitive load characteristics based on the energy distribution information;

[0083] S1015. Normalize and fuse the emotional state features, attention distribution features, stress level features, and cognitive load features to obtain an emotional physiological feature vector.

[0084] Specifically, facial expression recognition technology is used to obtain learners' expression data from the VR environment, and the learners' basic emotional state is determined through facial expression analysis algorithms. Eye tracking technology is used to collect learners' eye movement data, analyze their focus areas and frequent change points, and evaluate attention distribution and learning focus. Heart rate monitoring equipment is used to record learners' heart rate changes, and heart rate variability is analyzed to determine learners' emotional fluctuations and stress levels. By collecting brain wave signals, frequency analysis is performed to identify learners' cognitive load and mental state. The collected multimodal data is cleaned to remove interference caused by environmental noise and equipment errors. Key physiological and emotional features are extracted from the cleaned data, and the extracted key physiological and emotional features are normalized by the Z-score standardization method, and different modal features are mapped to the same numerical range to obtain standardized emotional physiological feature vectors.

[0085] For example, in a VR learning environment, a high-definition camera is used to collect learners' facial images, and the OpenCV library is used for image preprocessing, such as grayscale and histogram equalization. Then, an Action Unit-based facial expression recognition algorithm, such as SVM or random forest, is used to classify learners' micro-expressions and identify basic emotions such as happiness, surprise, disgust, and fear. At the same time, the built-in eye tracking module of the VR headset records the learner's eye movement trajectory and gaze point at a frequency of 120Hz. Through speed threshold algorithms such as I-VT, events such as eye saccades, gazes, and blinks are extracted, and the gaze point heat map is calculated to quantify the learner's attention to different knowledge points. In addition, the smart bracelet worn by the learner collects ECG signals in real time, and uses time-frequency analysis methods such as wavelet transform to extract features such as heart rate and HRV, and the learner's stress level is judged through a logistic regression model. The learners' EEG signals are recorded at a sampling rate of 1kHz by wearing 64 electrodes on the EEG electrode cap. The energy distribution of the α, β, θ, δ and other frequency bands is analyzed using algorithms such as fast Fourier transform. Combined with eye movement data, the learners' cognitive load and level of involvement are evaluated.

[0086] S102: Compare the emotional physiological feature vector with a pre-constructed personalized emotional physiological baseline model of the target learner to determine whether the learning state of the target learner is normal.

[0087] Specifically, during the VR learning process, the learner's emotional physiological characteristics are tracked in real time, and the learner's current emotional state and attention level are judged by comparing them with the personalized emotional physiological baseline model. If the deviation from the baseline model exceeds the preset threshold, a warning of learning interest and involvement level is triggered.

[0088] Based on the learners' personal attribute information, such as age, gender, learning style, etc., user profiling and clustering analysis are performed on the emotional physiological feature vectors to identify the emotional physiological patterns of different user groups and establish a personalized emotional physiological baseline model for subsequent judgment of learning interest and involvement.

[0089] like Figure 3 The figure is a flowchart of the steps of constructing a personalized emotional physiological baseline model provided by an embodiment of the present invention, referring to Figure 3 As an optional implementation, the personalized emotional physiological baseline model is constructed by the following steps:

[0090] S201, obtaining personal attribute information of multiple target learners, and obtaining a target emotional physiological feature vector of each target learner in a normal learning state;

[0091] S202, performing cluster analysis on target learners according to personal attribute information and target emotional physiological feature vectors to obtain several user groups;

[0092] S203, performing numerical statistics on the target emotional physiological characteristic vectors of the target learners of the same user group in each dimension, determining the emotional physiological characteristic baseline value of the corresponding dimension according to the mean value of each dimension, and determining the emotional physiological characteristic fluctuation range of the corresponding dimension according to the standard deviation of each dimension;

[0093] S204, determining an emotional physiological baseline model of the user group according to the emotional physiological characteristic baseline value and the emotional physiological characteristic fluctuation range;

[0094] S205 , adjusting the emotional physiological baseline model according to the personal attribute information of each target learner in the user group to obtain a personalized emotional physiological baseline model for each target learner.

[0095] Specifically, learners’ personal attribute information, including age, gender, and learning style, is obtained and used as the basic data for user portraits. Based on learners’ personal attribute information and emotional physiological feature vectors, learners are clustered and analyzed to identify different user groups. For each user group, the commonality and individuality of their emotional physiological characteristics are analyzed to establish an emotional physiological baseline model at the group level. Based on the baseline model at the group level, a personalized emotional physiological baseline model is established in combination with learners’ personal attributes.

[0096] Exemplarily, before learning, personal attribute information of learners, such as age, gender, learning style, etc., is obtained through questionnaires and other methods, and quantified into feature vectors as basic data for user portraits. Then, clustering algorithms such as K-means are used to perform cluster analysis on learners based on their personal attribute feature vectors and emotional physiological feature vectors to identify different user groups. For each user group, the commonality and individuality of their emotional physiological characteristics are analyzed, and a group-level emotional physiological baseline model is established using statistical methods, such as using the mean and standard deviation to describe the normal emotional physiological level of the group. On this basis, combined with the learner's personal attributes, machine learning algorithms such as support vector machines are used to establish a personalized emotional physiological baseline model to describe the normal emotional physiological level of the learner under different learning states.

[0097] like Figure 4 FIG. 1 is a flow chart of step S102 provided in an embodiment of the present invention, referring to FIG. Figure 4 As an optional implementation, the emotional physiological feature vector is compared with a pre-built personalized emotional physiological baseline model of the target learner to determine whether the learning state of the target learner is normal, which specifically includes:

[0098] S1021, determining target emotional physiological characteristic values ​​of the target learner in each dimension according to the emotional physiological characteristic vector;

[0099] S1022. Determine the target emotional physiological characteristic baseline values ​​and target emotional physiological characteristic fluctuation ranges of the target learner in each dimension according to the personalized emotional physiological baseline model;

[0100] S1023, determining the emotion physiological characteristic deviation value of the corresponding dimension according to the difference between the target emotion physiological characteristic value and the target emotion physiological characteristic baseline value of the corresponding dimension, and judging whether the emotion physiological characteristic deviation value is within the emotion physiological characteristic fluctuation range of the corresponding dimension;

[0101] S1024, when the deviation values ​​of the emotional physiological characteristics of each dimension are all within the corresponding emotional physiological characteristic fluctuation range, it is determined that the learning state of the target learner is normal;

[0102] S1025: When the deviation value of the emotion-physiology characteristic of at least one dimension is not within the corresponding emotion-physiology characteristic fluctuation range, it is determined that the learning state of the target learner is abnormal.

[0103] Specifically, sensors are used to obtain the learner's real-time emotional physiological characteristic data during VR learning, including facial expressions, eye movements, heart rate, and EEG signals. The acquired real-time emotional physiological characteristic data is compared with the pre-established personalized emotional physiological baseline model, and the machine learning algorithm is used to determine whether the learner's current target emotional physiological characteristic values ​​in each dimension are normal; taking emotional state and attention level as an example, if the learner's emotional state and attention level deviate from the personalized baseline model by more than the preset threshold, a warning of learning interest and involvement is triggered; during the VR learning process, the learner's emotional physiological characteristic changes are continuously tracked, and the personalized emotional physiological baseline model is updated in real time to improve the accuracy of emotional state and attention level judgment. When the learning interest and involvement warning is triggered multiple times in a row, a dialogue is conducted with the learner through the natural language interaction system to understand the learner's difficulties and needs, and provide personalized learning guidance and encouragement.

[0104] For example, in a virtual reality learning environment, emotional physiological characteristic data including facial expressions, eye movements, heart rate, and EEG signals are collected in real time through wearable smart bracelet devices, head-mounted display devices, EEG devices, and high-definition cameras. These data are transmitted to the processing center in real time via Bluetooth or Wi-Fi, and the facial expression images are analyzed using neural network algorithms, such as convolutional neural networks (CNNs), while fast Fourier transform (FFT) is used to perform frequency domain analysis on heart rate and EEG signals, thereby accurately identifying the learner's emotional state and attention level.

[0105] When the system detects that the learner's emotions or attention levels deviate from the personalized baseline model by more than a preset threshold, such as when the emotional state exceeds the baseline by 20% or the attention drops by more than 15%, the system will automatically trigger the early warning mechanism. In addition, the system continuously tracks and records the emotional and physiological data of each learner, and uses machine learning algorithms such as support vector machines (SVM) to continuously update and optimize the personalized emotional and physiological baseline model. When the system triggers an early warning multiple times in a row, the natural language processing system will be activated to interact with the learner through a virtual assistant, asking about his or her current difficulties and needs, while providing personalized learning suggestions and encouragement.

[0106] S103: When the learning state of the target learner is abnormal, obtain target learning content that is highly relevant to the current learning content and that the target learner is interested in, and generate a target VR scene according to the target learning content.

[0107] Specifically, based on the warning of learning interest and involvement, knowledge points that are highly relevant to the current learning content and of interest to learners are recommended from the learning content knowledge graph. By changing the task settings, plot development and interaction methods in the VR scene, learners are guided to refocus their attention on learning.

[0108] like Figure 5 FIG. 1 is a flow chart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 5 As an optional implementation, target learning content that is highly relevant to the current learning content and of interest to the target learner is obtained, and a target VR scene is generated according to the target learning content, which specifically includes:

[0109] S1031, determining, according to a preset learning content knowledge graph, candidate learning content whose relevance to the current learning content is greater than or equal to a preset first threshold;

[0110] S1032, determining the target learning content that the target learner is interested in from the candidate learning content through a collaborative filtering algorithm;

[0111] S1033, determining the cognitive load level of the target learner according to the emotional physiological feature vector, and determining the cognitive load state of the target learner according to the cognitive load level and a preset cognitive load threshold;

[0112] S1034, setting learning tasks, plot development and interaction methods according to the target learning content, and adjusting the information presentation method and information presentation style according to the cognitive load state to obtain the target VR scene.

[0113] Specifically, based on the learner's learning behavior data, a learner portrait is constructed to obtain the learner's interest preferences and knowledge mastery; based on the learning content knowledge graph, a collaborative filtering algorithm is used to find knowledge points that are highly relevant to the current learning content and of interest to the learners; based on the recommended knowledge points, a matching VR learning scene is designed to increase learner participation by setting attractive learning tasks and plots; in the VR learning scene, the task difficulty and plot direction are dynamically adjusted, and based on the learner's real-time feedback, the learner's attention is always focused on the learning content; a machine learning algorithm is used to analyze the learner's learning behavior and physiological feedback data in real time to determine whether the learner's level of involvement has decreased; if it is detected that the learner's level of involvement has decreased, an early warning mechanism is triggered to recommend new knowledge points of interest, and the VR scene settings are adjusted accordingly; based on the learner's learning effects and feedback, the recommendation algorithm and VR scene generation strategy are optimized to continuously improve the accuracy of the early warning of learning interest and involvement.

[0114] For example, in order to construct a learner portrait, we can first collect the learner's behavior data on the VR platform, such as click-through rate, dwell time, homework completion, etc., use the scikit-learn library in Python, and use the decision tree algorithm to analyze the data to determine the learner's interest preference and knowledge mastery. For example, by analyzing the behavior data of 1,000 learners, it is found that 80% of the learners stay longer than the average at a certain policy learning location, and the homework completion rate reaches 90%, so it is judged that these learners have a high interest and mastery of this. Then, according to the knowledge graph of the learning content, a user-based collaborative filtering algorithm is used, and the cosine similarity is used to calculate the interest between the learner and the knowledge point. Assuming that the current learning content is the support policy, the algorithm calculates that the interest of the knowledge point industry support with a high degree of relevance to the support policy is 8. Based on this, a VR learning scene that matches statistics can be designed. The scene contains a virtual market survey task, and learners need to complete the survey report by collecting and analyzing data. The task difficulty is divided into three levels, elementary, intermediate and advanced, which are dynamically adjusted according to the learner's knowledge mastery. For example, for beginners, the system will provide more prompts and guidance, while for advanced learners, they need to solve more complex data analysis problems on their own. In the VR learning scene, the convolutional neural network algorithm under the TensorFlow framework is used to analyze the learner's behavioral data and physiological feedback data in real time, such as eye tracking, heart rate changes, etc. By setting thresholds, such as eye tracking frequency is lower than 20 times per minute or heart rate is higher than 100 times per minute, the system will determine that the learner's level of engagement may decrease. If a decrease in engagement is detected, the system will trigger an early warning mechanism, recommend new knowledge points of interest, and increase learners' operational interaction. Finally, based on the learners' learning effects and feedback, the deep learning algorithm is used to optimize the recommendation algorithm and VR scene generation strategy. At the same time, based on the learners' feedback, the task design and plot direction in the VR scene are optimized, such as increasing the scene interactivity and task challenge, thereby improving the accuracy of the early warning of learning interest and engagement.

[0115] In some optional embodiments, while recommending knowledge points of interest, the information presentation method and style in the VR scene are dynamically adjusted according to the learner's cognitive load level, such as reducing irrelevant information, increasing the display ratio and duration of key information, optimizing the interface layout and color matching, etc., to avoid learner cognitive overload.

[0116] Specifically, the learner's cognitive load level data is obtained, and the learner's current cognitive load status is judged according to the preset cognitive load threshold; if the cognitive load exceeds the threshold, the information presentation method in the VR scene is dynamically adjusted, and the strategies adopted include reducing the display ratio of irrelevant information in the scene and shortening its duration; increasing the display ratio of key information and extending its duration; according to the cognitive load level, the interface layout of the VR scene is dynamically optimized, and the position, size, spacing, etc. of the interface elements are adjusted; according to the cognitive load level, the color matching of the VR scene is dynamically adjusted, and soft and comfortable color combinations are selected to avoid using overly bright and stimulating colors.

[0117] For example, in order to realize personalized and intelligent VR learning experience, the learner's cognitive load level data is first obtained through eye tracking technology and heart rate monitoring. For example, eye tracking technology is used to record the learner's gaze point, gaze time and scanning path in the VR scene, and the learner's heart rate variability is recorded through heart rate monitoring equipment. These data are combined with the cognitive load assessment model, such as linear regression analysis, and the cognitive load threshold is set to 70%. When the calculated cognitive load index exceeds this threshold, the system will automatically trigger the adjustment of the information presentation method. Specifically, the display ratio of irrelevant information in the scene will be reduced from 30% to 10%, and the duration will be shortened from 5 seconds to 2 seconds; the display ratio of key information will be increased from the original 50% to 80%, and the duration will be extended from 3 seconds to 6 seconds. In terms of interface layout optimization, machine learning algorithms, such as models based on gradient boosting decision trees, are used to dynamically adjust the position, size, spacing and other parameters of interface elements, such as increasing the size of key buttons by 20% and moving the position to the center right of the line of sight to improve the convenience of operation. In terms of color matching, the system will select color combinations based on the cognitive load level. For example, it will use the Lab color space algorithm to select soft tones that match the current psychological state of the learner, such as reducing the color brightness by 20% and shifting the tones to a warmer area.

[0118] like Figure 6 FIG. 1 is another flow chart of the steps of the VR interactive learning imaging method provided by an embodiment of the present invention, referring to FIG. Figure 6 As an optional embodiment, the learning imaging method further includes the following steps:

[0119] S301, obtaining hand movement data and eye movement data of a target learner;

[0120] S302, performing gesture recognition on the target learner according to the hand motion data to obtain operation intention information;

[0121] S303, determining the focus information of the target learner according to the eye movement data;

[0122] S304: Optimize the VR interaction mode in real time according to the operation intention information, the focus information, and the personal preference / habit information of the target learner.

[0123] When learners interact with VR scenes, the embodiments of the present invention obtain the learners' operation intentions and focus through technologies such as hand movement recognition and eye tracking, and optimize the VR interaction mode in real time according to the learners' personal preferences and habits, thereby improving the fluency and accuracy of the operation.

[0124] Specifically, through gesture recognition technology, learners' specific hand movement data is obtained, and these data are analyzed to identify their operation intentions. This step uses machine learning models to classify gestures and analyze their intentions. Eye tracking technology is used to record learners' eye movements and gaze points to determine their focus. Through data analysis, the areas that learners are most interested in in the VR environment are identified. Based on the data of operation intentions and focus points, learners' personal preferences and habits are analyzed through algorithm models, so as to make real-time adjustments to the VR interaction method. This process uses a real-time feedback mechanism to ensure the adaptability and personalization of the interaction method. Through the above-mentioned adjusted interaction method, the smoothness and accuracy of the operation are continuously monitored, and the model is continuously optimized through deep learning algorithms to improve the overall interaction effect. Integrate gesture recognition and eye tracking data, and improve the accuracy and efficiency of data processing through fusion learning algorithms to ensure that learners' experience in the VR environment is more natural and efficient. In addition, a user interface can be developed to allow learners to adjust VR settings according to their needs, such as adjusting sensitivity, selecting interaction modes, etc., so as to further personalize the learning experience.

[0125] Exemplarily, the gesture image is preprocessed by a deep learning algorithm, such as using a convolutional neural network to extract features from hand motion data, and the extracted features are input into a support vector machine for classification, with a recognition accuracy of 95%. In this process, 10,000 hand motion samples were collected, of which 80% were training sets, 10% were validation sets, and 10% were test sets. At the same time, eye tracking technology was used to record learners' eye movements and gaze points, and the gaze point data was analyzed using a K-means clustering algorithm to determine the learners' focus in the VR environment, with an accuracy of 90%. On this basis, combined with gesture recognition and eye tracking data, an association rule mining algorithm was used to analyze the relationship between learners' operating intentions and focus, so as to adjust the VR interaction mode in real time. For example, when a learner looks at a certain area and makes a gesture, the system will adjust the interaction mode according to his preferences and habits. In order to improve the fluency and accuracy of the operation, a deep reinforcement learning algorithm was introduced, and the model was optimized at a learning rate of 1, which improved the overall interaction effect by 20%. By integrating gesture recognition and eye tracking data through a fusion learning algorithm and processing time series data using a long short-term memory network, the data processing accuracy was increased to 98%. In addition, a user interface was developed that allows learners to adjust VR settings according to their needs. For example, the sensitivity of the interaction mode is divided into 10 levels, and learners can choose the appropriate level according to their comfort.

[0126] like Figure 7 FIG. 1 is another flow chart of the steps of the VR interactive learning imaging method provided by an embodiment of the present invention, referring to FIG. Figure 7 As an optional embodiment, the learning imaging method further includes the following steps:

[0127] S401, determining the target learner's emotional engagement and cognitive load level according to the emotional physiological feature vector;

[0128] S402, determining a plurality of knowledge points according to the target learning content, and calculating the knowledge point relevance between the knowledge points;

[0129] S403, acquiring the target learner's interactive behavior data, and determining the target learner's interactive behavior pattern according to the interactive behavior data;

[0130] S404. Optimize the learning path of each knowledge point through a reinforcement learning algorithm according to the emotional engagement, cognitive load level, knowledge point relevance, and interactive behavior pattern.

[0131] The embodiment of the present invention comprehensively analyzes multi-dimensional data such as emotional investment, knowledge point relevance, cognitive load and interactive operations in the learning process, and continuously optimizes the overall learning path through a reinforcement learning algorithm, thereby maximizing the learner's overall learning experience and effect while ensuring the integrity and logic of the knowledge system.

[0132] Specifically, the multi-dimensional data of emotional engagement, knowledge point relevance, cognitive load level and interactive behavior characteristics are input into the reinforcement learning model, and trial and error and feedback are continuously conducted to determine the optimal learning path planning strategy; when generating the learning path, the integrity and logic of the knowledge system are used as constraints to ensure the systematicness and coherence of the learning content and avoid knowledge gaps or logical confusion. The learning experience and learning effect before and after optimization are compared through A / B testing, and the evaluation results are fed back to the reinforcement learning model to guide the algorithm to continuously iterate and optimize, and continuously improve the overall learning quality.

[0133] Exemplarily, in the above-mentioned learning process, S4 collects learners' heart rate data by wearing smart bracelets, and uses emotion recognition algorithms to analyze learners' facial expressions to comprehensively judge learners' emotional engagement. For example, when the learner's average heart rate is 10% higher than usual, and the facial expression shows a focused state, it can be judged that their emotional engagement is high. At the same time, a knowledge point association map is constructed based on the learner's knowledge point access sequence in S5, and the association between different knowledge points is calculated using the graph neural network algorithm. After learners visit the assistance policy, the frequency of visiting industrial assistance is 80%, which can be inferred that the association between the two knowledge points is high. In addition, the learner's gaze point and gaze duration data are obtained through eye tracking technology in S6, combined with the Flesch readability index of the learning material, and the learner's cognitive load level is dynamically evaluated using cognitive load theory. When the learner's average gaze duration on the material with a complexity of 75 exceeds 5 seconds, it is considered that its cognitive load is high. Furthermore, key interaction features such as the number of clicks, sliding speed, and search keywords are extracted from the user log of the VR platform in S7, and the learner's interactive behavior pattern is characterized using a sequence pattern mining algorithm. The above multi-dimensional data is input into the reinforcement learning model, and the optimal learning path planning is obtained through continuous trial and error and updating strategies through the Q-Learning algorithm. When generating the learning path, the topological sorting of knowledge points is used as a constraint to ensure the systematicness and coherence of the learning content. Finally, the A / B test is used to compare the learning completion rate and satisfaction before and after optimization, and the results are fed back to the reinforcement learning model for parameter tuning to continuously improve the learning quality.

[0134] The above is a description of the method steps of the embodiment of the present invention. It can be recognized that the embodiment of the present invention collects multimodal data such as facial expressions, eye movements, heart rate, and EEG of learners during VR learning, and combines personal attribute information to establish a personalized emotional physiological baseline model; in the learning process, the learner's emotional physiological characteristics are tracked in real time, and compared with the baseline model to determine the learning interest and degree of involvement; when the learning interest is detected to be reduced, relevant and interesting knowledge points are recommended from the knowledge graph, and the VR scene presentation method is dynamically adjusted according to the cognitive load; at the same time, the interaction method is optimized through gesture recognition and eye tracking to improve the operation fluency; finally, the multidimensional data is comprehensively analyzed, and the reinforcement learning algorithm is used to continuously optimize the learning path, while ensuring the integrity of the knowledge system. Maximize the learning experience and effect. The embodiment of the present invention realizes intelligent personalized imaging optimization of the VR interactive learning process, analyzes the learner's interest and involvement in the learning content based on the emotional changes and physiological data in the learning process, and adjusts the subsequent learning content and presentation method, style and learning path in real time, which improves the learner's learning efficiency and learning experience.

[0135] like Figure 8 FIG. 1 is a schematic diagram of a structure of a VR interactive learning imaging device provided by an embodiment of the present invention, referring to FIG. Figure 8 , an embodiment of the present invention provides a learning imaging device based on VR interaction, comprising:

[0136] The emotional physiological feature extraction module is used to obtain the multimodal emotional physiological data of the target learner in the current VR scene, and extract the emotional physiological feature vector based on the multimodal emotional physiological data;

[0137] The emotional physiological feature comparison module is used to compare the emotional physiological feature vector with the pre-built personalized emotional physiological baseline model of the target learner to determine whether the learning state of the target learner is normal;

[0138] The target VR scene generation module is used to obtain target learning content that is highly relevant to the current learning content and of interest to the target learner when the target learner's learning state is abnormal, and to generate a target VR scene based on the target learning content.

[0139] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0140] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned VR-interactive learning imaging method. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0141] like Fig. 9 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Fig. 9 , an embodiment of the present invention provides an electronic device, including:

[0142] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0143] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the VR interactive learning imaging method of the embodiment of the present invention;

[0144] Input / output interface 903, used to implement information input and output;

[0145] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0146] A bus 905 that transmits information between the various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0147] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0148] like Fig.10FIG. 1 is a schematic diagram of a storage medium according to an embodiment of the present invention. Fig.10 The embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1001, and the one or more programs 1001 can be executed by one or more processors to implement the above-mentioned VR interaction-based learning imaging method.

[0149] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0151] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0152] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0153] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0154] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0155] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0156] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0157] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0158] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0159] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A learning imaging method based on VR interaction, characterized in that: The following steps are involved: Acquire multimodal emotional physiological data of a target learner in a current VR scene, and extract an emotional physiological feature vector according to the multimodal emotional physiological data; Comparing the emotional physiological feature vector with a pre-constructed personalized emotional physiological baseline model of the target learner to determine whether the learning state of the target learner is normal; When the learning state of the target learner is abnormal, target learning content that is highly relevant to the current learning content and that the target learner is interested in is obtained, and a target VR scene is generated according to the target learning content.

2. The method for learning imaging based on VR interaction according to claim 1, characterized in that: The multimodal emotional physiological data includes facial expression data, eye movement data, heart rate data and brain wave data. The emotional physiological feature vector is extracted based on the multimodal emotional physiological data, which specifically includes: Performing emotion recognition on the target learner according to the facial expression data to obtain emotional state characteristics; Determining sight line change data of the target learner according to the eye movement data, and determining a gaze point heat map according to the sight line change data, and then extracting attention distribution features according to the gaze point heat map; Performing heart rate variability analysis on the target learner according to the heart rate data to obtain stress level characteristics; Performing frequency domain analysis on the brain wave data to obtain energy distribution information of each frequency band, and determining cognitive load characteristics according to the energy distribution information; The emotional state feature, the attention distribution feature, the stress level feature, and the cognitive load feature are normalized and fused to obtain the emotional physiological feature vector.

3. The method for learning imaging based on VR interaction according to claim 1, characterized in that: The personalized emotional physiological baseline model is constructed by the following steps: Acquire personal attribute information of a plurality of target learners, and acquire a target emotional physiological feature vector of each target learner in a normal learning state; Performing cluster analysis on the target learners according to the personal attribute information and the target emotional physiological feature vector to obtain a plurality of user groups; Performing numerical statistics on the target emotional physiological characteristic vectors of the target learners of the same user group in each dimension, determining the emotional physiological characteristic baseline value of the corresponding dimension according to the mean value of each dimension, and determining the emotional physiological characteristic fluctuation range of the corresponding dimension according to the standard deviation of each dimension; Determining an emotional physiological baseline model of the user group according to the emotional physiological characteristic baseline value and the emotional physiological characteristic fluctuation range; The emotional physiological baseline model is adjusted according to the personal attribute information of each target learner in the user group to obtain a personalized emotional physiological baseline model for each target learner.

4. The method for learning imaging based on VR interaction according to claim 1, characterized in that: The step of comparing the emotional physiological feature vector with the pre-built personalized emotional physiological baseline model of the target learner to determine whether the learning state of the target learner is normal specifically includes: Determining target emotional physiological characteristic values ​​of the target learner in each dimension according to the emotional physiological characteristic vector; Determine the target emotional physiological characteristic baseline value and target emotional physiological characteristic fluctuation range of the target learner in each dimension according to the personalized emotional physiological baseline model; Determine the emotional physiological characteristic deviation value of the corresponding dimension according to the difference between the target emotional physiological characteristic value and the target emotional physiological characteristic baseline value of the corresponding dimension, and judge whether the emotional physiological characteristic deviation value is within the emotional physiological characteristic fluctuation range of the corresponding dimension; When the deviation values ​​of the emotional physiological characteristics of each dimension are all within the corresponding fluctuation range of the emotional physiological characteristics, it is determined that the learning state of the target learner is normal; When the deviation value of the emotion-physiological characteristic of at least one dimension is not within the corresponding fluctuation range of the emotion-physiological characteristic, it is determined that the learning state of the target learner is abnormal.

5. The method for learning imaging based on VR interaction according to claim 1, characterized in that: The obtaining of target learning content that is highly relevant to the current learning content and that the target learner is interested in, and generating a target VR scene according to the target learning content, specifically includes: Determine, according to a preset learning content knowledge graph, candidate learning content whose relevance to the current learning content is greater than or equal to a preset first threshold; Determining the target learning content that the target learner is interested in from the candidate learning content by using a collaborative filtering algorithm; Determining the cognitive load level of the target learner according to the emotional physiological feature vector, and determining the cognitive load state of the target learner according to the cognitive load level and a preset cognitive load threshold; The learning task setting, plot development setting and interaction mode setting are performed according to the target learning content, and the information presentation mode and information presentation style are adjusted according to the cognitive load state to obtain the target VR scene.

6. The method for learning imaging based on VR interaction according to claim 1, characterized in that: The learning imaging method further comprises the following steps: Acquiring hand movement data and eye movement data of the target learner; Performing gesture recognition on the target learner according to the hand motion data to obtain operation intention information; Determining the focus information of the target learner according to the eye movement data; The VR interaction mode is optimized in real time according to the operation intention information, the focus information and the personal preference / habit information of the target learner.

7. A VR interactive learning imaging method according to any one of claims 1 to 6, characterized in that: The learning imaging method further comprises the following steps: Determining the target learner's emotional engagement and cognitive load level according to the emotional physiological feature vector; Determine a plurality of knowledge points according to the target learning content, and calculate the knowledge point association between the knowledge points; Acquiring the interactive behavior data of the target learner, and determining the interactive behavior pattern of the target learner according to the interactive behavior data; According to the emotional engagement, the cognitive load level, the knowledge point relevance and the interactive behavior pattern, the learning path of each of the knowledge points is optimized through a reinforcement learning algorithm.

8. A learning imaging device based on VR interaction, characterized in that: include: An emotion physiological feature extraction module is used to obtain multimodal emotion physiological data of the target learner in the current VR scene, and extract an emotion physiological feature vector based on the multimodal emotion physiological data; An emotion-physiology feature comparison module, used to compare the emotion-physiology feature vector with a pre-built personalized emotion-physiology baseline model of the target learner to determine whether the learning state of the target learner is normal; The target VR scene generation module is used to obtain target learning content that is highly relevant to the current learning content and that the target learner is interested in when the learning state of the target learner is abnormal, and generate a target VR scene based on the target learning content.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the VR interaction-based learning imaging method as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the VR interaction-based learning imaging method as described in any one of claims 1 to 7.

Images