Online self-study system and method integrating user behavior analysis and immersive interaction

Abstract

The invention relates to an online self-study system and method fusing user behavior analysis and immersive interaction, equipment and a medium. The system comprises a multi-source data acquisition module for acquiring behavior data and physiological data of a user in real time; the user behavior analysis module is used for analyzing the behavior data and the physiological data by utilizing a pre-trained user learning state model and generating learning state information containing fatigue characteristics and cognitive load characteristics; the scene parameter generation module is used for generating immersive learning scene parameters according to a preset environment generation rule on the basis of the learning state information; and the immersive presentation module constructs and presents an immersive learning scene dynamically adaptive to the user state according to the parameters. The technical problem that a traditional online self-study environment lacks personalized adaptation ability is solved, and immersive interaction experience of automatically optimizing the learning environment based on the real-time physiological and behavior state of the user is achieved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the cross field of online education technology and human-computer interaction technology, and particularly relates to an online self-study system and method fusing user behavior analysis and immersive interaction. BACKGROUND

[0002] With the development of online education technology, an online self-study system, a new type of learning carrier, has emerged. It provides a remote learning environment for users through the network, breaking the time and space limitations of traditional offline self-study. This further leads to the current mainstream online self-study method. In traditional technology, such systems can usually only provide basic learning resource display (such as document browsing and video playing) and simple interaction functions (such as answering questions and note recording), and lack real-time perception and dynamic response to the learning state of users. The current online self-study method has significant shortcomings: it lacks multi-source collection and fusion analysis of user behavior data and physiological data, cannot accurately identify the fatigue characteristics and cognitive load state of users, and thus cannot adjust the learning environment accordingly; the learning scene is usually designed to be fixed, and cannot dynamically generate adaptive environmental parameters (such as light intensity, sound type, and space layout) according to the real-time learning state of users, making it difficult to create an immersive learning experience; the self-study room lacks learning resource pushing or intelligent matching of user fatigue level and cognitive load index during the pushing process, and the accuracy of resource recommendation is insufficient, making it difficult to meet the personalized learning needs. This leads to low learning efficiency and increased fatigue of users during use, limiting the actual application effect of the online self-study system. SUMMARY

[0003] Therefore, it is necessary to provide an online self-study system and method fusing user behavior analysis and immersive interaction to solve the above problems.

[0004] In a first aspect, the present application provides an online self-study system fusing user behavior analysis and immersive interaction, comprising:

[0005] a multi-source data collection module for acquiring behavior data and physiological data of a user;

[0006] a user behavior analysis module for analyzing the behavior data and physiological data of the user using a pre-trained user learning state model to generate learning state information; the learning state information includes fatigue characteristics and cognitive load characteristics;

[0007] a scene parameter generation module for generating immersive learning scene parameters based on the learning state information using a preset environment generation rule;

[0008] an immersive presentation module for constructing an immersive learning scene based on the immersive learning scene parameters.

[0009] In one embodiment, the user behavior analysis module is further configured to:

[0010] In response to acquiring new behavioral data and physiological data of the user, incrementally updating the training data set using the new data;

[0011] Incrementally train the pre-trained user learning state model using the updated training dataset;

[0012] Utilize the updated user learning status model to analyze the user's behavior data and physiological data collected in real time to generate the user's current learning status information;

[0013] The learning effect is analyzed based on the current learning status information to generate learning effect information; the learning effect information is used to generate visual learning effect feedback in the user interface for users to evaluate their self-study status.

[0014] In one embodiment, the system further includes a resource push module for:

[0015] In response to a user's request for learning resources, extracting fatigue-related features and cognitive load-related features based on learning state information;

[0016] Generate a learning feature vector based on the extracted fatigue-related features and cognitive load-related features, where the learning feature vector includes a fatigue level quantification value and a cognitive load index;

[0017] Generate resource recommendation vectors based on the matching relationship between the learned feature vectors and the preset resource adaptation rules;

[0018] According to the resource recommendation vector, matching learning resources are selected from the learning resource library and pushed.

[0019] In one embodiment, the scene parameter generation module is further configured to:

[0020] Generate an ambient light intensity adjustment parameter based on the fatigue level quantization value in the learning feature vector;

[0021] Generate a background sound resource identifier based on the cognitive load index in the learning feature vector;

[0022] Combining the fatigue level quantification value with the cognitive load index, the spatial layout parameters of the virtual learning scene elements are generated;

[0023] The ambient light intensity adjustment parameters, background sound resource identifiers, and space layout parameters are combined into immersive learning scene parameters.

[0024] In one embodiment, the immersive presentation module is further configured to:

[0025] Analyze the spatial layout parameters in the immersive learning scene parameters and construct the spatial topology structure of the three-dimensional virtual learning space;

[0026] Load the ambient light intensity adjustment parameters in the immersive learning scene parameters to generate a dynamic ambient lighting model;

[0027] Generate an audio playback sequence with timing control logic based on the background sound effect resource identifier in the immersive learning scene parameters;

[0028] Synchronously drive the spatial topology, dynamic environmental lighting model, and audio playback sequence to render and generate an immersive learning scene.

[0029] In one embodiment, the immersive presentation module is further configured to synchronously drive the spatial topology, dynamic environment lighting model, and audio playback sequence to render and generate an immersive learning scene using the following formula:

[0030]

[0031] Among them, Scene(t) is the immersive learning scene dynamically generated over time t, Φ is the fusion operator, is the spatial topological structure generating function, α i (t) is the time attenuation coefficient, P s is the spatial layout parameter vector, V s is the base space mesh vertex set, T i (·) is the topological transformation function, Λ(P l , t) = k·e -βt ·P l is the ambient light adjustment function, β is the attenuation factor positively correlated with the user's fatigue level, P l is the ambient light intensity adjustment parameter, k is the reference light intensity coefficient, is the audio sequence scheduling function, δ(·) is the time trigger function, τ j The timestamp of the sound effect trigger, P a Identifies a collection of sound effect resources.

[0032] In one embodiment, the multi-source data acquisition module is further configured to:

[0033] Acquire the user's visual behavior raw data through the image acquisition unit;

[0034] Obtaining the user's physiological signal raw data through a wearable sensor unit;

[0035] Acquiring original data of user interaction operations through the user interaction unit;

[0036] Integrate visual behavior raw data, physiological signal raw data, and interactive operation raw data to form an initial multi-source data set;

[0037] Data cleaning and feature extraction operations are performed on the initial multi-source data set to obtain feature data related to user behavior as user behavior data, and feature data related to the user's physiological state as physiological data.

[0038] Secondly, this application also provides an online self-study method that integrates user behavior analysis and immersive interaction, including:

[0039] Obtain user behavioral and physiological data;

[0040] Utilize the pre-trained user learning state model to analyze the user's behavioral and physiological data to generate learning state information; the learning state information includes fatigue characteristics and cognitive load characteristics;

[0041] Utilize preset environment generation rules to generate immersive learning scenario parameters based on learning status information;

[0042] Build an immersive learning scenario based on immersive learning scenario parameters.

[0043] In a third aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above-mentioned online self-study system function that integrates user behavior analysis and immersive interaction.

[0044] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of implementing the above-mentioned online self-study system function that integrates user behavior analysis and immersive interaction are implemented.

[0045] The above-mentioned online self-study system and method, computer equipment and storage medium that integrate user behavior analysis and immersive interaction obtain the user's behavioral data and physiological data in real time through the multi-source data acquisition module, thereby realizing multi-dimensional perception of the user's learning status; the user behavior analysis module utilizes the pre-trained user learning status model to analyze the user's behavioral data and physiological data and generates learning status information including fatigue characteristics and cognitive load characteristics, accurately identifying the user's real-time learning status; the scene parameter generation module dynamically generates immersive learning scene parameters based on the learning status information and preset environment generation rules, so that the learning environment can be adaptively adjusted; the immersive presentation module constructs and presents dynamically adapted immersive learning scenes based on these parameters, solving the problem of low learning efficiency in traditional online self-study environments due to the lack of personalized adaptation capabilities, and realizing an immersive interactive experience that automatically optimizes the environment based on the user's physiological and behavioral status, significantly improving learning concentration and resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a structural diagram of the online self-study system that integrates user behavior analysis and immersive interaction of the present invention;

[0048] Figure 2 This is a flow chart of the online self-study method that integrates user behavior analysis and immersive interaction of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0050] In one embodiment, Figure 1 As shown, an online self-study system that integrates user behavior analysis and immersive interaction is provided. This embodiment takes the application of the system to a terminal as an example. It can be understood that the system can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal hardware includes an image acquisition unit, a wearable sensor unit, and a user interaction unit, etc., which collect user behavior and physiological data, and generate learning status information after analysis by a processor using a pre-trained model, and then generate immersive scene parameters and construct a scene; when applied to a structure containing a server, the terminal can transmit data to the server for processing, and the server can also push matching resources from the resource library according to the learning feature vector to meet the user's demand for a personalized immersive learning environment during online self-study.

[0051] In this embodiment, the system includes:

[0052] The multi-source data acquisition module 101 is used to obtain the user's behavioral data and physiological data.

[0053] The visual behavior raw data (e.g., eye movement trajectory, head posture) of the user can be captured by an image acquisition unit (e.g., a terminal built-in camera), the physiological signal raw data (e.g., heart rate, brain wave) can be collected by a wearable sensing unit (e.g., a smart bracelet or an electroencephalogram cap), and the interactive operation raw data (e.g., click, scroll event) can be recorded by a user interaction unit (e.g., a keyboard or a touch screen). The initial multi-source data set is formed by integrating the visual behavior raw data, the physiological signal raw data and the interactive operation raw data, and after data cleaning operation (e.g., outlier filtering, noise removal) and feature extraction operation (e.g., time series analysis to extract behavior features, frequency domain transformation to extract physiological features), the structured and standardized user behavior data (for subsequent behavior analysis) and the user physiological data (for physiological state evaluation) are generated as the input of the subsequent modules, realizing the fusion perception of multi-source heterogeneous data.

[0054] The user behavior analysis module 102 is configured to analyze the behavior data and the physiological data of the user by using a pre-trained user learning state model, and generate learning state information. The learning state information includes fatigue features and cognitive load features.

[0055] The user behavior data and the physiological data can be input into the pre-trained user learning state model (a data processing model pre-trained based on a machine learning algorithm, such as a neural network or an ensemble learning model) for feature (fatigue feature representing the degree of decline of user attention or physiological function, such as quantified by indicators such as reduced eye movement frequency, heart rate variability abnormality; cognitive load feature representing the state of user brain resource occupation, such as behavior data such as operation delay, error rate increase, which can be comprehensively determined in combination with brain wave frequency band energy distribution) fusion analysis. The model-embedded time sequence feature extraction layer can capture the dynamic changes of the behavior pattern, the physiological signal analysis layer can calculate the fatigue-related indicators (such as fatigue index based on pupil diameter fluctuation), and the cognitive state classification layer can output the cognitive load grade. The learning state information including the fatigue quantitative value and the cognitive load index is generated. The module can also perform a dynamic optimization mechanism. When new behavior data or physiological data is input, the new data is added to the historical training data set to trigger incremental update, and the user learning state model is dynamically optimized by an incremental training algorithm (such as online gradient descent). The updated model is used to re-analyze the real-time data to generate more accurate current learning state information. Based on the learning effect evaluation result of the information, visual feedback is generated for the user to calibrate the learning state, realizing the adaptive co-evolution of model analysis and user cognitive state.

[0056] The scene parameter generation module 103 is configured to generate immersive learning scene parameters based on the learning state information by using preset environment generation rules.

[0057] Among them, ambient light intensity adjustment parameters can be generated based on the fatigue characteristics in the learning status information (such as calculating the light intensity attenuation coefficient according to the preset linear mapping relationship based on the fatigue level quantification value, and the high fatigue level corresponds to the low color temperature and weak lighting parameters to relieve visual stress); background sound resource identifiers can be generated based on the cognitive load characteristics (for example, a high cognitive load index triggers a white noise identifier to improve concentration, and a low index matches the natural sound identifier to relax the nerves); and then, the spatial layout parameters of the virtual learning scene elements are generated by combining the multidimensional correlation analysis of the fatigue level quantification value and the cognitive load index (such as when the fatigue value is higher than the threshold and the cognitive load is low, a parameter vector for expanding the virtual desktop space is generated, and the visual search burden is reduced by adjusting the distribution density of the interface elements); the above-mentioned ambient light intensity adjustment parameters, background sound resource identifiers and spatial layout parameters are combined into a structured immersive learning scene parameter set.

[0058] The immersive presentation module 104 is configured to construct an immersive learning scene based on immersive learning scene parameters.

[0059] Among them, the spatial layout parameter vector in the scene parameters can be parsed to construct the basic grid structure of the three-dimensional virtual learning space, and the distribution of interface elements can be dynamically adjusted through the topological transformation function (such as optimizing the desktop layout density according to the fatigue level and cognitive load index to reduce the visual search burden); the ambient light intensity adjustment parameters can be loaded synchronously to generate a dynamic ambient lighting model (such as automatically reducing the light intensity or adjusting the color temperature based on fatigue characteristics to relieve visual fatigue); at the same time, according to the background sound resource identifier, an audio playback sequence with timing control logic is generated (such as triggering white noise or natural sound effects through precise timestamps to match the rhythm of cognitive load changes); the spatial topology structure, dynamic lighting model and audio sequence are synchronously driven by the fusion operator to realize multi-element spatiotemporal alignment rendering, output a continuously evolving immersive learning scene, realize a closed-loop mapping from parameters to the environment, and solve the problem that the traditional online self-study environment lacks real-time response capabilities.

[0060] The above-mentioned online self-study system that integrates user behavior analysis and immersive interaction, the multi-source data acquisition module obtains the user's behavioral data and physiological data in real time, the user behavior analysis module uses a pre-trained user learning state model to analyze the behavioral data and physiological data, and generates learning state information including fatigue characteristics and cognitive load characteristics, the scene parameter generation module dynamically generates immersive learning scene parameters based on the learning state information according to preset environment generation rules, and the immersive presentation module parses the parameters, by constructing the spatial topology structure of the three-dimensional virtual learning space, loading the dynamic environment lighting model and generating a time-controlled audio playback sequence, synchronously driving the rendering to generate a continuously evolving immersive learning scene; solving the problem that traditional online self-study systems cannot perceive the user status in real time due to the lack of personalized adaptation capabilities, resulting in fixed and inefficient learning scenes, and realizing automatic optimization of the learning environment according to the user's physiological and behavioral state, relieving fatigue, and improving concentration and learning efficiency.

[0061] In one embodiment, the user behavior analysis module 102 is further configured to:

[0062] In response to acquiring new behavioral data and physiological data of the user, incrementally updating the training data set using the new data;

[0063] Incrementally train the pre-trained user learning state model using the updated training dataset;

[0064] Utilize the updated user learning status model to analyze the user's behavior data and physiological data collected in real time to generate the user's current learning status information;

[0065] The learning effect is analyzed based on the current learning status information to generate learning effect information; the learning effect information is used to generate visual learning effect feedback in the user interface for users to evaluate their self-study status.

[0066] Specifically, in response to obtaining new behavioral data and physiological data of the user, the new data can be integrated with historical training data to generate a training data set for incremental updates; the pre-trained user learning state model can be incrementally trained through an online learning mechanism, and the model can be iteratively optimized by gradually incorporating new data; the user learning state model after parameter update is used to perform feature fusion analysis on the real-time collected user behavioral data and physiological data, and generate current learning state information including real-time fatigue quantification indicators and cognitive load index; based on the current learning state information, combined with preset learning effect evaluation rules, the dimensions of learning efficiency, attention concentration, etc. are quantitatively analyzed to generate learning effect information (an evaluation result generated by comprehensive learning state characteristics and indicators such as task completion), which can be converted into visual forms such as charts and trend curves for feedback in the user interface, so that users can evaluate their own self-study status and adjust their learning strategies.

[0067] In one embodiment, the system further includes a resource push module for:

[0068] In response to a user's request for learning resources, extracting fatigue-related features and cognitive load-related features based on learning state information;

[0069] Generate a learning feature vector based on the extracted fatigue-related features and cognitive load-related features, where the learning feature vector includes a fatigue level quantification value and a cognitive load index;

[0070] Generate resource recommendation vectors based on the matching relationship between the learned feature vectors and the preset resource adaptation rules;

[0071] According to the resource recommendation vector, matching learning resources are selected from the learning resource library and pushed.

[0072] Exemplarily, upon receiving a learning resource request sent by a user, characteristic parameters related to fatigue status and characteristic parameters representing the degree of cognitive load are extracted from the learning status information generated by the user behavior analysis module; based on the extracted characteristic parameters, a learning feature vector is generated containing a fatigue level quantification value (obtained by mapping the fatigue characteristic parameters to a preset level interval) and a cognitive load index (generated by normalizing the cognitive load characteristic parameters); the learning feature vector is matched with a preset resource adaptation rule (a set of pre-set mapping rules between feature vectors and resource types), and a resource recommendation vector (a structured data vector for indicating the priority and category of recommended resources) containing a target resource type identifier and a priority weight is generated based on the matching result; according to the resource recommendation vector, the learning resource with the highest matching degree is retrieved and selected from a learning resource library (a structured database storing various types of learning resources), and pushed to the user interaction interface in a visual manner.

[0073] In one embodiment, the scene parameter generation module 103 is further configured to:

[0074] Generate an ambient light intensity adjustment parameter based on the fatigue level quantization value in the learning feature vector;

[0075] Generate a background sound resource identifier based on the cognitive load index in the learning feature vector;

[0076] Combining the fatigue level quantification value with the cognitive load index, the spatial layout parameters of the virtual learning scene elements are generated;

[0077] The ambient light intensity adjustment parameters, background sound resource identifiers, and space layout parameters are combined into immersive learning scene parameters.

[0078] Specifically, based on the fatigue level quantization value in the learning feature vector (the quantization result of mapping the fatigue feature to the preset level range), the ambient light intensity adjustment parameter (the adjustment parameter of high fatigue level corresponding to low color temperature and weak light) can be generated through the preset light intensity mapping rule (such as the linear correspondence between fatigue level and color temperature and brightness); based on the cognitive load index in the learning feature vector (the normalized value that represents the degree of user brain resource utilization), the background sound resource identifier is generated according to the preset sound effect matching rule (such as high cognitive load triggering white noise resource identifier, low cognitive load matching natural sound identifier); combined with the fatigue level quantization Based on a comprehensive analysis of the fatigue value and the cognitive load index, the spatial layout parameters of the virtual learning scene elements (used to define the position, size and interaction hierarchy of elements in the virtual scene) are generated through a spatial layout generation algorithm (adjusting the interface element spacing according to the fatigue value and optimizing the visual focus distribution based on the cognitive load index. For example, when the fatigue level quantification value is >70 and the cognitive load characteristic is <30, the desktop element spacing is expanded to 150%). The ambient light intensity adjustment parameters, background sound resource identifiers and spatial layout parameters are combined into an immersive learning scene parameter set (a structured parameter set that drives the construction of an immersive learning scene) according to a preset data structure.

[0079] In one embodiment, the immersive presentation module 104 is further configured to:

[0080] Analyze the spatial layout parameters in the immersive learning scene parameters and construct the spatial topology structure of the three-dimensional virtual learning space;

[0081] Load the ambient light intensity adjustment parameters in the immersive learning scene parameters to generate a dynamic ambient lighting model;

[0082] Generate an audio playback sequence with timing control logic based on the background sound effect resource identifier in the immersive learning scene parameters;

[0083] Synchronously drive the spatial topology, dynamic environmental lighting model, and audio playback sequence to render and generate an immersive learning scene.

[0084] Exemplarily, the spatial layout parameters in the immersive learning scene parameters are parsed, and the parameters can be mapped into topological transformation instructions of the basic space grid vertex set through the three-dimensional modeling engine to construct a dynamically adjusted three-dimensional virtual learning space topology structure (a three-dimensional geometric layout model of the virtual learning space); the ambient light intensity adjustment parameters in the scene parameters are loaded, and the lighting rendering engine is used to generate an ambient lighting model that changes dynamically over time (such as using the attenuation factor associated with the fatigue level to calculate the light intensity attenuation curve, which is used to simulate the real-time changes of real ambient light); according to the background sound effect resource identifier in the scene parameters, an audio playback sequence containing a time trigger stamp is generated through the audio scheduling algorithm (including timing control data of the sound effect trigger timestamp, and the sound effect switching timing can be determined based on the cognitive load index); the spatial topology structure, dynamic ambient lighting model and audio playback sequence can be synchronously driven by the fusion operator to achieve spatiotemporal alignment rendering of multimodal scene elements, generate an immersive learning scene that evolves dynamically with the user state, and achieve collaborative rendering of multimodal scene elements.

[0085] In one embodiment, the immersive presentation module 104 is further configured to synchronously drive the spatial topology, the dynamic environment lighting model, and the audio playback sequence to render and generate an immersive learning scene using the following formula:

[0086]

[0087] Among them, Scene(t) is the immersive learning scene dynamically generated over time t, Φ is the fusion operator, is the spatial topological structure generating function, α i (t) is the time attenuation coefficient, P s is the spatial layout parameter vector, V s is the base space mesh vertex set, T i (·) is the topological transformation function, Λ(P l , t) = k·e -βt ·P l is the ambient light adjustment function, β is the attenuation factor positively correlated with the user's fatigue level, P l is the ambient light intensity adjustment parameter, k is the reference light intensity coefficient, is the audio sequence scheduling function, δ(·) is the time trigger function, τ j The timestamp of the sound effect trigger, P a Identifies a collection of sound effect resources.

[0088] Specifically, the fusion operator Φ (a preset set of multimodal data collaborative rendering algorithms) realizes the collaborative rendering of the outputs of each module through the spatiotemporal alignment algorithm: the spatial topology structure generation function The time attenuation coefficient αi (t) For the basic space grid vertex set V s Perform topological transformation function T i (·) to achieve dynamic adjustment of the virtual learning space layout, and the ambient light adjustment function Λ(P l , t) = k·e -βt P1 adjusts the ambient light intensity parameter P based on the attenuation factor β (e.g., β = 0.05 × fatigue level) that is positively correlated with the user's fatigue level. l Perform exponential decay modulation and generate a dynamic lighting model based on the reference light intensity coefficient k to dynamically adjust the light intensity; audio sequence scheduling function The time trigger function δ(·) is used to trigger the sound effect timestamp τ j For the sound effect resource identification set P a The resources in the virtual learning scene are scheduled in time sequence to generate an audio playback sequence; the spatial, lighting, and audio modal elements are collaboratively rendered according to time t through the fusion operator Φ to form an immersive learning scene that adapts to the user state. This solves the asynchronous rendering problem of elements such as space, lighting, and sound effects in virtual learning scenes and realizes dynamic scene adaptation based on user state.

[0089] In one embodiment, the multi-source data acquisition module 101 is further configured to:

[0090] Acquire the user's visual behavior raw data through the image acquisition unit;

[0091] Obtaining the user's physiological signal raw data through a wearable sensor unit;

[0092] Acquiring original data of user interaction operations through the user interaction unit;

[0093] Integrate visual behavior raw data, physiological signal raw data, and interactive operation raw data to form an initial multi-source data set;

[0094] Data cleaning and feature extraction operations are performed on the initial multi-source data set to obtain feature data related to user behavior as user behavior data, and feature data related to the user's physiological state as physiological data.

[0095] Exemplarily, the original data of the user's visual behavior, such as eye movement trajectory and head posture, (unprocessed raw data set) can be obtained through an image acquisition unit (such as a built-in camera in a terminal, a hardware device that captures the user's visual behavior); the original data of physiological signals such as heart rate and brain waves can be collected through a wearable sensing unit (a wearable sensor that obtains physiological signals, such as a smart bracelet); the original data of interactive operations such as clicks and scrolling can be recorded through a user interaction unit (a hardware module that receives user operation input, such as a mouse, keyboard, and touch screen); the above three types of raw data are integrated to form an initial multi-source data set containing multi-source heterogeneous information, and data cleaning operations such as outlier filtering and noise removal are performed to pre-process the raw data to improve data quality; effective features are extracted from the raw data through feature extraction algorithms such as time series analysis and frequency domain transformation, and behavioral feature data representing user operating habits (as user behavioral data) and physiological feature data reflecting physiological state (as physiological data) are separated.

[0096] The above-mentioned online self-study system that integrates user behavior analysis and immersive interaction obtains raw data such as user visual behavior, physiological signals and interactive operations in real time through a multi-source data acquisition module, and forms behavioral data and physiological data after cleaning and extraction. The user behavior analysis module uses a pre-trained learning state model to analyze the data, generate learning state information including fatigue characteristics and cognitive load characteristics, and incrementally trains the model based on the newly added data to optimize the analysis accuracy. At the same time, it generates visual learning effect feedback based on the learning state information; the scene parameter generation module generates ambient light intensity adjustment parameters, background sound resource identifiers and virtual scene element spatial layout parameters based on the fatigue level quantification value and cognitive load index in the learning state information, and combines them into immersive learning scene parameters; after parsing the parameters, the immersive presentation module constructs a three-dimensional virtual learning space topology structure, generates a dynamic ambient lighting model and a time-controlled audio playback sequence, and synchronously drives multimodal element rendering through specific formulas to generate an adaptive immersive learning scene; when responding to a learning resource request, the resource push module extracts fatigue and cognitive load characteristics to generate a learning feature vector, generates a resource recommendation vector based on preset rule matching, and pushes the adapted resource. Through the fusion analysis of multi-source data and dynamic optimization of models, accurate identification of user learning status is achieved, and adaptive scene parameters and resource recommendations are dynamically generated based on the learning status. This solves the problems of traditional systems such as lack of personalized adaptation, fixed scenes, and insufficient accuracy of resource push, creates an immersive interactive experience, effectively improves learning concentration and resource utilization efficiency, and alleviates user learning fatigue.

[0097] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0098] Based on the same inventive concept, the embodiment of the present application also provides an online self-study method that integrates user behavior analysis and immersive interaction for realizing the above-mentioned online self-study system that integrates user behavior analysis and immersive interaction. The implementation solution for solving the problem provided by this method is similar to the implementation solution recorded in the above-mentioned system. Therefore, the specific limitations in one or more embodiments of the online self-study method that integrates user behavior analysis and immersive interaction provided below can be found in the above limitations on the online self-study system that integrates user behavior analysis and immersive interaction, and will not be repeated here.

[0099] In an exemplary embodiment, Figure 2 As shown in the figure, an online self-study method that integrates user behavior analysis and immersive interaction is provided, including:

[0100] S01, obtaining user's behavioral data and physiological data;

[0101] S02, using a pre-trained user learning state model to analyze the user's behavioral data and physiological data to generate learning state information; the learning state information includes fatigue characteristics and cognitive load characteristics;

[0102] S03, using a preset environment generation rule to generate immersive learning scene parameters based on the learning state information;

[0103] S04: Build an immersive learning scene based on the immersive learning scene parameters.

[0104] In one embodiment, the method further comprises:

[0105] S11, in response to acquiring newly added behavioral data and physiological data of the user, incrementally updating the training data set using the newly added data;

[0106] S12, incrementally training the pre-trained user learning state model using the updated training dataset;

[0107] S13, using the updated user learning state model to analyze the user behavior data and physiological data collected in real time to generate the user's current learning state information;

[0108] S14, analyzing the learning effect based on the current learning status information and generating learning effect information; the learning effect information is used to generate visual learning effect feedback in the user interface for the user to evaluate his or her self-study status.

[0109] In one embodiment, the method further comprises:

[0110] S21, in response to the user's request for learning resources, extracting fatigue-related features and cognitive load-related features based on the learning state information;

[0111] S22, generating a learning feature vector based on the extracted fatigue-related features and cognitive load-related features, where the learning feature vector includes a fatigue level quantification value and a cognitive load index;

[0112] S23, generating a resource recommendation vector based on the matching relationship between the learned feature vector and the preset resource adaptation rule;

[0113] S24, based on the resource recommendation vector, select matching learning resources from the learning resource library and push them.

[0114] In one embodiment, the immersive learning scenario parameters are generated based on the learning state information using a preset environment generation rule, including:

[0115] S31, generating an ambient light intensity adjustment parameter based on the fatigue level quantized value in the learned feature vector;

[0116] S32, generating a background sound effect resource identifier based on the cognitive load index in the learning feature vector;

[0117] S33, combining the fatigue level quantification value and the cognitive load index to generate spatial layout parameters of the virtual learning scene elements;

[0118] S34, combining the ambient light intensity adjustment parameters, background sound resource identifiers and space layout parameters into immersive learning scene parameters.

[0119] In one embodiment, constructing an immersive learning scenario based on immersive learning scenario parameters includes:

[0120] S41, parsing the spatial layout parameters in the immersive learning scene parameters and constructing the spatial topology structure of the three-dimensional virtual learning space;

[0121] S42, loading the ambient light intensity adjustment parameter in the immersive learning scene parameter to generate a dynamic ambient lighting model;

[0122] S43, generating an audio playback sequence with time sequence control logic according to the background sound effect resource identifier in the immersive learning scene parameter;

[0123] S44, synchronously driving the space topology structure, the dynamic environment lighting model and the audio playback sequence to render the immersive learning scene.

[0124] In one of the embodiments, the method further comprises S51, synchronously driving the space topology structure, the dynamic environment lighting model and the audio playback sequence to render the immersive learning scene by the following formula:

[0125]

[0126] wherein Scene(t) is the immersive learning scene dynamically generated with time t, Φ is a fusion operator, is a space topology structure generation function, α i (t) is a time decay coefficient, P s is a space layout parameter vector, V s is a basic space grid vertex set, T i (·) is a topology transformation function, Λ(P l , t) = k·e -βt ·P l is an environment lighting adjustment function, β is a decay factor positively correlated with the user fatigue level, P l is an environment light intensity adjustment parameter, k is a reference light intensity coefficient, is an audio sequence scheduling function, δ(·) is a time trigger function, τ j is an audio effect trigger timestamp, P a is an audio effect resource identifier set.

[0127] In one of the embodiments, the behavior data and the physiological data of the user are acquired, comprising:

[0128] S61, acquiring visual behavior original data of the user through an image acquisition unit;

[0129] S62, acquiring physiological signal original data of the user through a wearable sensing unit;

[0130] S63, acquiring interactive operation original data of the user through a user interaction unit;

[0131] S64, integrating the visual behavior original data, the physiological signal original data and the interactive operation original data to form an initial multi-source data set;

[0132] S65, performing data cleaning and feature extraction operations on the initial multi-source data set to obtain feature data related to the user behavior as user behavior data, and feature data related to the user's physiological state as physiological data.

[0133] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of an online self-study system that integrates user behavior analysis and immersive interaction as described above.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned system embodiments are implemented.

[0135] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0136] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. An online self-study system that integrates user behavior analysis and immersive interaction, characterized by: The system comprises: Multi-source data acquisition module, used to obtain user behavioral data and physiological data; A user behavior analysis module is used to analyze the user's behavior data and physiological data using a pre-trained user learning state model to generate learning state information; the learning state information includes fatigue characteristics and cognitive load characteristics; A scene parameter generation module, configured to generate immersive learning scene parameters based on the learning state information using a preset environment generation rule; An immersive presentation module is used to construct an immersive learning scene based on the immersive learning scene parameters.

2. The system according to claim 1, wherein: The user behavior analysis module is also used to: In response to acquiring new behavioral data and physiological data of the user, incrementally updating the training data set using the new data; Incrementally training the pre-trained user learning state model using the updated training data set; Utilize the updated user learning status model to analyze the user's behavior data and physiological data collected in real time to generate the user's current learning status information; Analyze the learning effect based on the current learning state information and generate learning effect information; The learning effect information is used to generate visual learning effect feedback in the user interface for users to evaluate their self-study status.

3. The system according to claim 1, wherein: The system also includes a resource push module for: In response to a user's request for learning resources, extracting the fatigue-related features and the cognitive load-related features according to the learning state information; generating a learning feature vector based on the extracted fatigue-related features and cognitive load-related features, wherein the learning feature vector includes a fatigue level quantification value and a cognitive load index; Generate a resource recommendation vector based on a matching relationship between the learned feature vector and a preset resource adaptation rule; According to the resource recommendation vector, matching learning resources are selected from the learning resource library and pushed.

4. The system according to claim 3, characterized in that The scene parameter generation module is also used for: generating an ambient light intensity adjustment parameter based on the fatigue level quantized value in the learned feature vector; generating a background sound effect resource identifier based on the cognitive load index in the learning feature vector; Combining the fatigue level quantification value with the cognitive load index, generating spatial layout parameters of virtual learning scene elements; The ambient light intensity adjustment parameter, background sound resource identifier and space layout parameter are combined into the immersive learning scene parameter.

5. The system according to claim 4, characterized in that The immersive presentation module is further configured to: Parsing the spatial layout parameters in the immersive learning scene parameters to construct a spatial topological structure of the three-dimensional virtual learning space; Loading the ambient light intensity adjustment parameter in the immersive learning scene parameter to generate a dynamic ambient lighting model; Generate an audio playback sequence with timing control logic according to the background sound effect resource identifier in the immersive learning scene parameters; The spatial topology structure, dynamic environment lighting model and audio playback sequence are synchronously driven to render and generate an immersive learning scene.

6. The system according to claim 4, characterized in that The immersive presentation module is further configured to synchronously drive the spatial topology, dynamic environment lighting model, and audio playback sequence to render and generate an immersive learning scene using the following formula: Among them, Scene(t) is the immersive learning scene dynamically generated over time t, Φ is the fusion operator, is the spatial topological structure generating function, α i (t) is the time attenuation coefficient, P s is the spatial layout parameter vector, V s is the base space mesh vertex set, T i (·) is the topological transformation function, Λ(P l , t) = k·e -βt ·P l is the ambient light adjustment function, β is the attenuation factor positively correlated with the user's fatigue level, P l is the ambient light intensity adjustment parameter, k is the reference light intensity coefficient, is the audio sequence scheduling function, δ(·) is the time trigger function, τ j The timestamp of the sound effect trigger, P a Identifies a collection of sound effect resources.

7. The system according to claim 1, wherein: The multi-source data acquisition module is also used for: Acquire the user's visual behavior raw data through the image acquisition unit; Obtaining the user's physiological signal raw data through a wearable sensor unit; Acquiring original data of user interaction operations through the user interaction unit; Integrating the visual behavior raw data, physiological signal raw data, and interactive operation raw data to form an initial multi-source data set; Data cleaning and feature extraction operations are performed on the initial multi-source data set to obtain feature data related to user behavior as the user behavior data, and feature data related to the user's physiological state as the physiological data.

8. An online self-study method that integrates user behavior analysis and immersive interaction, characterized in that: The method comprises: Obtain user behavioral and physiological data; Analyzing the user's behavioral data and physiological data using a pre-trained user learning state model to generate learning state information; the learning state information includes fatigue characteristics and cognitive load characteristics; Generate immersive learning scenario parameters based on the learning state information using preset environment generation rules; An immersive learning scene is constructed based on the immersive learning scene parameters.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the system function steps according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the system function according to any one of claims 1 to 7 are realized.

Images