Immersive ideological and political education scene simulation experience system based on augmented reality

Abstract

The invention provides an immersive ideological and political education scene simulation experience system based on augmented reality, and the system comprises a scene construction module which generates a dynamic virtual ideological and political scene, and an augmented reality presentation module which carries out the three-dimensional fusion, rendering and outputting of a virtual object and a real environment. The multi-mode interaction and response module collects gestures, voice and physiological signals and recognizes intention to drive scene dynamic feedback, the learning evaluation and personalized adaptation module generates quantitative evaluation according to user behavior data and dynamically adjusts scene content or difficulty, and immersive and personalized ideological and political education experience is achieved. The immersive ideological and political education experience can be realized, the scene content and difficulty can be dynamically adjusted according to the user behavior data, and the learning effect and the personalized experience are improved.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and more specifically, to an immersive ideological and political education scenario simulation experience system based on augmented reality. Background Technology

[0002] In today's digital age, ideological and political education, as an important means of cultivating correct values, faces problems such as the monotony of traditional teaching methods and low student participation. With the development of augmented reality (AR) technology, its application in education has gradually gained attention. AR technology, by integrating virtual information with the real environment, brings users an immersive experience, effectively enhancing the fun and interactivity of learning. However, existing technologies, when applying AR to ideological and political education, often lack deep integration and personalized adaptation of teaching content. For example, most systems can only provide static virtual scenes and cannot dynamically generate and adjust scenario content according to the teaching theme; at the same time, the handling of user interaction is relatively simple, failing to achieve deep integration and semantic understanding of multimodal interactive input, resulting in a limited user experience. Furthermore, existing systems are also weak in learning effectiveness evaluation, lacking multi-dimensional behavioral data analysis and dynamic adjustment mechanisms, making it difficult to meet the learning needs of different students.

[0003] In implementing the embodiments of the present invention, the prior art has at least the following problems or defects: First, it is impossible to dynamically generate virtual scenarios with dynamic plots based on the theme of ideological and political education, which makes it difficult to update teaching content; second, it lacks deep integration and semantic understanding of user multimodal interactive input, making it impossible to achieve accurate interactive response; third, the evaluation of learning effect is not comprehensive enough, and it is impossible to dynamically adjust the teaching content and difficulty based on user behavior data, making it difficult to meet personalized learning needs. Summary of the Invention

[0004] This invention provides an augmented reality-based immersive ideological and political education scenario simulation experience system, comprising:

[0005] The scenario building module is used to generate and manage virtual ideological and political education scenarios with dynamic plots based on the themes of ideological and political education;

[0006] The augmented reality rendering module is used to acquire information about the real environment, and to perform three-dimensional spatial fusion and rendering of the virtual objects in the virtual ideological and political education scenario with the information about the real environment to generate and output augmented reality images;

[0007] The multimodal interaction and response module is used to collect various interactive inputs from users, perform semantic understanding and intent recognition on the interactive inputs, generate corresponding interactive instructions, and drive the virtual ideological and political education scenario to generate dynamic feedback based on the interactive instructions.

[0008] The learning assessment and personalized adaptation module is used to generate quantitative learning assessment results based on multi-dimensional behavioral data of users during the experience process, and dynamically adjust the presentation content or difficulty parameters of the subsequent virtual ideological and political education scenarios based on the quantitative learning assessment results.

[0009] Furthermore, the scenario construction module includes:

[0010] The dynamic parameterized scene generation unit is used to dynamically combine and generate a virtual scene containing interactive virtual objects from a 3D model database based on the ideological and political education theme and preset scene element parameters, wherein the scene element parameters include historical period feature parameters and environmental atmosphere parameters.

[0011] An adjustable narrative network unit is used to construct a directed graph-based narrative logic network for ideological and political education. The narrative logic network includes multiple scenario nodes and plot branch paths connecting the scenario nodes. Each plot branch path is associated with a logical judgment condition. The adjustable narrative network unit activates the corresponding logical judgment condition based on user profile information or real-time interaction results to determine the current scenario progression path.

[0012] Furthermore, the augmented reality rendering module includes:

[0013] The multi-source sensor fusion registration unit is used to simultaneously acquire real-world images collected by image sensors and motion data collected by inertial measurement units, and calculate high-precision user position, attitude, and three-dimensional structure information of the real-world environment based on the visual SLAM algorithm and the motion data, and generate a spatial registration matrix.

[0014] The virtual-real light and shadow fusion and occlusion processing unit is used to place the virtual object in the three-dimensional structure of the real environment according to the spatial registration matrix, and calculate the light and shadow effect of the virtual object in real time based on the lighting estimation results of the real environment, while performing real-time occlusion relationship calculation and rendering between the virtual object and objects in the real environment.

[0015] An adaptive rendering output unit is used to dynamically adjust the rendering resolution and frame rate of the augmented reality image based on the performance parameters of the augmented reality display device and the user's viewpoint movement speed, and output the final rendered image.

[0016] Furthermore, the multimodal interaction and response module includes:

[0017] The multi-channel input sensing unit is used to simultaneously collect the user's gesture input, voice input, and physiological signal input obtained through biosensors, forming the original interactive data stream;

[0018] An intent recognition and instruction generation unit is used to perform fusion analysis on the original interactive data stream. The fusion analysis includes: recognizing the operation type and target virtual object corresponding to the gesture input, recognizing keywords and command intent in the voice input, and analyzing the emotional state reflected by the physiological signal input; and generating structured interactive instructions based on the analysis results.

[0019] The scenario dynamic response unit is used to call the physics engine to perform real-time physical simulation updates of the state of the virtual object according to the interaction instructions, or to drive the virtual characters in the virtual ideological and political education scenario to perform dialogue and behavioral feedback, and trigger the plot advancement in the adjustable narrative network unit.

[0020] Furthermore, the learning assessment and personalized adaptation module includes:

[0021] A multi-dimensional behavioral data acquisition unit is used to continuously record the user's operation sequence, key selection nodes, task completion time, emotional tendency of voice content, and physiological indicator change curves monitored by biosensors during the experience process, as the multi-dimensional behavioral data.

[0022] The multi-dimensional evaluation model calculation unit is used to calculate the user's quantitative indicators in the dimensions of knowledge cognition, value judgment, emotional identification and behavioral practice in parallel based on the multi-dimensional behavioral data.

[0023] The personalized strategy adaptation unit is used to generate personalized scenario adjustment parameters for the current user based on the quantitative indicators output by the multi-dimensional evaluation model calculation unit according to the preset adaptation rule library, and to feed back the personalized scenario adjustment parameters to the scenario construction module and the augmented reality presentation module.

[0024] Furthermore, in the calculation unit of the multi-dimensional evaluation model, the calculation formula for the quantitative index of the emotional identification dimension—emotional engagement E—is as follows:

[0025]

[0026] in, The attention focus index is calculated as the ratio of the time a user's gaze lingers on a key virtual object to the total time spent on it. The normalized physiological response fluctuation index, This represents the difference between the average heart rate and the baseline heart rate during the experience. Baseline heart rate; The positive sentiment index of voice is derived from the acoustic feature analysis of user voice. , , These are the weighting coefficients for the attention focus index, physiological response fluctuation index, and positive voice emotion index, respectively. ; The time decay coefficient, This represents the time difference between the current assessment moment and the critical trigger moment of the scenario.

[0027] Furthermore, the learning assessment and personalized adaptation module also includes a comprehensive assessment report generation unit, which is used for:

[0028] Receive the quantitative indicators of each dimension output by the calculation unit of the multi-dimensional evaluation model;

[0029] The comprehensive evaluation value S of the learning effect under the current ideological and political education theme is calculated using the following formula:

[0030]

[0031] in, The quantification value represents the i-th dimension, and N is the total number of dimensions; The dynamic weight of the i-th dimension at time t is adaptively adjusted based on the teaching objectives and user history of the ideological and political education theme. The semantic fit score between the user's choice at the key decision point j and the preset ideal answer; The decision-making impact coefficient;

[0032] Based on the comprehensive evaluation value S of the learning effect and the quantitative indicators of each dimension, a quantitative learning evaluation result combining text and graphics is generated.

[0033] Furthermore, it also includes:

[0034] The scenario generation algorithm module includes:

[0035] The multimodal teaching resource parsing unit is used to receive and parse the input original materials for ideological and political education, including text cases, historical video materials, audio archives and image materials; the parsing includes: entity recognition, event extraction and sentiment analysis of text, key scene and character action recognition of video and image, speech-to-text conversion and sentiment tagging of audio, and generation of a structured multimodal ideological and political education knowledge graph;

[0036] The narrative logic automatic construction unit is connected to the multimodal teaching resource parsing unit. It is used to learn the structural features and plot transformation patterns of high-quality historical narrative logic networks based on the multimodal ideological and political education knowledge graph and using a graph neural network model. It can also automatically generate candidate narrative logic networks and their corresponding plot-driven rule sets for new ideological and political education themes.

[0037] The dynamic script and scene parameter generation unit is connected to the narrative logic automatic construction unit. It is used to call a pre-trained large language model based on the selected candidate narrative logic network to generate dynamic dialogue scripts and scene description texts that conform to the historical background and character settings, and output a set of virtual scene configuration parameters that match the dynamic dialogue scripts and scene description texts. The set of virtual scene configuration parameters is provided to the scene construction module.

[0038] Furthermore, it also includes:

[0039] The cloud-based collaboration and persistence module includes:

[0040] The distributed user state management unit is used to create and maintain an independent state container for each online user session. The state container stores the user's interaction context, the current snapshot of the virtual ideological and political education scenario, and the real-time evaluation intermediate data generated by the learning evaluation and personalized adaptation module in real time. The unit supports synchronizing the state change events of a specific user to other user terminals in the same collaborative scenario based on a publish-subscribe mechanism.

[0041] The massive behavioral data analysis and model optimization unit is used to receive and persistently store all anonymized multi-dimensional behavioral data and corresponding quantitative learning evaluation results to form a historical training dataset. This unit periodically uses the historical training dataset to jointly train and optimize the evaluation model in the multi-dimensional evaluation model calculation unit, the adaptation rule base in the personalized strategy adaptation unit, and the graph neural network model and large language model in the scenario generation algorithm module, and distributes the optimized model parameters to each terminal module.

[0042] The security and access control unit is used to manage user authentication, data access permissions, and access levels for different ideological and political education scenarios. This unit implements end-to-end encrypted data transmission and performs differential privacy protection processing on all stored personalized data.

[0043] Furthermore, the adaptive rendering output unit is specifically used for:

[0044] A rendering resource constraint model is constructed based on the type of augmented reality display device, remaining battery power, real-time computing load, and network bandwidth status.

[0045] Under the rendering resource constraint model, with the goal of maintaining a preset minimum acceptable interactive frame rate, rendering resources are dynamically allocated through an optimization algorithm, the optimization including:

[0046] For virtual objects located in the user's field of vision and strongly related to the current plot progression, high-precision models and real-time dynamic lighting and shadows are used for rendering.

[0047] For virtual objects located at the edge of the field of view or in the secondary background, a level-of-detail model and static light and shadow maps are used for rendering;

[0048] When high-speed movement of the user's viewpoint or system resource shortage is detected, the rendering resolution of non-critical virtual objects is automatically reduced and asynchronous time warp technology is enabled for image compensation.

[0049] The images output by the adaptive rendering output unit are compatible with a variety of augmented reality display devices, including optical see-through head-mounted displays, video see-through head-mounted displays, and mobile smart terminals with depth sensing capabilities.

[0050] The embodiments of the present invention have at least the following beneficial effects:

[0051] 1. Through the synergistic effect of the scenario construction module and the augmented reality presentation module, virtual ideological and political education scenarios with dynamic plots can be dynamically generated and managed based on the theme of ideological and political education. These scenarios can then be integrated and rendered in three-dimensional space with the real environment to generate immersive augmented reality visuals. This not only enriches the content and form of ideological and political education but also enhances user participation and immersion, solving the problems of the monotony and lack of appeal of traditional ideological and political education methods. It allows users to understand and experience the content of ideological and political education more intuitively, thus enhancing learning effectiveness.

[0052] 2. The multimodal interaction and response module simultaneously collects and deeply integrates various interactive inputs, including user gestures, voice, and physiological signals. It accurately identifies user semantics and intentions, generating corresponding interactive commands to drive dynamic feedback within the virtual scenario. This significantly improves the naturalness and accuracy of the interaction, solving the problems of limited interaction methods and poor user experience in existing technologies. It enables users to interact with virtual ideological and political education scenarios in a more natural and convenient way, enhancing user experience and engagement.

[0053] 3. The learning assessment and personalized adaptation module can generate quantitative learning assessment results based on multi-dimensional behavioral data of users during the experience, and dynamically adjust the presentation content or difficulty parameters of subsequent virtual ideological and political education scenarios based on this. This function realizes personalized adaptation of ideological and political education, solving the problem of lack of personalized teaching and accurate assessment in existing technologies. It can provide customized learning content and difficulty according to each user's learning progress and characteristics, thereby better meeting the learning needs of different users and improving the overall effectiveness of ideological and political education. Attached Figure Description

[0054] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example and not limitation, wherein:

[0055] Figure 1 This is a schematic diagram of the structure of an immersive ideological and political education scenario simulation experience system based on augmented reality, provided in an embodiment of the present invention. Detailed Implementation

[0056] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way.

[0057] The following is for reference. Figure 1 , Figure 1 This is a schematic diagram of the structure of an augmented reality-based immersive ideological and political education scenario simulation experience system provided in an embodiment of the present invention. Figure 1 As shown, an immersive ideological and political education scenario simulation experience system based on augmented reality includes:

[0058] The scenario construction module 101 is used to generate and manage virtual ideological and political education scenarios with dynamic plots based on the theme of ideological and political education;

[0059] Augmented reality rendering module 102 is used to acquire real environment information, perform three-dimensional spatial fusion and rendering of virtual objects in the virtual ideological and political education scenario with the real environment information, and generate and output augmented reality images.

[0060] The multimodal interaction and response module 103 is used to collect various interactive inputs from users, perform semantic understanding and intent recognition on the interactive inputs, generate corresponding interactive instructions, and drive the virtual ideological and political education scenario to generate dynamic feedback according to the interactive instructions.

[0061] The learning assessment and personalized adaptation module 104 is used to generate quantitative learning assessment results based on the user's multi-dimensional behavioral data during the experience process, and dynamically adjust the presentation content or difficulty parameters of the subsequent virtual ideological and political education scenario based on the quantitative learning assessment results.

[0062] Specifically, the scenario construction module includes a dynamically parameterized scenario generation unit and an adjustable narrative network unit. The dynamically parameterized scenario generation unit dynamically combines and generates virtual scenarios containing interactive virtual objects from a 3D model database based on the ideological and political education theme and preset scenario element parameters. These scenario element parameters include historical period characteristic parameters and environmental atmosphere parameters, which define the style, background, and atmosphere of the virtual scenario. For example, historical period characteristic parameters could be architectural styles or clothing characteristics of a specific historical period, while environmental atmosphere parameters could be lighting effects or weather conditions. The adjustable narrative network unit is used to construct a directed graph-based narrative logic network for ideological and political education. This network includes multiple scenario nodes and plot branch paths connecting these nodes, with each plot branch path associated with a logical judgment condition. Scenario nodes can be key events or scenes in the virtual scenario, plot branch paths represent different plot development paths, and logical judgment conditions determine the plot's direction based on user interaction or behavior.

[0063] In some embodiments, the scenario building module includes:

[0064] The dynamic parameterized scene generation unit is used to dynamically combine and generate a virtual scene containing interactive virtual objects from a 3D model database based on the ideological and political education theme and preset scene element parameters, wherein the scene element parameters include historical period feature parameters and environmental atmosphere parameters.

[0065] An adjustable narrative network unit is used to construct a directed graph-based narrative logic network for ideological and political education. The narrative logic network includes multiple scenario nodes and plot branch paths connecting the scenario nodes. Each plot branch path is associated with a logical judgment condition. The adjustable narrative network unit activates the corresponding logical judgment condition based on user profile information or real-time interaction results to determine the current scenario progression path.

[0066] The scenario construction module includes a dynamically parameterized scenario generation unit and an adjustable narrative network unit. The dynamically parameterized scenario generation unit dynamically combines and generates virtual scenarios containing interactive virtual objects from a 3D model database based on the ideological and political education theme and preset scenario element parameters. These scenario element parameters define the style, background, and atmosphere of the virtual scenario, such as historical period characteristics and environmental atmosphere parameters. These parameters ensure that the generated virtual scenario accurately reflects the theme and background of ideological and political education, such as the architectural style and clothing characteristics of a specific historical period. The adjustable narrative network unit constructs a directed graph-based narrative logic network for ideological and political education. This network consists of multiple scenario nodes and plot branch paths, each associated with a logical judgment condition. Scenario nodes can be key events or scenes in the virtual scenario, plot branch paths represent different plot development paths, and logical judgment conditions determine the plot's direction based on user interaction or behavior. In this way, the scenario construction module can dynamically generate virtual ideological and political education scenarios with dynamic plots based on the theme, providing users with an immersive educational experience.

[0067] Specifically, the dynamic parameterized scene generation unit generates virtual scenes based on the theme of ideological and political education and preset scene element parameters. For example, when the theme is modern Chinese history, historical period characteristic parameters may include architectural styles and clothing characteristics during the May Fourth Movement, while environmental atmosphere parameters may include lighting effects and weather conditions. These parameters are extracted from a 3D model database and combined to generate virtual scenes with specific historical backgrounds. The adjustable narrative network unit dynamically adjusts the narrative logic by constructing a directed graph. Scene nodes in the directed graph represent key events or scenes in the virtual scene, plot branch paths represent different plot development paths, and logical judgment conditions are used to determine the direction of the plot based on user interactions or behaviors. For example, choices made or tasks completed by users in the virtual scene will affect the development of subsequent plots, thus achieving personalized scene progression paths. This dynamic adjustment mechanism allows virtual ideological and political education scenes to flexibly change according to real-time user interactions, enhancing user participation and learning effectiveness.

[0068] Preferably, when constructing the narrative logic network, the adjustable narrative network unit activates corresponding logical judgment conditions based on user profile information or real-time interaction results. User profile information is constructed by collecting and analyzing user personal information, learning history, and preferences, reflecting user characteristics and needs. Real-time interaction results are data generated by users' operations and interactions in the virtual scenario, such as user clicks, selections, and voice commands. By combining user profile information and real-time interaction results, the adjustable narrative network unit can more accurately adjust the development path of the virtual scenario, enabling each user to obtain a personalized learning experience.

[0069] Furthermore, the construction of narrative logic networks can be optimized using machine learning algorithms. For example, graph neural network models can be used to learn the structural features and plot transition patterns of high-quality historical narrative logic networks, thereby automatically generating candidate narrative logic networks and their corresponding plot-driven rule sets for new ideological and political education themes. This data-driven approach to generating narrative logic can further enhance the diversity and adaptability of virtual ideological and political education scenarios, meeting the learning needs of different users.

[0070] In some embodiments, the augmented reality rendering module includes:

[0071] The multi-source sensor fusion registration unit is used to simultaneously acquire real-world images collected by image sensors and motion data collected by inertial measurement units, and calculate high-precision user position, attitude, and three-dimensional structure information of the real-world environment based on the visual SLAM algorithm and the motion data, and generate a spatial registration matrix.

[0072] The virtual-real light and shadow fusion and occlusion processing unit is used to place the virtual object in the three-dimensional structure of the real environment according to the spatial registration matrix, and calculate the light and shadow effect of the virtual object in real time based on the lighting estimation results of the real environment, while performing real-time occlusion relationship calculation and rendering between the virtual object and objects in the real environment.

[0073] An adaptive rendering output unit is used to dynamically adjust the rendering resolution and frame rate of the augmented reality image based on the performance parameters of the augmented reality display device and the user's viewpoint movement speed, and output the final rendered image.

[0074] Specifically, the augmented reality rendering module includes a multi-source sensor fusion registration unit, a virtual-real lighting and shadow fusion and occlusion processing unit, and an adaptive rendering output unit. The multi-source sensor fusion registration unit simultaneously acquires images of the real environment from image sensors and motion data from inertial measurement units. Based on the visual SLAM algorithm and motion data, it calculates the user's position, posture, and the 3D structural information of the real environment to generate a spatial registration matrix. The visual SLAM algorithm here is a technique for simultaneous localization and mapping using visual sensors, used to determine the user's position and posture in space in real time. The spatial registration matrix is ​​a mathematical model used to describe the spatial relationship between virtual objects and the real environment. The virtual-real lighting and shadow fusion and occlusion processing unit, based on the spatial registration matrix, places virtual objects in the real environment and calculates the lighting effects of the virtual objects according to the lighting conditions of the real environment, while also handling the occlusion relationship between virtual objects and real objects. The adaptive rendering output unit dynamically adjusts the rendering resolution and frame rate according to the performance parameters of the augmented reality display device and the user's viewpoint movement speed to ensure the smoothness and stability of the image.

[0075] When rendering and integrating 3D space, the augmented reality (AR) rendering module constructs a rendering resource constraint model based on the type of AR display device, remaining battery power, real-time computing load, and network bandwidth. Under this model, the module dynamically allocates rendering resources through optimization algorithms to maintain a preset minimum acceptable interactive frame rate. For example, virtual objects located in the user's field of view and strongly related to the current plot are rendered using high-precision models and real-time dynamic lighting and shadows, while virtual objects located at the edge of the field of view or in the secondary background are rendered using level-of-detail models and static lighting and shadow maps. When high-speed movement of the user's viewpoint or system resource constraints are detected, the rendering resolution of non-critical virtual objects is automatically reduced, and asynchronous time warp technology is used for image compensation. This adaptive rendering strategy ensures a high-quality AR experience under different device and network conditions while optimizing the efficiency of system resource utilization.

[0076] In some embodiments, the multimodal interaction and response module includes:

[0077] The multi-channel input sensing unit is used to simultaneously collect the user's gesture input, voice input, and physiological signal input obtained through biosensors, forming the original interactive data stream;

[0078] An intent recognition and instruction generation unit is used to perform fusion analysis on the original interactive data stream. The fusion analysis includes: recognizing the operation type and target virtual object corresponding to the gesture input, recognizing keywords and command intent in the voice input, and analyzing the emotional state reflected by the physiological signal input; and generating structured interactive instructions based on the analysis results.

[0079] The scenario dynamic response unit is used to call the physics engine to perform real-time physical simulation updates of the state of the virtual object according to the interaction instructions, or to drive the virtual characters in the virtual ideological and political education scenario to perform dialogue and behavioral feedback, and trigger the plot advancement in the adjustable narrative network unit.

[0080] The multimodal interaction and response module includes a multi-channel input sensing unit, an intent recognition and command generation unit, and a contextual dynamic response unit. The multi-channel input sensing unit simultaneously collects user gesture input, voice input, and physiological signal input obtained through biosensors, forming the raw interactive data stream. Gesture input refers to the user's intention expressed through hand gestures, such as waving or clicking; voice input refers to the user's interaction through voice commands; and physiological signal input refers to user physiological data obtained through biosensors, such as heart rate and skin conductance, used to analyze the user's emotional state. The intent recognition and command generation unit fuses and analyzes the raw interactive data stream, identifying the operation type and target virtual object of the gesture input, keywords and command intent in the voice input, and the emotional state reflected in the physiological signals, and generates structured interactive commands. The contextual dynamic response unit then calls the physics engine to update the state of virtual objects based on the interactive commands, or drives virtual characters to perform dialogues and behavioral feedback, triggering plot progression.

[0081] The intent recognition and command generation unit employs deep learning algorithms to fuse and analyze multimodal data when processing the raw interactive data stream. For example, for voice input, the system converts speech into text using speech recognition technology and extracts keywords and command intent using natural language processing algorithms; for gesture input, the system identifies gestures and their corresponding operational targets using computer vision technology; for physiological signal input, the system analyzes physiological data using signal processing algorithms to determine the user's emotional state. These analysis results are integrated to generate structured interactive commands. Upon receiving an interactive command, the contextual dynamic response unit invokes the physics engine based on the command's specific content to perform real-time physical simulation updates of virtual objects, such as changing the position or state of virtual objects or triggering actions and dialogues of virtual characters. Furthermore, the system dynamically adjusts its response strategy based on the user's interaction history and contextual information to provide a more natural and coherent interactive experience.

[0082] In some embodiments, the learning assessment and personalized adaptation module includes:

[0083] A multi-dimensional behavioral data acquisition unit is used to continuously record the user's operation sequence, key selection nodes, task completion time, emotional tendency of voice content, and physiological indicator change curves monitored by biosensors during the experience process, as the multi-dimensional behavioral data.

[0084] The multi-dimensional evaluation model calculation unit is used to calculate the user's quantitative indicators in the dimensions of knowledge cognition, value judgment, emotional identification and behavioral practice in parallel based on the multi-dimensional behavioral data.

[0085] The personalized strategy adaptation unit is used to generate personalized scenario adjustment parameters for the current user based on the quantitative indicators output by the multi-dimensional evaluation model calculation unit according to the preset adaptation rule library, and to feed back the personalized scenario adjustment parameters to the scenario construction module and the augmented reality presentation module.

[0086] Specifically, the learning assessment and personalized adaptation module includes a multi-dimensional behavioral data collection unit, a multi-dimensional assessment model calculation unit, and a personalized strategy adaptation unit. The multi-dimensional behavioral data collection unit continuously records the user's operation sequence, key selection nodes, task completion time, emotional tone of voice content, and physiological indicator change curves monitored by biosensors during the experience. This data, as multi-dimensional behavioral data, comprehensively reflects the user's performance in the virtual ideological and political education scenario. Operation sequence refers to a series of operations performed by the user in the virtual environment, such as clicking and selecting; key selection nodes refer to important decision points made by the user in the scenario; task completion time refers to the time spent by the user to complete a specific task; emotional tone of voice content refers to the user's emotional state obtained through voice analysis; and physiological indicator change curves refer to changes in the user's physiological data monitored by biosensors, such as heart rate and skin conductance. Based on this behavioral data, the multi-dimensional assessment model calculation unit calculates quantitative indicators for the user in dimensions such as knowledge cognition, value judgment, emotional identification, and behavioral practice. The personalized strategy adaptation unit generates personalized scenario adjustment parameters for the current user based on quantitative indicators and a preset adaptation rule library, and feeds these parameters back to the scenario building module and the augmented reality presentation module.

[0087] In some embodiments, the calculation formula for the quantitative index of emotional identification dimension—emotional engagement E—in the multi-dimensional evaluation model calculation unit is as follows:

[0088]

[0089] in, The attention focus index is calculated as the ratio of the time a user's gaze lingers on a key virtual object to the total time spent on it. The normalized physiological response fluctuation index, This represents the difference between the average heart rate and the baseline heart rate during the experience. Baseline heart rate; The positive sentiment index of voice is derived from the acoustic feature analysis of user voice. , , These are the weighting coefficients for the attention focus index, physiological response fluctuation index, and positive voice emotion index, respectively. ; The time decay coefficient, This represents the time difference between the current assessment moment and the critical trigger moment of the scenario.

[0090] The attention focus index is calculated as the ratio of the duration of the user's gaze on a key virtual object to the total time spent on it, reflecting the user's level of attention to important information. The physiological response fluctuation index is a normalized value obtained by dividing the difference between the average heart rate and the baseline heart rate during the experience by the baseline heart rate, used to measure the user's level of physiological excitement during the experience. Furthermore, the voice emotion positivity index is based on the acoustic feature analysis of the user's voice, reflecting the user's emotional tendency. Weighting coefficients correspond to the above three indices, and their sum is 1, used to balance the contribution of each indicator in the calculation of emotional engagement. The time decay coefficient and time difference are used to consider the impact of time factors on emotional engagement, where the time difference is the time difference between the current evaluation moment and the key trigger moment of the scenario, reflecting the changing trend of emotional engagement over time. The settings of these parameters ensure that the calculation of emotional engagement can comprehensively reflect the user's emotional state in different dimensions.

[0091] In some embodiments, the learning assessment and personalized adaptation module further includes a comprehensive assessment report generation unit, which is used for:

[0092] Receive the quantitative indicators of each dimension output by the calculation unit of the multi-dimensional evaluation model;

[0093] The comprehensive evaluation value S of the learning effect under the current ideological and political education theme is calculated using the following formula:

[0094]

[0095] in, The quantification value represents the i-th dimension, and N is the total number of dimensions; The dynamic weight of the i-th dimension at time t is adaptively adjusted based on the teaching objectives and user history of the ideological and political education theme. The semantic fit score between the user's choice at the key decision point j and the preset ideal answer; The decision-making impact coefficient;

[0096] Based on the comprehensive evaluation value S of the learning effect and the quantitative indicators of each dimension, a quantitative learning evaluation result combining text and graphics is generated.

[0097] In this invention, the learning assessment and personalized adaptation module also includes a comprehensive assessment report generation unit. Its function is to receive the quantitative indicators for each dimension output by the multi-dimensional assessment model calculation unit and calculate the comprehensive evaluation value of learning effectiveness under the current ideological and political education theme. This unit generates a quantitative learning assessment result combining text and graphics by comprehensively considering the quantitative indicators of multiple dimensions and the user's performance at key decision points. This process not only provides a comprehensive assessment of learning effectiveness but also intuitively reflects the user's overall performance in the ideological and political education scenario simulation through the comprehensive evaluation value, providing important feedback information for teachers and learners and helping to further optimize teaching content and methods.

[0098] Quantitative indicators of various dimensions This refers to the quantitative evaluation results of users in dimensions such as knowledge cognition, value judgment, emotional identification, and behavioral practice. These indicators are derived through a multi-dimensional evaluation model calculation unit. Dynamic weights The weights are adaptively adjusted based on the teaching objectives and emphases of ideological and political education themes, as well as users' historical performance, to reflect the importance of different dimensions at different points in time. Key Decision Points This refers to key decision points for users in virtual ideological and political education scenarios. The semantic fit score between these points and the preset ideal answer reflects the accuracy and rationality of the user's decision. (Decision Influence Coefficient) This is a parameter used to adjust the degree of influence of key decision points. The formula for calculating the overall evaluation value S combines these parameters, deriving a numerical value that reflects the user's overall learning performance through a weighted summation. This calculation method can comprehensively reflect the user's performance in different dimensions and further refine the measurement of learning effectiveness through the evaluation of key decision points.

[0099] In some embodiments, it also includes:

[0100] The scenario generation algorithm module includes:

[0101] The multimodal teaching resource parsing unit is used to receive and parse the input original materials for ideological and political education, including text cases, historical video materials, audio archives and image materials; the parsing includes: entity recognition, event extraction and sentiment analysis of text, key scene and character action recognition of video and image, speech-to-text conversion and sentiment tagging of audio, and generation of a structured multimodal ideological and political education knowledge graph;

[0102] The narrative logic automatic construction unit is connected to the multimodal teaching resource parsing unit. It is used to learn the structural features and plot transformation patterns of high-quality historical narrative logic networks based on the multimodal ideological and political education knowledge graph and using a graph neural network model. It can also automatically generate candidate narrative logic networks and their corresponding plot-driven rule sets for new ideological and political education themes.

[0103] The dynamic script and scene parameter generation unit is connected to the narrative logic automatic construction unit. It is used to call a pre-trained large language model based on the selected candidate narrative logic network to generate dynamic dialogue scripts and scene description texts that conform to the historical background and character settings, and output a set of virtual scene configuration parameters that match the dynamic dialogue scripts and scene description texts. The set of virtual scene configuration parameters is provided to the scene construction module.

[0104] In this invention, a scenario generation algorithm module is used to automate the generation and optimization of ideological and political education scenarios. This module includes a multimodal teaching resource parsing unit, a narrative logic automatic construction unit, and a dynamic script and scenario parameter generation unit. These units work together to parse input original ideological and political education materials, such as text, images, and audio, into a structured knowledge graph, and automatically generate a narrative logic network and virtual scenario configuration parameters based on this graph. This process not only improves the efficiency of scenario generation but also ensures the close relevance of scenario content to the theme of ideological and political education, providing users with a richer and more personalized learning experience.

[0105] The multimodal teaching resource parsing unit receives and parses the input original materials for ideological and political education, including textual cases, historical video materials, audio archives, and image materials. The parsing process involves entity recognition, event extraction, and sentiment analysis of the text; key scene and character action recognition of the videos and images; and speech-to-text conversion and sentiment tagging of the audio. These parsing results are integrated into a structured multimodal ideological and political education knowledge graph. The narrative logic automatic construction unit, based on the knowledge graph, uses a graph neural network model to learn the structural features and plot transition patterns of high-quality historical narrative logic networks, and generates candidate narrative logic networks and their corresponding plot-driven rule sets for new ideological and political education themes. The dynamic script and scene parameter generation unit, based on the selected narrative logic network, calls a pre-trained large language model to generate dynamic dialogue scripts and scene description text, and outputs a matching set of virtual scene configuration parameters. These parameter sets are ultimately provided to the scenario construction module for virtual scene generation.

[0106] In some embodiments, it also includes:

[0107] The cloud-based collaboration and persistence module includes:

[0108] The distributed user state management unit is used to create and maintain an independent state container for each online user session. The state container stores the user's interaction context, the current snapshot of the virtual ideological and political education scenario, and the real-time evaluation intermediate data generated by the learning evaluation and personalized adaptation module in real time. The unit supports synchronizing the state change events of a specific user to other user terminals in the same collaborative scenario based on a publish-subscribe mechanism.

[0109] The massive behavioral data analysis and model optimization unit is used to receive and persistently store all anonymized multi-dimensional behavioral data and corresponding quantitative learning evaluation results to form a historical training dataset. This unit periodically uses the historical training dataset to jointly train and optimize the evaluation model in the multi-dimensional evaluation model calculation unit, the adaptation rule base in the personalized strategy adaptation unit, and the graph neural network model and large language model in the scenario generation algorithm module, and distributes the optimized model parameters to each terminal module.

[0110] The security and access control unit is used to manage user authentication, data access permissions, and access levels for different ideological and political education scenarios. This unit implements end-to-end encrypted data transmission and performs differential privacy protection processing on all stored personalized data.

[0111] The distributed user state management unit creates and maintains an independent state container for each online user session, storing the user's interaction context, the current snapshot of the virtual ideological and political education scenario, and real-time evaluation intermediate data in real time. This unit supports a publish-subscribe mechanism to synchronize state change events of a specific user to other user terminals in the same collaborative scenario. The massive behavioral data analysis and model optimization unit receives and persistently stores all anonymized multi-dimensional behavioral data and corresponding quantitative learning evaluation results, forming a historical training dataset. This unit periodically uses the historical training dataset to jointly train and optimize the evaluation model, the adaptation rule base, and the model in the scenario generation algorithm module, and distributes the optimized model parameters to each terminal module. The security and access control unit manages user authentication, data access permissions, and access levels for different ideological and political education scenario content, implements end-to-end encrypted data transmission, and performs differential privacy protection processing on all stored personalized data.

[0112] The distributed user state management unit ensures high availability and consistency of data through distributed database technology when synchronizing user states. For example, it uses distributed caching and message queue technologies to quickly synchronize user state change events, ensuring smooth multi-user collaboration. The massive behavioral data analysis and model optimization unit uses machine learning algorithms to analyze historical behavioral data and extract key features for model training. For example, it uses deep learning frameworks to model user behavior sequences and optimize the accuracy of the evaluation model. The security and access control unit employs advanced encryption algorithms and privacy protection technologies, such as differential privacy algorithms, to ensure the security and privacy of user data when implementing data encryption and privacy protection. Through these optimization measures, the cloud collaboration and persistence module not only improves system performance and reliability but also enhances data security and privacy protection capabilities.

[0113] In some embodiments, the adaptive rendering output unit is specifically used for:

[0114] A rendering resource constraint model is constructed based on the type of augmented reality display device, remaining battery power, real-time computing load, and network bandwidth status.

[0115] Under the rendering resource constraint model, with the goal of maintaining a preset minimum acceptable interactive frame rate, rendering resources are dynamically allocated through an optimization algorithm, the optimization including:

[0116] For virtual objects located in the user's field of vision and strongly related to the current plot progression, high-precision models and real-time dynamic lighting and shadows are used for rendering.

[0117] For virtual objects located at the edge of the field of view or in the secondary background, a level-of-detail model and static light and shadow maps are used for rendering;

[0118] When high-speed movement of the user's viewpoint or system resource shortage is detected, the rendering resolution of non-critical virtual objects is automatically reduced and asynchronous time warp technology is enabled for image compensation.

[0119] The images output by the adaptive rendering output unit are compatible with a variety of augmented reality display devices, including optical see-through head-mounted displays, video see-through head-mounted displays, and mobile smart terminals with depth sensing capabilities.

[0120] When optimizing rendering, the adaptive rendering output unit constructs a rendering resource constraint model based on the type of augmented reality display device, remaining battery power, real-time computing load, and network bandwidth. Under this model, the system dynamically allocates rendering resources through optimization algorithms, aiming to maintain a preset minimum acceptable interactive frame rate. For example, virtual objects located in the user's field of view and strongly related to the current plot progression are rendered using high-precision models and real-time dynamic lighting and shadows; while virtual objects located at the edge of the field of view or in the secondary background are rendered using level-of-detail models and static lighting and shadow maps. When high-speed movement of the user's viewpoint or system resource constraints are detected, the rendering resolution of non-critical virtual objects is automatically reduced, and asynchronous time warp technology is used for image compensation. This dynamic adjustment mechanism ensures a high-quality augmented reality experience under different device and network conditions, while optimizing the efficiency of system resource utilization.

[0121] In its implementation, the adaptive rendering output unit is specifically optimized for different types of augmented reality display devices. For example, for optical see-through head-mounted displays, the system optimizes transparency and lighting effects to ensure a natural integration of virtual objects with the real environment; while for video see-through head-mounted displays, the focus is on optimizing real-time image processing speed and rendering accuracy. Regarding rendering resource allocation, the system dynamically adjusts its rendering strategy based on the device's remaining battery power and real-time computing load. For instance, when the device's battery is low, the system prioritizes allocating resources to critical virtual objects to ensure the normal operation of core interactive functions. Simultaneously, the system adjusts the rendering resolution based on network bandwidth conditions to ensure a smooth interactive experience even in low-bandwidth environments. Through these optimization measures, the adaptive rendering output unit not only improves system compatibility and stability but also optimizes resource utilization efficiency, ensuring users receive a smooth interactive experience across various augmented reality display devices.

[0122] The above description is merely an explanation of some preferred embodiments of the present invention and the technical principles employed. Those skilled in the art should understand that the scope of the invention as described in the embodiments of the present invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. An immersive ideological and political education scenario simulation experience system based on augmented reality, characterized in that, include: The scenario building module is used to generate and manage virtual ideological and political education scenarios with dynamic plots based on the themes of ideological and political education; The augmented reality rendering module is used to acquire information about the real environment, and to perform three-dimensional spatial fusion and rendering of the virtual objects in the virtual ideological and political education scenario with the information about the real environment to generate and output augmented reality images; The multimodal interaction and response module is used to collect various interactive inputs from users, perform semantic understanding and intent recognition on the interactive inputs, generate corresponding interactive instructions, and drive the virtual ideological and political education scenario to generate dynamic feedback based on the interactive instructions. The learning assessment and personalized adaptation module is used to generate quantitative learning assessment results based on multi-dimensional behavioral data of users during the experience process, and dynamically adjust the presentation content or difficulty parameters of the subsequent virtual ideological and political education scenarios based on the quantitative learning assessment results.

2. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 1, characterized in that, The scenario construction module includes: The dynamic parameterized scene generation unit is used to dynamically combine and generate a virtual scene containing interactive virtual objects from a 3D model database based on the ideological and political education theme and preset scene element parameters, wherein the scene element parameters include historical period feature parameters and environmental atmosphere parameters. An adjustable narrative network unit is used to construct a directed graph-based narrative logic network for ideological and political education. The narrative logic network includes multiple scenario nodes and plot branch paths connecting the scenario nodes. Each plot branch path is associated with a logical judgment condition. The adjustable narrative network unit activates the corresponding logical judgment condition based on user profile information or real-time interaction results to determine the current scenario progression path.

3. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 2, characterized in that, The augmented reality rendering module includes: The multi-source sensor fusion registration unit is used to simultaneously acquire real-world images collected by image sensors and motion data collected by inertial measurement units, and calculate high-precision user position, attitude, and three-dimensional structure information of the real-world environment based on the visual SLAM algorithm and the motion data, and generate a spatial registration matrix. The virtual-real light and shadow fusion and occlusion processing unit is used to place the virtual object in the three-dimensional structure of the real environment according to the spatial registration matrix, and calculate the light and shadow effect of the virtual object in real time based on the lighting estimation results of the real environment, while performing real-time occlusion relationship calculation and rendering between the virtual object and objects in the real environment. An adaptive rendering output unit is used to dynamically adjust the rendering resolution and frame rate of the augmented reality image based on the performance parameters of the augmented reality display device and the user's viewpoint movement speed, and output the final rendered image.

4. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 1, characterized in that, The multimodal interaction and response module includes: The multi-channel input sensing unit is used to simultaneously collect the user's gesture input, voice input, and physiological signal input obtained through biosensors, forming the original interactive data stream; An intent recognition and instruction generation unit is used to perform fusion analysis on the original interactive data stream. The fusion analysis includes: recognizing the operation type and target virtual object corresponding to the gesture input, recognizing keywords and command intent in the voice input, and analyzing the emotional state reflected by the physiological signal input; and generating structured interactive instructions based on the analysis results. The scenario dynamic response unit is used to call the physics engine to perform real-time physical simulation updates of the state of the virtual object according to the interaction instructions, or to drive the virtual characters in the virtual ideological and political education scenario to perform dialogue and behavioral feedback, and trigger the plot advancement in the adjustable narrative network unit.

5. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 1, characterized in that, The learning assessment and personalized adaptation module includes: A multi-dimensional behavioral data acquisition unit is used to continuously record the user's operation sequence, key selection nodes, task completion time, emotional tendency of voice content, and physiological indicator change curves monitored by biosensors during the experience process, as the multi-dimensional behavioral data. The multi-dimensional evaluation model calculation unit is used to calculate the user's quantitative indicators in the dimensions of knowledge cognition, value judgment, emotional identification and behavioral practice in parallel based on the multi-dimensional behavioral data. The personalized strategy adaptation unit is used to generate personalized scenario adjustment parameters for the current user based on the quantitative indicators output by the multi-dimensional evaluation model calculation unit according to the preset adaptation rule library, and to feed back the personalized scenario adjustment parameters to the scenario construction module and the augmented reality presentation module.

6. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 5, characterized in that, In the calculation unit of the multi-dimensional evaluation model, the formula for calculating the quantitative index of emotional identification dimension—emotional engagement E—is as follows: in, The attention focus index is calculated as the ratio of the time a user's gaze lingers on a key virtual object to the total time spent on it. The normalized physiological response fluctuation index, This represents the difference between the average heart rate and the baseline heart rate during the experience. Baseline heart rate; The positive sentiment index of voice is derived from the acoustic feature analysis of user voice. , , These are the weighting coefficients for the attention focus index, physiological response fluctuation index, and positive voice emotion index, respectively. ; The time decay coefficient, This represents the time difference between the current assessment moment and the critical trigger moment of the scenario.

7. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 6, characterized in that, The learning assessment and personalized adaptation module also includes a comprehensive assessment report generation unit, which is used for: Receive the quantitative indicators of each dimension output by the calculation unit of the multi-dimensional evaluation model; The comprehensive evaluation value S of the learning effect under the current ideological and political education theme is calculated using the following formula: in, The quantification value represents the i-th dimension, and N is the total number of dimensions; The dynamic weight of the i-th dimension at time t is adaptively adjusted based on the teaching objectives and user history of the ideological and political education theme. The semantic fit score between the user's choice at the key decision point j and the preset ideal answer; The decision-making influence coefficient; Based on the comprehensive evaluation value S of the learning effect and the quantitative indicators of each dimension, a quantitative learning evaluation result combining text and graphics is generated.

8. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 1, characterized in that, Also includes: The scenario generation algorithm module includes: The multimodal teaching resource parsing unit is used to receive and parse the input original materials for ideological and political education, including text cases, historical video materials, audio archives and image materials; the parsing includes: entity recognition, event extraction and sentiment analysis of text, key scene and character action recognition of video and image, speech-to-text conversion and sentiment tagging of audio, and generation of a structured multimodal ideological and political education knowledge graph; The narrative logic automatic construction unit is connected to the multimodal teaching resource parsing unit. It is used to learn the structural features and plot transformation patterns of high-quality historical narrative logic networks based on the multimodal ideological and political education knowledge graph and using a graph neural network model. It can also automatically generate candidate narrative logic networks and their corresponding plot-driven rule sets for new ideological and political education themes. The dynamic script and scene parameter generation unit is connected to the narrative logic automatic construction unit. It is used to call a pre-trained large language model based on the selected candidate narrative logic network to generate dynamic dialogue scripts and scene description texts that conform to the historical background and character settings, and output a set of virtual scene configuration parameters that match the dynamic dialogue scripts and scene description texts. The set of virtual scene configuration parameters is provided to the scene construction module.

9. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 1 or 8, characterized in that, Also includes: The cloud-based collaboration and persistence module includes: The distributed user state management unit is used to create and maintain an independent state container for each online user session. The state container stores the user's interaction context, the current snapshot of the virtual ideological and political education scenario, and the real-time evaluation intermediate data generated by the learning evaluation and personalized adaptation module in real time. The unit supports synchronizing the state change events of a specific user to other user terminals in the same collaborative scenario based on a publish-subscribe mechanism. The massive behavioral data analysis and model optimization unit is used to receive and persistently store all anonymized multi-dimensional behavioral data and corresponding quantitative learning evaluation results to form a historical training dataset. This unit periodically uses the historical training dataset to jointly train and optimize the evaluation model in the multi-dimensional evaluation model calculation unit, the adaptation rule base in the personalized strategy adaptation unit, and the graph neural network model and large language model in the scenario generation algorithm module, and distributes the optimized model parameters to each terminal module. The security and access control unit is used to manage user authentication, data access permissions, and access levels for different ideological and political education scenarios. This unit implements end-to-end encrypted data transmission and performs differential privacy protection processing on all stored personalized data.

10. The augmented reality-based immersive ideological and political education scenario simulation experience system according to claim 3 or 9, characterized in that, The adaptive rendering output unit is specifically used for: A rendering resource constraint model is constructed based on the type of augmented reality display device, remaining battery power, real-time computing load, and network bandwidth status. Under the rendering resource constraint model, with the goal of maintaining a preset minimum acceptable interactive frame rate, rendering resources are dynamically allocated through an optimization algorithm, the optimization including: For virtual objects located in the user's field of vision and strongly related to the current plot progression, high-precision models and real-time dynamic lighting and shadows are used for rendering. For virtual objects located at the edge of the field of view or in the secondary background, a level-of-detail model and static light and shadow maps are used for rendering; When high-speed movement of the user's viewpoint or system resource shortage is detected, the rendering resolution of non-critical virtual objects is automatically reduced and asynchronous time warp technology is enabled for image compensation. The images output by the adaptive rendering output unit are compatible with a variety of augmented reality display devices, including optical see-through head-mounted displays, video see-through head-mounted displays, and mobile smart terminals with depth sensing capabilities.

Images