Artificial intelligence simulation interactive teaching method and system based on holographic data

Abstract

The invention provides an artificial intelligence simulation interactive teaching method and system based on holographic data. The method comprises the following steps: acquiring a multi-modal interactive data stream; constructing a behavior causal graph according to the multi-modal interaction data stream and the spatial region set; according to the behavior causal graph and all the interaction behavior nodes, generating an anti-fact path possibly existing in the teaching behavior of the student, and constructing a feedback fragment script of the anti-fact path; acquiring position data and action events of each frame of the student, constructing a basic space access counting graph through an indicator function to obtain an interactive memory score of the student, and performing measurement analysis on the interactive memory score to obtain a space memory score; generating a feedback strategy set and a node feedback priority ranking list; and mapping the feedback strategy set and the node feedback priority ranking list into a holographic feedback behavior, and executing the holographic feedback behavior in the actual teaching process of the student to guide the student to complete correct cognitive correction and operation repair.

Description

Technical Field

[0001] This invention belongs to the field of holographic data, and in particular relates to an artificial intelligence simulation interactive teaching method and system based on holographic data. Background Technology

[0002] With the rapid evolution of technologies such as holographic display, spatial interaction, and artificial intelligence, educational scenarios are undergoing a shift from two-dimensional information transmission to three-dimensional immersive cognition. Holographic data-based teaching systems can visualize and structure abstract knowledge, presenting it to learners through spatial mapping, enabling exploratory learning via multimodal channels such as vision, movement, and speech. Especially in applications such as engineering practice, medical skills, and scientific experiments, holographic teaching technology can provide a level of simulation and immersion that traditional textbooks and videos cannot replicate, significantly enhancing the perceptibility and operability of knowledge. However, current holographic teaching systems still have key technological shortcomings in integrating artificial intelligence to achieve "intelligent simulation-interactive teaching." On the one hand, most systems only support static content display or simple interactions based on preset scripts, lacking the ability to understand individual student behavior paths and failing to provide truly explanatory feedback for errors. On the other hand, existing AI teaching systems are mostly built on two-dimensional interfaces or shallow sensor data, lacking effective modeling of exploratory behavior, attention distribution, and operational strategies in three-dimensional space, and especially unable to identify areas missed or blind spots in student cognition within space.

[0003] Furthermore, current teaching feedback mechanisms are primarily outcome-oriented, often presenting prompts such as "Is it completed?" or "Is it correct?" They fail to demonstrate the causal relationship between student behavior and learning objectives, and lack the ability to dynamically guide students in optimization and exploration. These problems severely restrict the role of AI in holographic teaching scenarios, preventing the system from fully adapting to the different cognitive paths and learning biases of students, and limiting the realization of key values ​​such as personalized teaching, intelligent feedback, and adaptive learning paths. Summary of the Invention

[0004] The purpose of this invention is to design an artificial intelligence simulation interactive teaching method and system based on holographic data, which can adaptively simulate the teaching process, guide cognitive transfer, and dynamically adjust the pace of content, truly realizing a three-in-one teaching experience of intelligence, immersion, and interaction, and significantly improving students' learning efficiency and depth of understanding.

[0005] To achieve the above objectives, a first aspect of the present invention provides an artificial intelligence simulation interactive teaching method based on holographic data, the method comprising:

[0006] The system acquires spatial location data, action events, gaze trajectory, and speech data of the target location, and preprocesses them to generate a multimodal interactive data stream. Each frame contains behavior encoding, gaze region number, and speech emotion vector. The gaze trajectory is projected onto a predefined set of spatial regions to obtain the gaze region number.

[0007] A behavior causal graph is constructed based on the multimodal interaction data stream and the spatial region set, so that the behavior causal graph generates corresponding interaction behavior nodes and edges based on the multimodal interaction data stream. The interaction behavior nodes represent teaching behavior events, and the edges represent causal relationships between teaching behavior events. The interaction behavior nodes of the behavior causal graph are constructed by the multimodal interaction data stream based on the interaction event window.

[0008] Based on the behavioral cause-effect graph and all interactive behavior nodes, generate possible counterfactual paths for students in teaching behavior, and construct feedback fragment scripts for the counterfactual paths;

[0009] The system acquires the student's position data and action events for each frame, and combines these with the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set. A basic spatial access counting map is constructed using an indicator function to obtain the student's interactive memory score. Based on the measurement and analysis of the interactive memory score, a spatial memory score is obtained to construct a spatial memory map representing the student's spatial memory score for each region in holographic teaching. If the current spatial memory score is less than a preset threshold, the current region is classified into the attention blind spot set.

[0010] By combining the spatial memory map, the set of attentional blind spots, the counterfactual path, and the feedback fragment script, a fusion model of feedback strategies is established to generate a set of feedback strategies and a list of node feedback priority rankings.

[0011] The feedback strategy set and node feedback priority sorting list are mapped to holographic feedback behaviors, and the holographic feedback behaviors are executed in the actual teaching process of students to guide students to complete the correct cognitive correction and operational repair.

[0012] Furthermore, the acquisition of spatial location data, action events, gaze trajectory, and voice data of the target location, and the preprocessing thereof to generate a multimodal interactive data stream, specifically includes:

[0013] Acquire raw data streams, including spatial location data, action events, gaze trajectories, and voice data;

[0014] A resampling strategy combining sliding time windows and linear interpolation is adopted to unify the spatial location data and action events to a fixed time step of 20Hz, resulting in synchronized spatial location data sequences and action event sequences, which are the three-dimensional coordinates and operation vectors at each time step, respectively. The synchronized spatial location data sequences and action event sequences are then subjected to modal fusion processing to obtain behavior codes.

[0015] The gaze trajectory is projected onto a predefined set of spatial regions, and the gaze direction and spatial projection result are mapped to gaze region numbers. Missing values ​​are filled by the maximum value of the most recent gaze times.

[0016] The speech data is input into an acoustic feature encoder to obtain a speech emotion vector;

[0017] Based on each time step t, the behavior encoding, gaze region number, and voice emotion vector are concatenated to form a complete multimodal interactive data stream.

[0018] Further, the step of constructing a behavior causal graph based on the multimodal interaction data stream and the spatial region set, so that the behavior causal graph generates corresponding interaction behavior nodes and edges based on the multimodal interaction data stream, specifically includes:

[0019] The multimodal interactive data stream sliding window is divided into interactive event windows of length ΔT = 2s, wherein each interactive event window is abstracted as an interactive behavior node, and the node set of the behavior cause-effect graph is constructed based on the interactive behavior nodes.

[0020] Calculate the causal connection strength between any two interactive behavior nodes, retain all edges with causal connection strength greater than a preset threshold, and construct a behavior causal graph, where the edges represent the causal relationship between the behavior events corresponding to the two interactive behavior nodes.

[0021] Furthermore, the causal connection strength is further enhanced by introducing a regularization term based on spatial regions to simulate the difference in causal weights between continuous operation within a teaching area and cross-regional jumps.

[0022] Furthermore, the step of generating counterfactual paths that students may exhibit in their teaching behavior based on the behavioral cause-effect graph and all interactive behavior nodes, and constructing feedback fragment scripts for the counterfactual paths, specifically includes:

[0023] A depth-first search is performed on the behavior causal graph to obtain a set of all candidate paths from the start node to the end node, and the semantic matching degree of each path in the candidate path set is calculated with a preset reference path to obtain a semantic matching degree score.

[0024] The path with the lowest semantic matching score is selected as the current actual path, and the current actual path is considered to have a deviation.

[0025] A deviation point is selected in the current actual path, and an initial counterfactual path is generated. The counterfactual path is searched again in the graph. Starting from the deviation point, a subsequent path with the highest similarity to a local segment of the preset reference path is selected as a replacement segment to form the final counterfactual path.

[0026] For each interactive behavior node in the counterfactual path, find its corresponding behavior code in the multimodal interactive data stream, and map the behavior code to replay animation frames and voice prompt labels to construct a feedback script fragment.

[0027] Furthermore, the step of selecting a subsequent path with the highest similarity to a local segment of the preset reference path from the deviation point as the replacement segment specifically involves:

[0028] Among all downstream nodes of the deviation point, select the path that forms the longest continuous high-match segment and replace the original path from the starting point to the deviation point with it.

[0029] Furthermore, the acquisition of student position data and action events for each frame, combined with the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set, constructs a basic spatial access counting map through an indicator function to obtain the student's interactive memory score. Based on the measurement and analysis of the interactive memory score, a spatial memory score is obtained to construct the spatial memory map. Specifically, this includes:

[0030] Traverse the interactive data sequence t=1 to T. When the gaze region is numbered i, the current region represents the current gaze region. Each occurrence is regarded as an access record and counted as an interactive memory score, which is used to comprehensively measure the gaze and operation.

[0031] The interactive memory score is regularized by introducing a semantic adjacency matrix between regions to indicate that two regions have physical or task continuity in order to obtain a spatial memory score. A spatial memory map is then constructed based on the spatial memory score to represent the student's spatial memory score for each region in holographic teaching.

[0032] Specifically, when the spatial memory score is less than a preset threshold, the current area is classified into the blind spot set. After comparing it with the set of areas to be covered corresponding to the counterfactual path, if there is an intersection between the current blind spot set and the corresponding set of areas to be covered, it is considered that the current teaching behavior has a serious spatial coverage deficiency, and students will be guided to revisit these areas with high priority in the subsequent feedback strategy.

[0033] Furthermore, the step of combining the spatial memory map, the set of attentional blind spots, counterfactual paths, and feedback fragment scripts to establish a fusion model of feedback strategies to generate a set of feedback strategies and a priority ranking list of node feedback specifically includes:

[0034] All deviation points in the counterfactual path are traversed, and a feedback requirement score is calculated for each deviation point to reflect the degree to which the deviation point should be fed back in the feedback strategy.

[0035] The deviation points are sorted according to the feedback requirement score for each deviation point, and the feedback level ranges are divided.

[0036] The feedback level range is bound to the corresponding feedback fragment script to form a structured feedback strategy unit, and a node feedback priority sorting list is constructed.

[0037] All structured feedback strategy units are grouped together to form a feedback strategy set.

[0038] Furthermore, the mapping of the feedback strategy set and the node feedback priority sorting list to holographic feedback behavior, and the execution of the holographic feedback behavior during the actual teaching process to guide students to complete correct cognitive correction and operational repair, specifically includes:

[0039] Based on the order of the node feedback priority sorting list, feedback actions are executed sequentially starting from the structured feedback strategy unit with the highest priority; and after each feedback action, it is detected in real time whether the current student behavior has achieved the goal.

[0040] If the feedback objective is detected as completed, proceed to the next high-priority policy unit; if it is not completed, the feedback will be repeated automatically.

[0041] Once all feedback strategies have been executed, the teaching session is marked as over.

[0042] A second aspect of the present invention provides an artificial intelligence simulation interactive teaching system based on holographic data, the system comprising:

[0043] The holographic interactive data acquisition unit is used to acquire spatial location data, action events, gaze trajectory, and voice data of the target location, and preprocess them to generate a multimodal interactive data stream, wherein each frame contains behavior encoding, gaze region number, and voice emotion vector; wherein the gaze trajectory is projected onto a predefined set of spatial regions to obtain the gaze region number;

[0044] The causal structure graph generation unit is used to construct a behavioral causal graph based on the multimodal interaction data stream and the spatial region set, so that the behavioral causal graph generates corresponding interactive behavior nodes and edges based on the multimodal interaction data stream. The interactive behavior nodes represent teaching behavior events, and the edges represent causal relationships between teaching behavior events. The interactive behavior nodes of the behavioral causal graph are constructed by the multimodal interaction data stream based on the interactive event window.

[0045] The counterfactual path generation unit is used to generate counterfactual paths that students may have in teaching behavior based on the behavior causal graph and all interactive behavior nodes, and to construct feedback fragment scripts for the counterfactual paths.

[0046] The spatial memory modeling unit is used to acquire the student's position data and action events for each frame. Combined with the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set, a basic spatial access counting map is constructed through an indicator function to obtain the student's interactive memory score. Based on the measurement and analysis of the interactive memory score, a spatial memory score is obtained to construct a spatial memory map, representing the student's spatial memory score for each region in holographic teaching. If the current spatial memory score is less than a preset threshold, the current region is classified into the attention blind spot set.

[0047] The feedback strategy generation unit is used to combine the spatial memory map, the set of attention blind spots, the counterfactual path, and the feedback fragment script to establish a fusion model of feedback strategies, so as to generate a set of feedback strategies and a node feedback priority ranking list.

[0048] The teaching guidance execution unit is used to map the feedback strategy set and the node feedback priority sorting list into holographic feedback behaviors, and execute the holographic feedback behaviors during the actual teaching process of students to guide students to complete the correct cognitive correction and operational repair.

[0049] The beneficial technical effects of the present invention are at least as follows:

[0050] To address the aforementioned problems, this invention provides an AI simulation teaching method and system for holographic interactive teaching scenarios. Its core lies in constructing an intelligent mechanism capable of simultaneously perceiving, understanding, and providing feedback on students' behavioral paths and spatial exploration states. During the learning process, the system can collect and analyze holographic interactive data in real time, establishing a structured relationship model between student actions and learning objectives. This allows for the identification of potential error paths and misunderstandings of causal relationships, and the generation of personalized feedback in a dynamic manner to guide students in reconstructing their learning paths. Simultaneously, the system continuously models students' interaction trajectories and attentional area distribution in three-dimensional space, identifying key knowledge nodes that were not fully addressed or overlooked during the cognitive process, and generating a spatially hierarchical guidance mechanism based on this. Unlike traditional feedback methods based on static evaluation and result judgment, this invention emphasizes the integrated feedback capability of process interpretability and spatial guidance, achieving a leap from post-evaluation to a real-time simulation teaching model.

[0051] Through the above mechanisms, the system can adaptively simulate the teaching process, guide cognitive transfer, and dynamically adjust the pace of content, truly achieving a three-in-one teaching experience of intelligence, immersion, and interaction, and significantly improving students' learning efficiency and depth of understanding. Attached Figure Description

[0052] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0053] Figure 1 This is a flowchart of the artificial intelligence simulation interactive teaching method based on holographic data according to the present invention.

[0054] Figure 2 This is a framework diagram of the artificial intelligence simulation interactive teaching system based on holographic data according to the present invention. Detailed Implementation

[0055] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0056] In one or more embodiments, such as Figure 1 As shown, an artificial intelligence simulation interactive teaching method based on holographic data is disclosed, the method comprising the following:

[0057] S1. Acquire spatial location data, action events, gaze trajectory, and voice data of the target location, and preprocess them to generate a multimodal interactive data stream, wherein each frame contains behavior encoding, gaze region number, and voice emotion vector; wherein the gaze trajectory is projected onto a predefined set of spatial regions to obtain the gaze region number.

[0058] Specifically, this step aims to uniformly collect, align, and encode various heterogeneous data from the interaction between students and the holographic teaching environment, forming a structured temporal data stream. This provides a stable and standardized input foundation for subsequent causal and spatial modeling. Considering that this patent involves high-dimensional multimodal interactive behavior in real three-dimensional space, the collected data must possess temporal synchronization, spatial continuity, and semantic representation. The innovation of this step lies in designing a complete data acquisition and preprocessing workflow. This workflow can simultaneously capture and fuse multiple behavioral signals such as vision, action, and speech, ultimately forming a unified multimodal behavioral vector sequence.

[0059] The specific inputs for this step include:

[0060] Spatial location data is jointly acquired by a holographic camera array (such as Azure Kinect or OptiTrack system) and a head-mounted inertial positioning module (such as HTC ViveTracker), with a sampling frequency of approximately 30–60 Hz, to obtain the three-dimensional coordinate path of the student in the holographic teaching space.

[0061] Action events are collected by the desktop interactive console (equipped with physical buttons and knobs) and the virtual touch interactive panel, recording the type and trigger time of each operation (such as grab, rotate, click to confirm).

[0062] The gaze trajectory is obtained by a head-mounted eye-tracking module (such as Pupil LabsVR or TobiiProGlasses), providing the student's gaze direction vector and the position of the landing point in the virtual space, which is used to analyze the attention area.

[0063] Voice data is collected through a wearable microphone array and processed by a voice framing and emotion feature extraction module to generate continuous voice emotion codes.

[0064] Take a typical scenario as an example: A student is assembling a virtual engine in a holographic simulation experiment. When he clicks on a component using the console, the system simultaneously records its spatial coordinates, operation type, whether the gaze point covers the component, and whether the speech is accompanied by questions or self-talk.

[0065] The first issue to address is the inconsistency in time steps of multimodal data. In the original data stream, spatial location P raw (t) and action event A rawThe data (t) is sampled at 30Hz and 10Hz respectively, while speech and gaze data are recorded at the millisecond level. To ensure consistency in subsequent modeling, the system adopts a resampling strategy combining sliding time windows and linear interpolation, unifying the time step to a fixed 20Hz, to obtain the synchronized spatial location data sequence P(t) and action event sequence A(t), which are the three-dimensional coordinates and operation vectors at each time step, respectively.

[0066] Modal fusion is then performed. P(t) and A(t) are jointly fed into an action-position fusion subnetwork, which is a two-layer fully connected structure: the first layer has an input dimension of 3 (coordinates) + N (one-hot encoding of operation type) and outputs a 64-dimensional representation; the second layer compresses this to a 32-dimensional behavior representation. The fusion function is denoted as:

[0067] E1(t)=f e (P(t),A(t)) (1)

[0068] Where P(t) is the three-dimensional spatial position vector at time t, generated by the three-dimensional positioning module; A(t) is the action type vector in one-hot encoded form, representing the operation event performed by the student at the current time step; f e It is a set of fixed-structure feedforward neural networks (two fully connected layers, with dimensions [64→32], and the activation function is ReLU). The network is trained with parameters using a self-supervised task based on operation label prediction during the system pre-training phase.

[0069] Eye Track G raw (t) is projected onto the system's predefined spatial region set Z = {Z1, Z2, ..., Z} n Each area, such as the dashboard, engine component A, and energy control console, is a spatial block defined during the instructional design phase. The system maps the gaze direction and spatial projection results to the gaze area number G(t) at the current time step, and missing values ​​are filled with the maximum value of the most recent gaze count.

[0070] The speech data V(t) is generated by an acoustic feature encoder. The acoustic module extracts parameters including pitch, speech rate, speech energy, and formants, which are then compressed into a 32-dimensional embedding vector by a sentiment classifier consisting of two stacked bidirectional LSTMs, representing the emotional state of the current time segment (e.g., neutral, confused, positive).

[0071] Finally, at each time step t, the encoding results of the three sub-modalities are concatenated to form a complete multimodal interaction vector D(t), the structure of which is as follows:

[0072] D(t)=[E1(t),G(t),V(t)] (2)

[0073] Here, E1(t) is the action-position fusion vector (32-dimensional); G(t) is the student's gaze region number, a discrete integer value corresponding to a region in the region set Z; V(t) is the continuous vector representation of speech emotion (32-dimensional), generated by the acoustic emotion recognition network. This data is stored in a sliding window format, with each window being 5 seconds long (i.e., 100 frames), for subsequent model reading at time steps.

[0074] All numerical features were normalized before training to adapt to the requirements of subsequent neural network modeling.

[0075] Furthermore, this step outputs the following two core variables: D(t): a structured multimodal interaction data stream, used for subsequent causal graph modeling and spatial attention modeling; Z: a ​​pre-defined set of spatial region divisions in the teaching system, used to represent the spatial mapping of student gaze behavior, and used in subsequent steps to build a spatial memory map.

[0076] S2. Construct a behavior causal graph based on the multimodal interaction data stream and the spatial region set, so that the behavior causal graph generates corresponding interactive behavior nodes and edges based on the multimodal interaction data stream. The interactive behavior nodes represent teaching behavior events, and the edges represent causal relationships between teaching behavior events. The interactive behavior nodes of the behavior causal graph are constructed by the multimodal interaction data stream based on the interactive event window.

[0077] Specifically, this step aims to construct a causal graph structure of student behavior during the holographic interactive teaching process, based on the structured interactive data D(t) and spatial region set Z output in the previous step. This structure will serve as the foundation for subsequent feedback path generation and error interpretation, providing core support for the interpretability of intelligent feedback in the patented solution. This step, combining the multimodal characteristics of spatial location, action behavior, visual gaze, and vocal state in holographic teaching, proposes an event causal modeling mechanism suitable for 3D immersive interactive scenarios. Compared to traditional methods that only use operation sequences or result judgments, this step can construct a causal graph driven by spatiotemporal behavioral chains, attention transfer, and linguistic emotion, possessing significant structural expressive power and cognitive interpretability.

[0078] Furthermore, firstly, the system divides the time series D(t) into interactive event windows of length ΔT = 2s, with each window abstracted as a teaching event node. For the center time point t of each window... i The present invention constructs the representation vector h of this node. i :

[0079] h i =[E1(t) i ),one_hot(G(t i )),V(ti (3)

[0080] Among them, E1(t) i ): The 32-dimensional action-position code output by the fusion neural network in step 1, containing the operation type at that moment and its positional relationship in three-dimensional space; one_hot(G(t) i ): The one-hot encoding of the gaze region ID on the region set Z, with dimension |Z|; V(t) i ): Voice emotion embedding, a 32-dimensional vector output by a bidirectional LSTM model, representing the current emotional state of the student.

[0081] These nodes h i This forms the node set of the interaction behavior graph. Then, this invention designs a causal edge construction mechanism with spatial region similarity regulation to mine the temporal-spatial dependencies between operational behaviors. For any two nodes h... i with h j (where t) i <t j ), calculate the causal connection strength between them:

[0082] s ij =σ(w T ·[h i ||h j ||(h i -h j ) 2 ]-λ·δ(G(t i ),G(t j (4)

[0083] Where: [·||·] represents vector concatenation; (h i -h j ) 2 Represents the Euclidean feature differences between nodes; w is a parameter vector, representing the score weights for learning the causal directionality between nodes; σ(·) is the Sigmoid function; δ(G(t) i ),G(t j )) represents the semantic distance between spatial regions, which is 0 when the two gaze regions belong to the same physics teaching module, and 1 otherwise; λ is a regularization factor that enhances the edge-building penalty for cross-region switching events.

[0084] Understandably, the innovation of the above formula lies in:

[0085] In addition to traditional feature difference measures, a regulation term δ(G(t) for spatial structure consistency is also introduced. i ),G(t j This simulates the difference in causal weights between students continuously operating within a teaching area and switching between different areas.

[0086] This mechanism avoids high-frequency edge building between different semantic regions, thereby strengthening the ontological causal chain of the behavior aggregation block and helping to focus on key behavior sequences in subsequent counterfactual path generation.

[0087] Next, all those that satisfy s ij Directed edges with a value of θ (suggested threshold of 0.6) are preserved to construct the final causal graph. Where V = {h i Let} be the node set, and E be the causal edge set. Each edge in the graph represents that behavior event i causes behavior event j to occur, subject to the triple constraints of temporal sequence, semantic consistency of operations, and spatial similarity.

[0088] This structure also introduces a self-checking mechanism: if the number of outgoing and incoming edges of certain nodes is lower than the threshold (e.g., <2), then the node is considered to be a noise point in student behavior or a behavior that has not achieved the teaching goal. The system can record it as a potential feedback item for subsequent teaching prompts.

[0089] The final output includes the following two variables: A behavioral causal graph, structured as a directed graph, with edge weights representing causal strength; {h i}: A collection of interactive behavior nodes used for subsequent path reconstruction and error visualization.

[0090] This step is designed to fully incorporate the practical teaching scenario requirements of the patent, including: 1) Spatial region division Z plays a structural regulatory role in causal mapping, achieving dual modeling of space and behavior; 2) Vocal emotion embedding V(t) is no longer passive information but participates in causal intensity modeling, establishing a measurable connection between emotion changes and behavioral pattern changes; 3) The addition of a regularized control term based on spatial structure significantly optimizes the rationality of the teaching logic in constructing causal edges. This causal modeling mechanism enables subsequent counterfactual feedback not only to be based on the inversion of the operational sequence but also to reflect the dynamic path of operational logic and attention shift, which is the structural foundation for realizing intelligent explanatory teaching feedback in the patent system.

[0091] S3. Based on the behavior causal graph and all interactive behavior nodes, generate counterfactual paths that students may have in teaching behavior, and construct feedback fragment scripts for the counterfactual paths.

[0092] Specifically, the core task of this step is to: based on the behavioral cause-effect graph output in the previous step... Set of interactive event nodes for teaching {h i}, generating counterfactual paths P that students may take in their teaching behavior. cf And construct the visual feedback fragment F of this path. cf .

[0093] In holographic teaching, students' behavioral paths are typically goal-oriented, starting with an initial operation and progressing through several interactive behaviors to reach the endpoint of the teaching task. For example, in a virtual engine assembly task, the goal is to complete the assembly process. The system predefines a reference behavioral path P. ref The process includes picking up the component, placing it in the designated location, fixing the bolts, and then conducting an electrical test.

[0094] The system first uses a cause-effect graph. In the middle, perform a depth-first search to obtain a set of all candidate paths from the start node to the end node. And compare each path with the reference path P ref Perform semantic matching degree calculation:

[0095]

[0096] in, It is path P i The event vector of the j-th node in the vector; r j It is the preset vector of the j-th node in the reference path (generated by manual annotation by teaching experts or system task model); cos(·,·) represents cosine similarity, which measures the similarity of event behaviors in semantic space; the output value ∈ [0,1], the higher the value, the closer the path is to the teaching objective.

[0097] This invention selects Match(P) i ,P ref The lowest path is taken as the current actual path P. act This path is considered to have a deviation. Subsequently, a deviation point h is selected on this path. k (Even at the position with the largest deviation along the entire path), and generate a new counterfactual path P. cf The path is searched again from the graph, starting from h. k Start by selecting a path that connects to P ref Subsequent paths with higher local segment similarity are used as replacement segments.

[0098] This replacement is implemented using a dynamic path switching strategy. (In h) k Among all downstream nodes, select the path P that forms the longest consecutive high-matching segment. alt and replace P with it. act [k:] segment. This ultimately forms a counterfactual path:

[0099] P cf =P act [:k-1]||P alt (6)

[0100] Among them, P act[:k-1] represents the original path from the starting point to the node preceding the deviation point; P alt For the improved segment obtained by re-searching from the graph structure, it satisfies the condition of P. ref The structure achieves maximum local similarity and employs a conservative modification strategy that replaces only erroneous segments, thus maintaining the interpretability of student behavior.

[0101] In order to make this counterfactual path available for subsequent interactive feedback, the system also provides each h i ∈P cf Find the corresponding action records (i.e., A(t) and P(t)) in the original data stream D(t), map them to replay animation frames and voice prompt labels, and construct the feedback script fragment F. cf This includes:

[0102] Action name, operation location (provided by P(t); gaze point region (mapped to Z by G(t); if there is a change in speech, insert a virtual teacher voice prompt script.

[0103] Furthermore, there are two output variables: P cf : Structured counterfactual behavior path, only the path structure is included, without action execution; F cf : A replayable feedback fragment script for use by the subsequent rendering module.

[0104] S4. Obtain the student's position data and action events for each frame. Combine the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set. Construct a basic spatial access counting map using an indicator function to obtain the student's interactive memory score. Analyze the interactive memory score to obtain a spatial memory score, and construct a spatial memory map to represent the student's spatial memory score for each region in holographic teaching. If the current spatial memory score is less than a preset threshold, the current region is assigned to the attention blind spot set.

[0105] Specifically, this step plays a crucial role in the entire patent: it constructs a spatial memory map based on student interaction behavior and feedback paths. Used to systematically identify spatial areas (i.e., attentional blind spots Z) that students have not fully encountered or understood in holographic interactive teaching. missed This allows for spatial guidance and cognitive repair of subsequent feedback strategies.

[0106] The input for this step comes from the previous step and the structure output of the preceding steps, specifically including:

[0107] P cf The counterfactual path output in step 3 contains the recommended sequence of behaviors to be performed for the teaching task, with each node corresponding to an operation target area;

[0108] Fcf : Corresponding to P cf The feedback demonstration script can be mapped to the teaching objects that students should focus on;

[0109] G(t): gaze trajectory data from step 1, recording the spatial region index corresponding to the student's gaze point in each frame;

[0110] Z = {Z1, Z2, ..., Z} n}: The system's predefined set of teaching space areas, derived from step 1, defines the spatial location range of the teaching objects;

[0111] P(t) and A(t): The student's operation position and action type in each frame, encoded as E1(t) by step 1.

[0112] Understandably, in actual teaching scenarios, students may have completed a certain degree of the operational process (see Step 2 Modeling and Step 3 Counterfactual Correction), but there are still several key teaching areas that have not been observed, operated, or even approached. These omissions may directly lead to deviations in teaching objectives. Therefore, the goal of this step is to construct a cognitive accessibility map based on spatial regions as basic units and compare it with the standard attention areas in the counterfactual path to identify those blind spots that should have been addressed but were not covered.

[0113] First, a basic spatial access counting map is constructed. The interaction data sequence t = 1 to T is traversed; when G(t) = i, it indicates that the current gaze region is Z. i Each occurrence is considered an access record and is counted in V. i :

[0114]

[0115] in, This is an indicator function that indicates whether the gaze in the current frame is directed towards region Z. i ; This indicates whether the student's operation occurred within the region, determining whether P(t) falls within the region boundary; γ is the operation enhancement factor, used to increase the importance of actual operation behavior to spatial memory (generally set to 2-5); V i That is, region Z i The interactive memory score comprehensively measures attention and operation.

[0116] Furthermore, to enhance the model's perception of the semantic structure of instruction, this invention introduces an inter-region semantic adjacency matrix. Where R ij =1 indicates region Z i With Z jIt possesses physical or task continuity. For example, the tool area and the target equipment area belong to the same spatial unit under the same operational process, and their connectivity should be reflected in the fact that even if they are never observed but are adjacent, they can still be included in the mild blind spot warning. Therefore, this invention defines a regularized spatial memory score M. i :

[0117]

[0118] The first item is the normalized access ratio, representing region Z. i The proportion of total student attention occupied; the second term is the cross-regional attention difference regularization term, if Z i Frequent access to adjacent areas and Z i If ignored, this item increases, thus lowering the final score M. i λ is the regularization adjustment factor, which is generally set to 0.1 to 0.3. This regularization mechanism enables the model to have the ability to predict structural consistency, that is, if a certain region should be of concern but the actual deviation is too large, it should be identified.

[0119] Furthermore, when M i When θ (a set threshold, recommended to be 0.03–0.05) is less than 0.05, the system will target region Z. i Included in the blind zone set Z missed With P cf The corresponding set of areas to be covered, Z cf After comparison, if If the system determines that there is a serious lack of spatial coverage in the current teaching practices, it will guide students to revisit these areas with high priority in subsequent feedback strategies.

[0120] Output the following two variables: This indicates that students in holographic teaching are familiar with each region Z. i Spatial memory scores can be viewed as regional cognitive maps; Z missed Note that the blind spot set is one of the core inputs for feedback strategy fusion.

[0121] S5. Combining the spatial memory map, attention blind spot set, counterfactual path, and feedback fragment script, establish a fusion model of feedback strategies to generate a set of feedback strategies and a node feedback priority ranking list.

[0122] Specifically, the task of this step is to utilize the structured output obtained in the previous stage—the spatial memory map. With attention blind spot set Z missed (From step 4) and counterfactual path P cf and feedback fragment F cf(Based on step 3), a fusion model of the feedback strategy is established. This step does not directly present feedback, nor is it responsible for generating script content or animation. Instead, it establishes a feedback strategy modeling mechanism that can automatically integrate the two dimensions of behavioral deviation and spatial omission. That is, it completes the modeling and generation of the feedback strategy structure for subsequent guidance of execution (step 6).

[0123] The essence of this step is modeling: using the data structure constructed in the previous steps, a scoring function and strategic decision-making logic are built, the feedback content is structured, and different feedback response levels are assigned to different areas or behaviors (such as whether replay is needed, whether voice prompts are needed, etc.). This strategy model should be configurable, logically transparent, and capable of structured output.

[0124] Input for this step:

[0125] P cf Counterfactual behavior path: Indicates the suggested alternative action path;

[0126] F cf Each path node corresponds to a scripted feedback segment, including prompts and operation information;

[0127] Each teaching space area Z i Attention scores are derived from students' actual interaction performance;

[0128] Z missed The set of spatial regions that students did not pay sufficient attention to during the task.

[0129] First, the system analyzes the counterfactual path P. cf All deviation points h k (Generated from step 3) Iterate through the data and calculate a feedback requirement score w for each deviation point. k This score reflects the degree to which the node should receive feedback in the strategy:

[0130]

[0131] Among them, z k Represents node h k Corresponding teaching space area This information comes from F. cf Original motion record; This indicates the student's attention rating in this area, derived from... This is a blind spot indicator function; when the area where the node is located is a blind spot, it is assigned a value of 1. α and β are adjustable coefficients; the recommended initial values ​​are α = 0.7 and β = 0.3, used to balance scoring quantification and explicit region bias. kThe larger the value, the more emphasis should be placed on that node in the feedback strategy.

[0132] Furthermore, the system is configured according to each h k w k The values ​​are sorted among the nodes, and then the feedback level intervals are divided. For example, the rating w k Nodes with a value >0.8 receive high-priority feedback and are handled using a strategy of path highlighting, animation, and voice prompts; Nodes with a value >0.5 <w k Nodes with a value ≤0.8 are marked with path playback and action text annotations; w k Nodes with a value ≤0.5 will only display auxiliary information upon user request. This mapping is statically stored in the form of a policy template table for easy rule maintenance.

[0133] Next, the system will assign the feedback requirement level of each node to its script content F. cf [k] is bound together to form a structured feedback strategy unit r. k Its structure includes: h k : Feedback target node; w k Feedback weight; F cf [k]: Feedback content; type k Feedback strategy templates (such as voice + highlighting, animation + prompts, etc.); triggers k Triggering conditions (such as whether interaction is required to trigger, whether continuous feedback is required, etc.).

[0134] Finally, all policy units are combined to form a feedback policy set:

[0135] R feedback ={r k |h k ∈P cf ,w k >θ} (10)

[0136] Where θ is the lower threshold for feedback triggering, to avoid wasting resources on low-value feedback (θ = 0.3 is recommended).

[0137] In strategy modeling, all structure generation does not rely on ad-hoc judgments, but rather constructs a clear logical chain through structured scoring functions, feedback level mapping, and template binding. For example:

[0138] Assume P cf The region corresponding to the middle node h4 is Z7, and this region is... The median score is M7 = 0.12, and Z7 ∈ Z. missed ,but:

[0139] w4=0.7·(1-0.12)+0.3·1=0.916

[0140] Because w4 > 0.8, the system sets the strategy for this node to voice guidance + action replay + highlighted area;

[0141] and F cf The animation path in [4] is bound to r4 and marked as a high-priority feedback strategy.

[0142] This step outputs two variables:

[0143] R feedback : Structured feedback policy set, each policy unit r k Includes behavioral nodes, ratings, content, and feedback methods;

[0144] S priority : Node feedback priority sorting list, used for feedback scheduling and multi-channel synchronization control in the next step.

[0145] S6. Map the feedback strategy set and node feedback priority sorting list to holographic feedback behavior, and execute the holographic feedback behavior in the actual teaching process of students to guide students to complete the correct cognitive correction and operational repair.

[0146] Specifically, this step, as the endpoint of the entire system process, involves converting the generated feedback strategy R... feedback and feedback priority sequence S priority The feedback is mapped to real, visible, and interactive holographic feedback behaviors, and these feedback actions are executed during the actual teaching process to guide students to complete correct cognitive correction and operational repair.

[0147] Furthermore, the system first relies on S priority The sorting starts from the highest priority r. k Begin executing the feedback actions sequentially. For each r... k The system identifies its corresponding teaching space area Z. k Based on the operational objectives, by searching a pre-loaded holographic resource library, the system extracts the necessary voice prompts, motion replay frames, character animation scripts, and spatial highlight styles for the strategy, and immediately renders this feedback into the interactive scene within the student's field of vision. For example:

[0148] If r k If the path playback + voice guidance is specified, the standard process of the operation corresponding to the node will be played automatically in the holographic scene where the student is located, and a voice prompt will be played to ask him to re-check the connection order of component B.

[0149] If r k Corresponding Z k In Z missed During the event, the system will highlight the area with virtual lighting and shadows, or summon and guide characters to the area;

[0150] If the trigger condition is set to wait for the behavior to complete, the system will wait until the student completes the expected operation (i.e., executes the action with r). k Bound Action A k and located at P(t)∈Z k After that, it will automatically jump to the next feedback step.

[0151] To ensure that feedback is understood and responded to, the system monitors in real time whether the current student behavior has achieved the target after each feedback is executed. This process is no longer a modeling function, but rather an actual operational status monitoring. The system samples the current action A(t) and spatial position P(t) every second and compares them with r. k The specified feedback target is compared. For example, in the target region Z. k Has A been completed? k The corresponding operational instructions are analyzed, and whether there are subsequent vocal emotion signals indicating that the student understands (such as the speech rate returning to normal or the tone changing from doubt to affirmation) to determine whether to skip, repeat, or reinforce the current feedback.

[0152] If the feedback objective is detected as completed, the system immediately jumps to the next high-priority strategy unit; if it is not completed, the system will automatically repeat the feedback once, or activate a stronger level of prompt (e.g., upgrading from voice to role-playing demonstration). The entire process is integrated into the student's operation, with minimal interference and high perceptual adaptability, ensuring that the feedback is real-time, targeted, and guiding.

[0153] After all feedback strategies have been executed, the system records information such as whether each feedback was accepted and whether it was responded to correctly, and marks it as the end of the teaching session.

[0154] In one or more embodiments, such as Figure 2 As shown, an artificial intelligence simulation interactive teaching system based on holographic data is disclosed, the system comprising:

[0155] The holographic interactive data acquisition unit 101 is used to acquire spatial location data, action events, gaze trajectory and voice data of the target location, and preprocess them to generate a multimodal interactive data stream, wherein each frame contains behavior encoding, gaze region number and voice emotion vector; wherein the gaze trajectory is projected onto a predefined set of spatial regions to obtain the gaze region number;

[0156] The causal structure graph generation unit 102 is used to construct a behavioral causal graph based on the multimodal interaction data stream and the spatial region set, so that the behavioral causal graph generates corresponding interactive behavior nodes and edges based on the multimodal interaction data stream. The interactive behavior nodes represent teaching behavior events, and the edges represent causal relationships between teaching behavior events. The interactive behavior nodes of the behavioral causal graph are constructed by the multimodal interaction data stream based on the interactive event window.

[0157] The counterfactual path generation unit 103 is used to generate counterfactual paths that students may have in teaching behavior based on the behavior cause-effect graph and all interactive behavior nodes, and to construct feedback fragment scripts for the counterfactual paths.

[0158] The spatial memory modeling unit 104 is used to acquire the student's position data and action events for each frame. Combining the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set, it constructs a basic spatial access counting map through an indicator function to obtain the student's interactive memory score. Based on the measurement and analysis of the interactive memory score, a spatial memory score is obtained to construct a spatial memory map, representing the student's spatial memory score for each region in holographic teaching. If the current spatial memory score is less than a preset threshold, the current region is classified into the attention blind spot set.

[0159] The feedback strategy generation unit 105 is used to combine the spatial memory map, the set of attention blind spots, the counterfactual path and the feedback fragment script to establish a fusion model of feedback strategies, so as to generate a set of feedback strategies and a node feedback priority ranking list.

[0160] The teaching guidance execution unit 106 is used to map the feedback strategy set and the node feedback priority sorting list into holographic feedback behaviors, and execute the holographic feedback behaviors during the actual teaching process of students to guide students to complete the correct cognitive correction and operational repair.

[0161] It is worth noting that the specific workflow of the AI ​​simulation interactive teaching system based on holographic data provided in this embodiment of the invention is the same as that of the AI ​​simulation interactive teaching method based on holographic data described in the above embodiments, and will not be repeated here.

[0162] This invention also provides an artificial intelligence simulation interactive teaching device based on holographic data, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps described in the above embodiments of the artificial intelligence simulation interactive teaching method based on holographic data. Figure 1 The steps S1 to S6 described above; or, when the processor executes the computer program, it implements the functions of each module in the above system embodiments.

[0163] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the holographic data-based artificial intelligence simulation interactive teaching device.

[0164] The holographic data-based AI simulation interactive teaching device can be a desktop computer, laptop, handheld computer, or cloud server, among other computing devices. This device may include, but is not limited to, processors and memory. Those skilled in the art will understand that the holographic data-based AI simulation interactive teaching device may also include input / output devices, network access devices, buses, etc.

[0165] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the holographic data-based artificial intelligence simulation interactive teaching device, connecting all parts of the device via various interfaces and lines.

[0166] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the holographic data-based artificial intelligence simulation interactive teaching device by running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the operation of the air conditioner controller, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart memory card (SMC), secure digital card (SD), flash memory card, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0167] The modules integrated into the holographic data-based AI simulation interactive teaching equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0168] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0169] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An AI simulation-based interactive teaching method based on holographic data, characterized in that, The method includes: The system acquires spatial location data, action events, gaze trajectory, and speech data of the target location, and preprocesses them to generate a multimodal interactive data stream. Each frame contains behavior encoding, gaze region number, and speech emotion vector. The gaze trajectory is projected onto a predefined set of spatial regions to obtain the gaze region number. A behavior causal graph is constructed based on the multimodal interaction data stream and the spatial region set, so that the behavior causal graph generates corresponding interaction behavior nodes and edges based on the multimodal interaction data stream. The interaction behavior nodes represent teaching behavior events, and the edges represent causal relationships between teaching behavior events. The interaction behavior nodes of the behavior causal graph are constructed by the multimodal interaction data stream based on the interaction event window. Based on the behavioral cause-effect graph and all interactive behavior nodes, generate possible counterfactual paths for students in teaching behavior, and construct feedback fragment scripts for the counterfactual paths; The system acquires the student's position data and action events for each frame, and combines these with the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set. A basic spatial access counting map is constructed using an indicator function to obtain the student's interactive memory score. Based on the measurement and analysis of the interactive memory score, a spatial memory score is obtained to construct a spatial memory map representing the student's spatial memory score for each region in holographic teaching. If the current spatial memory score is less than a preset threshold, the current region is classified into the attention blind spot set. By combining the spatial memory map, the set of attentional blind spots, the counterfactual path, and the feedback fragment script, a fusion model of feedback strategies is established to generate a set of feedback strategies and a list of node feedback priority rankings. The feedback strategy set and node feedback priority sorting list are mapped to holographic feedback behaviors, and the holographic feedback behaviors are executed in the actual teaching process of students to guide students to complete the correct cognitive correction and operational repair.

2. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 1, characterized in that, The acquisition of spatial location data, action events, gaze trajectory, and voice data of the target location, and the preprocessing thereof to generate a multimodal interactive data stream, specifically includes: Acquire raw data streams, including spatial location data, action events, gaze trajectories, and voice data; A resampling strategy combining sliding time windows and linear interpolation is adopted to unify the spatial location data and action events to a fixed time step of 20Hz, resulting in synchronized spatial location data sequences and action event sequences, which are the three-dimensional coordinates and operation vectors at each time step, respectively. The synchronized spatial location data sequences and action event sequences are then subjected to modal fusion processing to obtain behavior codes. The gaze trajectory is projected onto a predefined set of spatial regions, and the gaze direction and spatial projection result are mapped to gaze region numbers. Missing values ​​are filled by the maximum value of the most recent gaze times. The speech data is input into an acoustic feature encoder to obtain a speech emotion vector; Based on each time step t, the behavior encoding, gaze region number, and voice emotion vector are concatenated to form a complete multimodal interactive data stream.

3. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 1, characterized in that, The step of constructing a behavior causal graph based on the multimodal interaction data stream and the spatial region set, so that the behavior causal graph generates corresponding interaction behavior nodes and edges based on the multimodal interaction data stream, specifically includes: The multimodal interactive data stream sliding window is divided into interactive event windows of length ΔT = 2s, wherein each interactive event window is abstracted as an interactive behavior node, and the node set of the behavior cause-effect graph is constructed based on the interactive behavior nodes. Calculate the causal connection strength between any two interactive behavior nodes, retain all edges with causal connection strength greater than a preset threshold, and construct a behavior causal graph, where the edges represent the causal relationship between the behavior events corresponding to the two interactive behavior nodes.

4. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 3, characterized in that, The causal connection strength also incorporates a regularization term based on spatial regions to simulate the difference in causal weights between continuous operation within a teaching area and cross-regional jumps.

5. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 1, characterized in that, The step of generating counterfactual paths that students may exhibit in teaching behavior based on the behavioral cause-effect graph and all interactive behavior nodes, and constructing feedback fragment scripts for the counterfactual paths, specifically includes: A depth-first search is performed on the behavior causal graph to obtain a set of all candidate paths from the start node to the end node, and the semantic matching degree of each path in the candidate path set is calculated with a preset reference path to obtain a semantic matching degree score. The path with the lowest semantic matching score is selected as the current actual path, and the current actual path is considered to have a deviation. A deviation point is selected in the current actual path, and an initial counterfactual path is generated. The counterfactual path is searched again in the graph. Starting from the deviation point, a subsequent path with the highest similarity to a local segment of the preset reference path is selected as a replacement segment to form the final counterfactual path. For each interactive behavior node in the counterfactual path, find its corresponding behavior code in the multimodal interactive data stream, and map the behavior code to replay animation frames and voice prompt labels to construct a feedback script fragment.

6. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 5, characterized in that, The step of selecting a subsequent path with the highest similarity to a local segment of the preset reference path from the deviation point as the replacement segment specifically involves: Among all downstream nodes of the deviation point, select the path that forms the longest continuous high-match segment and replace the original path from the starting point to the deviation point with it.

7. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 1, characterized in that, The process involves acquiring the student's position data and action events for each frame, combining this with the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set, and constructing a basic spatial access counting map using an indicator function to obtain the student's interactive memory score. Based on the measurement and analysis of this interactive memory score, a spatial memory score is obtained to construct the spatial memory map. Specifically, this includes: Traverse the interactive data sequence t=1 to T. When the gaze region is numbered i, the current region represents the current gaze region. Each occurrence is regarded as an access record and counted as an interactive memory score, which is used to comprehensively measure the gaze and operation. The interactive memory score is regularized by introducing a semantic adjacency matrix between regions to indicate that two regions have physical or task continuity in order to obtain a spatial memory score. A spatial memory map is then constructed based on the spatial memory score to represent the student's spatial memory score for each region in holographic teaching. Specifically, when the spatial memory score is less than a preset threshold, the current area is classified into the blind spot set. After comparing it with the set of areas to be covered corresponding to the counterfactual path, if there is an intersection between the current blind spot set and the corresponding set of areas to be covered, it is considered that the current teaching behavior has a serious spatial coverage deficiency, and students will be guided to revisit these areas with high priority in the subsequent feedback strategy.

8. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 1, characterized in that, The process involves combining the spatial memory map, attentional blind spot set, counterfactual paths, and feedback fragment scripts to establish a fusion model of feedback strategies, generating a set of feedback strategies and a priority ranking list for node feedback. Specifically, this includes: All deviation points in the counterfactual path are traversed, and a feedback requirement score is calculated for each deviation point to reflect the degree to which the deviation point should be fed back in the feedback strategy. The deviation points are sorted according to the feedback requirement score for each deviation point, and the feedback level ranges are divided. The feedback level range is bound to the corresponding feedback fragment script to form a structured feedback strategy unit, and a node feedback priority sorting list is constructed. All structured feedback strategy units are grouped together to form a feedback strategy set.

9. The artificial intelligence simulation interactive teaching method based on holographic data according to claim 8, characterized in that, The process of mapping the feedback strategy set and node feedback priority sorting list to holographic feedback behavior, and executing the holographic feedback behavior during actual teaching to guide students to complete correct cognitive correction and operational repair, specifically includes: Based on the order of the node feedback priority sorting list, feedback actions are executed sequentially starting from the structured feedback strategy unit with the highest priority; and after each feedback action, it is detected in real time whether the current student behavior has achieved the goal. If the feedback objective is detected as completed, proceed to the next high-priority policy unit; if it is not completed, the feedback will be repeated automatically. Once all feedback strategies have been executed, the teaching session is marked as over.

10. An AI simulation interactive teaching system based on holographic data, characterized in that: The system includes: The holographic interactive data acquisition unit is used to acquire spatial location data, action events, gaze trajectory, and voice data of the target location, and preprocess them to generate a multimodal interactive data stream, wherein each frame contains behavior encoding, gaze region number, and voice emotion vector; wherein the gaze trajectory is projected onto a predefined set of spatial regions to obtain the gaze region number; The causal structure graph generation unit is used to construct a behavioral causal graph based on the multimodal interaction data stream and the spatial region set, so that the behavioral causal graph generates corresponding interactive behavior nodes and edges based on the multimodal interaction data stream. The interactive behavior nodes represent teaching behavior events, and the edges represent causal relationships between teaching behavior events. The interactive behavior nodes of the behavioral causal graph are constructed by the multimodal interaction data stream based on the interactive event window. The counterfactual path generation unit is used to generate counterfactual paths that students may have in teaching behavior based on the behavior causal graph and all interactive behavior nodes, and to construct feedback fragment scripts for the counterfactual paths. The spatial memory modeling unit is used to acquire the student's position data and action events for each frame. Combined with the counterfactual path, feedback fragment script, gaze trajectory, and spatial region set, a basic spatial access counting map is constructed through an indicator function to obtain the student's interactive memory score. Based on the measurement and analysis of the interactive memory score, a spatial memory score is obtained to construct a spatial memory map, representing the student's spatial memory score for each region in holographic teaching. If the current spatial memory score is less than a preset threshold, the current region is classified into the attention blind spot set. The feedback strategy generation unit is used to combine the spatial memory map, the set of attention blind spots, the counterfactual path, and the feedback fragment script to establish a fusion model of feedback strategies, so as to generate a set of feedback strategies and a node feedback priority ranking list. The teaching guidance execution unit is used to map the feedback strategy set and the node feedback priority sorting list into holographic feedback behaviors, and execute the holographic feedback behaviors during the actual teaching process of students to guide students to complete the correct cognitive correction and operational repair.

Images